Skip to content Skip to navigation

CMS: Proof lacking that quality measures have unintended effects

March 3, 2015
by Lois A. Bowers, Senior Editor
| Reprints

The Centers for Medicare & Medicaid Services (CMS) says it has found “little empirical evidence to support or refute the possibility that undesired effects have occurred as a result of the use of quality measures” in nursing homes, hospitals and ambulatory settings. The remark was part of the agency’s 2015 Impact Assessment of Quality Measures Report [PDF], released Mach 2.

To reach its conclusion, CMS conducted a literature review that retrieved 9,584 citations but ultimately included only 33 of them. Studies reviewed mentioned CMS quality measures, included information about study design and had study designs that were judged to be not of poor quality, examined unintended consequences and were written in English. Post-acute care programs included in the reviewed research included the Nursing Home Quality Initiative, the Home Health Quality Reporting Program, the End-Stage Renal Disease Quality Incentive Program, the Hospice Quality Reporting Program, the Inpatient Rehabilitation Facilities Quality Reporting Program and the Long-Term Care Hospitals Quality Reporting Program.

The review addressed five types of undesired effects:

  1. Worsening quality in unmeasured areas (“teaching-to-the-test”),
  2. Providing overtreatment or unnecessary care,
  3. Reporting inaccurately high performance (“gaming of the data”),
  4. Avoiding high-risk or challenging patients (“cherry-picking”) and
  5. Worsening disparities in care.

“In nursing homes, there is insufficient evidence regarding teaching-to-the-test (zero out of two), cherry-picking (zero out of one) and gaming (zero out of one),” the report notes. Additional research will be needed to measure the effects of quality measures, CMS says, “particularly as CMS evolves its measure programs to incorporate outcome measures and increases the financial risks for providers with poor performance.” CMS also says it will try to minimize unintended consequences through measure design and selection as well as the use of exclusions.

The study team used the findings of this review to help develop provider surveys that CMS will conduct as part of its 2018 Impact Report, CMS says, adding that surveys will generate national estimates on the type and prevalence of unintended consequences associated with the use of CMS quality measures.