Skip to content Skip to navigation

Behind the 2009 ACHCA Leadership Award

April 1, 2009
by Eli Pick and John Sheridan
| Reprints
The award process raised questions about limitations of Five-Star data

Administrators who have distinguished themselves by meeting or exceeding defined, stringent performance standards will be recognized at The American College of Health Care Administrators' (ACHCA) 43rd Annual Convocation and Exposition “Leadership First: Achieving Quality Results,” May 15 through 19, in Providence, Rhode Island. (See sidebar, page 31)

The 2009 ACHCA Leadership Award conferred this year uses the Centers for Medicare & Medicaid Services (CMS) facility data and information provided to the public for the year 2008. The Quality Measures (QMs) for pain and restraints were specifically selected for this award because they have a minimum number of variables and are not impacted by risk adjustment or pre-selection bias in an organization's admission process. They were selected based on their utility as performance indicators rather than as population profiles.

Selection of QMs also took into account the need to control for organizations that specialize in high-risk populations, which may have disproportionate risk for complications compared with organizations whose populations have moderate to low risk of complications, thus skewing QM performance.

Striking comparisons

In developing the data for this year's award, we found some striking comparisons with the new Five-Star Quality Rating System initiated this year by CMS. Our findings raised questions about the limitations of the Five-Star data and the information being purveyed to consumers selecting nursing homes for their loved ones.

The Five-Star rating system is CMS's attempt to simplify and synthesize findings of a highly complex survey system, staffing level data, and select QMs and distill them into a hotel- or restaurant-type rating. In contrast to Five-Star, the ACHCA Leadership Award includes designated QM findings, health and fire safety survey deficiencies, occupancy levels, and population profiles. The ACHCA rating system identifies and recognizes the organizational leadership that achieves the top meaningful scores in these areas.

Table 1 shows a comparison of ACHCA and Five-Star rankings for organizations that qualified for the 2009 ACHCA Leadership Award.

Overall Star Rating

Facility Count


1 out of 5 stars



2 out of 5 stars



3 out of 5 stars



4 out of 5 stars



5 out of 5 stars






The across-the-board distribution of these facilities, with nearly half at three stars or fewer, indicates no positive relationship between the two rating systems. The metrics of the Five-Star ranking discount the significant achievements of the 216 organizations that ranked three stars or below, presenting them as average, below average, and well below average, respectively, despite their achievements documented in accessible public data. While the ACHCA Leadership Award does not portend to be an overall representation of the organization's performance, it does illustrate the shortcoming of a rating system that excludes significant achievements that the Leadership Award criteria reveal.

Facility data were obtained for ACHCA on February 27, 2009, from the Medicare download database for the QMs, Survey Inspections, and Five-Star Rating. eHealth Data Solutions, a long-term care data and information company based in Cleveland, Ohio, analyzed national, publicly available MDS data from CMS and rank ordered it within each state. In developing these data, QM tables were created in Microsoft Access and Excel using the QMs in the Nursing Home Compare database and these were then ranked. Facilities achieving the highest QM rank in their states were further matched for occupancy, administrator tenure, and deficiency citations on their annual survey, as well as any complaint surveys during the reporting period of 2008.

Table 2 shows the metrics for this prestigious award. Note that the inclusion of fire safety data goes beyond Five-Star data, which do not take this important function into account.

MDS data on four specific quality measures:

Long Stay High Risk Residents with Pressure Sores 0% -10% (U.S. average -12%)

Long Stay Pain 0% - 2% (U.S. average - 4%)

Long Stay Restraints 0% - 3% (U.S. average - 4%)

Short Stay Pain 0% - 9% (U.S. average - 20%)

Health and Fire Safety Survey data including both scope and severity:

Fewer than five deficiency citations for the combined Health and Fire Safety Surveys (U.S. average is nine)

Scope of E or lower

Occupancy data

2008 /2009 occupancy is 90% or higher (U.S. average was 83.5% in 2006, National Center for Health Statistics

Stability of organizational administrative leadership