September 6, 2016
Rankings and Reactions: CMS Launches Star Ratings for Hospitals
“The fault, dear Brutus, is not in our stars
But in ourselves.” [1] William Shakespeare
On July 27th, the Centers for Medicare and Medicaid Services (CMS) released its first-ever quality ratings for hospitals. Within the week, U.S. News and World Report released its much-followed 27th annual hospital rankings. The results could not have differed more.
Of U.S. News’ 20 “Honor Roll” hospitals, only the Mayo Clinic received 5 stars. Four received only 2 stars. These contradictory results reflect differing rating methodologies and illustrate the challenge of giving healthcare consumers actionable data for making healthcare decisions.
Healthcare providers have long fought against external performance assessments. It’s a losing battle. Amid much teeth-gnashing, hospitals are adapting to market demands for transparency. Expect the CMS hospital ratings to gain currency and shape demand for hospital services.
Giving Consumers the Hospital Ratings They Want
Desperate for guidance, consumers increasingly rely on provider ratings and report cards to select hospitals and doctors. “About three-quarters of Americans with Internet access have searched for health or medical information online.”[2]
Despite strong public support for more accessible quality data, the hospital industry fought the release of CMS’ new “consumer-friendly” star ratings. In response to industry concerns, CMS delayed their release by three months and conducted significant stakeholder outreach. This did little to dampen industry ire.
Since their release, industry representatives have lampooned the CMS ratings, noting the poor performance of teaching and safety net hospitals. In a blog accompanying the new ratings, CMS Quality Director Kate Goodrich acknowledged industry concerns while heralding the benefits of transparency to American healthcare consumers,
[CMS’ Hospital Compare website will] help millions of patients and their families learn about the quality of hospitals, compare facilities in their area side-by-side, and ask important questions about care quality when visiting a hospital or other health care provider.
Given the wide variation in hospital quality, outcomes and costs, external report cards help consumers make better choices. Highlighting this point, Dr. Goodrich notes, “researchers found that hospitals with more stars on the Hospital Compare website have tended to have lower death and readmission rates.”
Likewise, a National Bureau of Economic Research study found that hospitals with higher quality ratings for specific medical conditions, including heart attacks, pneumonia and congestive heart failure, disproportionately gain market share. Market forces work. When given choices, consumers select higher-performing hospitals.
Public demand for better hospital ratings is accelerating. Multiple independent sources, including U.S. News and CMS, have emerged to meet this need, but the conclusions they draw are contradictory and confusing.
The Confusing World of Hospital Ratings
All hospital ratings contain inherent biases. A widely-read Health Affairs study analyzed overlap between “high” and “low” performers in the following four 2012/2013 national hospital ratings: U.S. News; HealthGrades; The Leapfrog Group; and Consumer Reports. The results were stunning in their inconsistency:
No hospital was rated as a high performer by all four national rating systems. Only 10 percent of the 844 hospitals rated as a high performer by one rating system were rated as a high performer by any of the other rating systems.
The authors attributed the lack of overlap to different rating methodologies, different areas of focus and different criteria weighting. Their unsurprising conclusions are few hospitals excel across all service dimensions and a lack of performance standardization retards hospital performance improvement.
The Affordable Care Act has addressed the lack of hospital quality, cost and performance transparency through more intense data collection (currently 113 metrics) and distribution. Issuing star ratings based on collected data is a logical next step.
CMS’ star ratings for individual hospitals emerge from statistical analysis of 64 metrics across 7 categories: mortality; safety; readmissions; patient experience; effectiveness; timeliness; and imaging. The first 4 categories carry 22% weights and the last 3 carry 4% weights. CMS will update and publish its quality ratings quarterly. Individual hospital performance summaries, like this report for the Mayo Clinic, are clear, detailed and easily comparable.
While controversial, CMS’ star ratings are entirely data-driven. They took over two years to develop, leaned heavily on a prestigious technical expert pane and invited widespread public input. Their analytic methodology was statistically rigorous in selecting and assigning weights to metrics and performance categories. Their methodology employed clustering techniques to homogenize performance within targeted categories and establish a bell-curve distribution among the almost 4,000 rated hospitals.
By contrast, U.S. News targets specialty care and relies heavily (27.5% weighting) on reputational/opinion surveys by board-certified specialists. U.S. News also weights 30-day Medicare survival rates (37.5%); patient safety (5%) and other care-related factors such as nurse-staffing ratios (30%). All 20 of U.S. News’ “Honor Roll” hospitals are academic medical centers. Their methodology does not incorporate patient experience.
The following chart details the U.S. News “Honor Roll” hospitals and their concurrent CMS star rating:
The dispersion of star ratings among these “Honor Roll” hospitals is striking. While CMS’s rating methodology may not be perfect, it is data-driven and consistent. The inability of the nation’s most-recognized hospitals to all excel on CMS’ quality metrics testifies to unacceptable variation in hospital outcomes, quality, costs and customer experience.
Industry Reaction
Blustering is a professional sport in Washington. Too many healthcare leaders scorn the CMS’ methodology because they don’t like its results.
Dr. Darrell Kirch is the president and CEO of the Association of American Medical Colleges. In a Modern Healthcare commentary titled “CMS’ new star ratings are unfair to teaching and safety net hospitals,” Dr. Kirch castigates CMS’ ratings in hyper-charged language,
Instead of providing useful information, the new ratings paint a confusing and conflicting picture of the quality of U.S. hospital care because of a deeply flawed methodology that ignores important differences in the patient populations and the complexity of conditions that different types of hospitals treat.
These types of broad-based critiques are disingenuous. It would be equally fair to state that U.S. News’ intense focus on specialty care and dependence on specialist opinion surveys biases their rankings against well-run community hospitals. What else explains that all 20 “Honor Roll” hospitals are academic medical centers?
It’s worth noting that highly-rated U.S. News hospitals spend gargantuan sums advertising their success. The same will be true for CMS’ 5-star hospitals. During my recent Lake Michigan beach vacation, I saw several billboards where 5-Star Holland Community Hospital proclaimed itself the best hospital in Michigan.
Back to Dr. Kirch’s commentary. In fairness, he makes the following valid points regarding CMS’ methodology:
- that reporting more metrics lowers some hospitals’ scores;
- that socio-economic status likely influences care outcomes; and
- that current ratings may not accurately reflect the benefits of specialization.
CMS can and will adjust its methodology to address these concerns.
To drive his “unfairness” point home, Dr. Kirch notes that “only one major teaching hospital received five stars [and] nearly 90% rated three stars or below.” He then asks, “How is this possible?”
Ironically, that is exactly the question lower-rated hospitals should ask themselves. Despite the challenges Dr. Kirch outlined, Mayo Clinic, Cleveland Clinic, Mass General, New York Presbyterian and four other “Honor Roll” hospitals achieved 4 and 5-star ratings. Lower-rated hospitals should investigate what these institutions are doing right and replicate their high-level performance.
The beauty and wonder of market-driven reform is that companies respond to “signals” from customers to win their business. Enlightened hospitals engage in constant and consistent performance improvement.
“Yelping” It: Ratings Speak to Customers
In another intriguing Health Affairs study, researchers employed natural language processing to evaluate Yelp hospital ratings to HCAHPS (Hospital Consumer Assessment of Healthcare Providers and Systems) quality scores.
They found that hospitals receiving 3 Yelp ratings had almost identical ratings to CMS’ HCAHPS scores. They also found Yelp’s hospital reviews were more expansive, covering 12 domains (i.e. staff compassion, billing experience, nursing quality, amenities, etc.) beyond the 32 covered by HCAHPS.
Most intriguing, these researchers concluded that Yelp “may provide a more nuanced view [than HCAPHS] of aspects of hospital quality that patients value.” Their conclusions reinforce the ancient business wisdom of listening to, understand, and responding to customer needs.
Like rock ‘n roll, hospital ratings are here to stay. CMS has had positive experience with its star ratings for nursing homes, Medicare Advantage insurance plans and dialysis centers. Consumers find these ratings helpful and use them to make better healthcare purchasing decisions. CMS plans to develop star ratings for physicians. It’s adapt or die time for hospitals.
Hospitals that acclimate to healthcare consumerism and improve performance, particularly in patient engagement, will win consumer trust and remain viable market participants. Paraphrasing Shakespeare, “the stars” will reward hospitals that recognize “the faults in themselves” and work to correct them. Outcomes matter. Customers count. Value Rules!
[1] William Shakespeare[1] (Julius Caesar, Act I, Scene ii, lines 140-41)
[2] Steve Findlay, Health Affairs, April 2016, “Consumers’ Interest In Provider Ratings Grows, And Improved Report Cards And Other Steps Could Accelerate Their Use”, page 2
About the Author
Recent Posts
