Colleges and Hospitals
In the February 21, 2011 issue of The New Yorker, Malcom Gladwell, the author of The Tipping Point, takes a close look at US News & World Report's rating system for colleges and hospitals.
What he finds is that there is essentially no meaning to the ratings. The problem he exposes is that it is very hard to compare two very complex items that are meant to satisfy a complex list of needs.
He starts his analysis by looking at a similar rating system, Car and Driver's ranking of cars. He looks at their ranking of sports cars and finds that depending on what you like, the Porsche Cayman, Chevy Corvette, and Lotus Evora would rank 1,2,3, or 3,2,1, or 2,3,1, all depending on what you were interested in getting. If you use the same rating system for all cars you get one ranking, but if you adapt a ranking system to sports cars that elevates the value of sporty look and a fun ride, you get the opposite ranking. Ranking by cost gives you a third ranking altogether.
Just so for colleges.
Gladwell then uses the example of ranking countries by the incidence of suicide. That can be done and in that system each country gets one number to describe its rank. But the problem is that nations vary in the acceptability of suicide in their culture. Even in the US, where most deaths from isolated car injuries (one car, one driver, no known physical cause) are thought to be suicides, most such deaths are not listed as suicide.
Just so for colleges.
As Gladwell puts it, "There's no direct way to measure the quality of an institution- how well a college manages to inform, inspire, and challenge its students." So, instead of measuring the unmeasurable but important, the rating scales of US News & World Report simply measure the less important, that they hope reflects the core purpose of college. For example, 20% of US News & World Report rankings are based on faculty data, such as how much are the faculty paid. But good studies find no correlation at all between a professor's salary and his/her teaching excellence. The same goes for the reputation variable in the US News & World Report's rankings. National university presidents have to rank about 250 other schools from one to 250 on reputation to help US News & World Report devise this rank variable, it makes no sense.
Hospitals
Now we come to hospitals. Here Gladwell cites an important study from one of the world's top medical journals, the Annals of Internal Medicine. The study looked at the US News & World Report ranking of top US hospitals, and looked at objective measures of hospital excellence such as the chance of a person dying from a standard surgery, patient safety records, key technology availability. They found there was no connection at all between US News & World Report's reputation rating and the actual excellence in practice of a hospital.
The Absurdity of the US News & World Report College and Hospital Ranking System
Gladwell goes on to cite the fact a Michigan Supreme Court Chief Justice sent a ranking questionnaire to 100 of his colleagues. Penn State at that time was ranked by these elite members of the legal community as the fifth best law school in the United States. But at that time Penn State did not even have a law school!
The only variable that the US News & World Report college ranking firmly correlates with is how much money the school has. Wealthy schools alway outrank less wealthy schools.
BOTTOM LINE
When it comes to complex needs, such as those we require from colleges and hospitals, put the magazines away. Their rankings can only serve one purpose- to sell the magazine.
For colleges, think about what your child needs and is interested in and seek the college that provides that unique set of needs best for her or him.
For hospitals, keep in mind that no hospital actually solves a medical problem, only a person or a small group of people- your doctor, your specialist, your nurse. So one should only use the medical system with a clear eye towards what the questions are, what the needs are, at that moment, and be sure your doctor can organize the best team possible to help get those needs met.
Dr. Arthur Lavin
*Disclaimer* The comments contained in this electronic source of information do not constitute and are not designed to imply that they constitute any form of individual medical advice. The information provided is purely for informational purposes only and not relevant to any person's particular medical condition or situation. If you have any medical concerns about yourself or your family please contact your physician immediately. In order to provide our patients the best uninfluenced information that science has to offer,we do not accept samples of drugs, advertising tchotchkes, money, food, or any item from outside vendors.
No comments:
Post a Comment