View More View Less
  • 1 Semmelweis University Health Services Management Training Center POB 610 H-1528 Budapest Hungary
  • | 2 National Health Insurance Fund Budapest Hungary
Restricted access

One of the hottest topics of the international journals is the question: what is the effect of the public indicator-based quality assessment on the quality of care and on the decision of stakeholders, and which criteria should be applied for development of public quality assessment system. According to the international literature, the paper discusses 6 topics: (1) the ability of indicators to distinguish providers from the point of view of quality; (2) the appropriateness of outcome indicators to assess providers; (3) the ability of league tables to rank providers; (4) the people’s behaviour during choosing providers; (5) the impact of indicator-based public report; (6) recommendations for developing quality assessment system. Based on the literature review, the ability of indicators in distinguishing providers from the point of view of quality is doubtful primarily because of risk-adjustment problems. Other reasons are: the outcomes of care do not definitely refer to the quality of care process; the rankings of providers (league tables) based on more indicators are not reliable; people take into account mainly distance and the opinions of acquaintance when they choose providers; as a result of public reports the overall quality of care is declining. The publication of the results of measurement to assess providers has to be considered as a tool. For the purpose of helping people in choosing providers, the publication of patient satisfaction survey designed according to their preferences could achieve the desired effect. The quality improvement aims are definitely helped by the direct feedback to providers about the indicator values. Furthermore, a much finer picture can be made if the standardized audits of care and organisational processes are inserted into external assessment procedures.

  • Steinbrook, R.: Public Report Cards – Cardiac Surgery and beyond. N. Engl. J. Med., 2006, 355 , 1847–1849.

    Steinbrook R. , 'Public Report Cards – Cardiac Surgery and beyond ' (2006 ) 355 N. Engl. J. Med. : 1847 -1849.

    • Search Google Scholar
  • Mannion, R., Goddard, M.: Public disclosure of comparative clinical performance data: lessons from the Scottish experience. J. Eval. Clin. Pract., 2003, 9 , 277–286.

    Goddard M. , 'Public disclosure of comparative clinical performance data: lessons from the Scottish experience ' (2003 ) 9 J. Eval. Clin. Pract. : 277 -286.

    • Search Google Scholar
  • Mason, A. M. A., Street, A.: Publishing outcome data: is it an effective approach? J. Eval. Clin. Prac., 2006, 12 , 37–48.

    Street A. , 'Publishing outcome data: is it an effective approach? ' (2006 ) 12 J. Eval. Clin. Prac. : 37 -48.

    • Search Google Scholar
  • Adab, P., Rouse, A. M., Mohammed, A. M. et al.: Performance league tables: the NHS deserves better. BMJ, 2002, 324 , 95–98.

    Mohammed A. M. , 'Performance league tables: the NHS deserves better ' (2002 ) 324 BMJ : 95 -98.

    • Search Google Scholar
  • Takács, E., Belicza, É., Boncz, I.: Lessons learned from the NHS indicator system. Egészségügyi Menedzsment, 2003, 5/6 , 54–59. (In Hungarian)

    Boncz I. , 'Lessons learned from the NHS indicator system ' (2003 ) 5 Egészségügyi Menedzsment : 54 -59.

    • Search Google Scholar
  • Vallet, G., Perrin, A., Keller, C. et al.: Public access to information about the services and quality of care in public hospitals: the need for methodologic clarity. A survey of 44 university hospital directors and administrators. Presse Med., 2006, 35 , 388–392.

    Keller C. , 'Public access to information about the services and quality of care in public hospitals: the need for methodologic clarity. A survey of 44 university hospital directors and administrators ' (2006 ) 35 Presse Med. : 388 -392.

    • Search Google Scholar
  • Shahian, D. M., Normand, S. L., Torchiana, D. F. et al.: Cardiac surgery report cards: comprehensive review and statistical critique. Ann. Thorac. Surg., 2001, 72 , 2155–2168.

    Torchiana D. F. , 'Cardiac surgery report cards: comprehensive review and statistical critique ' (2001 ) 72 Ann. Thorac. Surg. : 2155 -2168.

    • Search Google Scholar
  • Werner, R. M., Asch, D. A.: The unintended consequences of publicly reporting quality information. JAMA, 2005, 293 , 1239–1244.

    Asch D. A. , 'The unintended consequences of publicly reporting quality information ' (2005 ) 293 JAMA : 1239 -1244.

    • Search Google Scholar
  • Smith, P. C., York, N.: Quality incentives: the case of UK general practitioners. Health Aff., 2004, 23 , 112–118.

    York N. , 'Quality incentives: the case of UK general practitioners ' (2004 ) 23 Health Aff. : 112 -118.

    • Search Google Scholar
  • Belicza, É., Takács, E., Boncz, I.: Indicator system development for evaluating the quality of health services. Orv. Hetil., 2004, 145 , 1567–1572. (In Hungarian)

    Boncz I. , 'Indicator system development for evaluating the quality of health services ' (2004 ) 145 Orv. Hetil. : 1567 -1572.

    • Search Google Scholar
  • Scott, I. A., Ward, M.: Public reporting of hospital outcomes based on administrative data: risks and opportunities. Med. J. Aust., 2006, 184 , 571–575.

    Ward M. , 'Public reporting of hospital outcomes based on administrative data: risks and opportunities ' (2006 ) 184 Med. J. Aust. : 571 -575.

    • Search Google Scholar
  • Jacobson, B., Mindell, J., McKee, M.: Hospital mortality league tables. Question what they tell you – and how useful they are. BMJ, 2003, 326 , 777–778.

    McKee M. , 'Hospital mortality league tables. Question what they tell you – and how useful they are ' (2003 ) 326 BMJ : 777 -778.

    • Search Google Scholar
  • Lilford, R., Mohammed, M. A., Spiegelhalter, D. et al.: Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma. Lancet, 2004, 363 , 1147–1154.

    Spiegelhalter D. , 'Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma ' (2004 ) 363 Lancet : 1147 -1154.

    • Search Google Scholar
  • Shiu, M. F.: Use of language should be more careful in describing league tables. BMJ, 2002, 324 , 542–542.

    Shiu M. F. , 'Use of language should be more careful in describing league tables ' (2002 ) 324 BMJ : 542 -542.

    • Search Google Scholar
  • Glance, L. G., Osler, T. M., Dick, A. W.: Evaluating trauma center quality: Does the choice of the severity-adjustment model make a difference? J. Trauma, 2005, 58 , 1265–1271.

    Dick A. W. , 'Evaluating trauma center quality: Does the choice of the severity-adjustment model make a difference? ' (2005 ) 58 J. Trauma : 1265 -1271.

    • Search Google Scholar
  • Glance, L. G., Dick, A., Osler, T. M. et al.: Impact of changing the statistical methodology on hospital and surgeon ranking: the case of the New York State cardiac surgery report card. Med. Care, 2006, 44 , 311–319.

    Osler T. M. , 'Impact of changing the statistical methodology on hospital and surgeon ranking: the case of the New York State cardiac surgery report card ' (2006 ) 44 Med. Care : 311 -319.

    • Search Google Scholar
  • Reed, J. F., Olenchock, S. A.: Comparative analysis of risk-adjusted bypass surgery stratification models in a community hospital. Heart Lung, 2003, 32 , 383–390.

    Olenchock S. A. , 'Comparative analysis of risk-adjusted bypass surgery stratification models in a community hospital ' (2003 ) 32 Heart Lung : 383 -390.

    • Search Google Scholar
  • Aylin, P., Jarman, B., Kelsey, T.: What hospital mortality league tables tell you. BMJ, 2003, 326 , 1397–1398.

    Kelsey T. , 'What hospital mortality league tables tell you ' (2003 ) 326 BMJ : 1397 -1398.

  • Seagroatt, V., Goldacre, M. J.: Hospital mortality league tables: influence of place of death. BMJ, 2004, 328 , 1235–1236.

    Goldacre M. J. , 'Hospital mortality league tables: influence of place of death ' (2004 ) 328 BMJ : 1235 -1236.

    • Search Google Scholar
  • Joyce, R., Webb, R., Peacock, J. L. et al.: Adjusted mortality rates: a tool for creating more meaningful league tables for stillbirth and infant mortality rates. Public Health, 2002, 116 , 315–321.

    Peacock J. L. , 'Adjusted mortality rates: a tool for creating more meaningful league tables for stillbirth and infant mortality rates ' (2002 ) 116 Public Health : 315 -321.

    • Search Google Scholar
  • Bradley, E. H., Herrin, J., Elbel, B. et al.: Hospital quality for acute myocardial infarction. Correlation among process measures and relationship with short-term mortality. JAMA, 2006, 296 , 72–78.

    Elbel B. , 'Hospital quality for acute myocardial infarction. Correlation among process measures and relationship with short-term mortality ' (2006 ) 296 JAMA : 72 -78.

    • Search Google Scholar
  • Krumholz, H. M., Rathore, S. S., Chen, J. et al.: Evaluation of a consumer-oriented Internet health care report card: the risk of quality ratings based on mortality data. JAMA, 2002, 287 , 1277–1287.

    Chen J. , 'Evaluation of a consumer-oriented Internet health care report card: the risk of quality ratings based on mortality data ' (2002 ) 287 JAMA : 1277 -1287.

    • Search Google Scholar
  • Mohammed, M. A., Mant, J., Bentham, L. et al.: Comparing processes of stroke care in high- and low-mortality hospitals in the West Midlands, UK. Int. J. Qual. Health Care, 2005, 17 , 31–36.

    Bentham L. , 'Comparing processes of stroke care in high- and low-mortality hospitals in the West Midlands, UK ' (2005 ) 17 Int. J. Qual. Health Care : 31 -36.

    • Search Google Scholar
  • Jacobs, R., Goddard, M., Smith, P. C.: How robust are hospital ranks based on composite performance measures? Med. Care, 2005, 43 , 1177–1184.

    Smith P. C. , 'How robust are hospital ranks based on composite performance measures? ' (2005 ) 43 Med. Care : 1177 -1184.

    • Search Google Scholar
  • Spiegelhalter, D. J.: Handling over-dispersion of performance indicators. Qual. Saf. Health Care, 2005, 14 , 347–351.

    Spiegelhalter D. J. , 'Handling over-dispersion of performance indicators ' (2005 ) 14 Qual. Saf. Health Care : 347 -351.

    • Search Google Scholar
  • Barker, R. M., Pearce, M. S., Irving, M.: Star wars, NHS style. BMJ, 2004, 329 , 107–109.

    Irving M. , 'Star wars, NHS style ' (2004 ) 329 BMJ : 107 -109.

  • Bird, S. M., Cox, D. R., Farewell, V. T. et al.: Performance indicators: good, bad, and ugly. J. R. Stat. Soc. Ser. A, 2005, 168 , 1–27.

    Farewell V. T. , 'Performance indicators: good, bad, and ugly ' (2005 ) 168 J. R. Stat. Soc. Ser. A : 1 -27.

    • Search Google Scholar
  • Broder, M. S., Payne-Simon, L., Brook, R. H.: Measures of surgical quality: what will patients know by 2005? J. Eval. Clin. Pract., 2005, 11 , 209–217.

    Brook R. H. , 'Measures of surgical quality: what will patients know by 2005? ' (2005 ) 11 J. Eval. Clin. Pract. : 209 -217.

    • Search Google Scholar
  • Jha, A. K.: Measuring hospital quality. What physicians do? How patients fare? Or both? JAMA, 2006, 296 , 95–97.

    Jha A. K. , 'Measuring hospital quality. What physicians do? How patients fare? Or both? ' (2006 ) 296 JAMA : 95 -97.

    • Search Google Scholar
  • Cheng, S. H., Song, H. Y.: Physician performance information and consumer choice: a survey of subjects with the freedom to choose between doctors. Qual. Saf. Health Care, 2004, 13 , 98–101.

    Song H. Y. , 'Physician performance information and consumer choice: a survey of subjects with the freedom to choose between doctors ' (2004 ) 13 Qual. Saf. Health Care : 98 -101.

    • Search Google Scholar
  • Marshall, M. N., Shekelle, P. G., Leatherman, Sh. et al.: The public release of performance data. What do we expect to gain? A review of the evidence. JAMA, 2000, 283 , 1866–1874.

    Leatherman Sh. , 'The public release of performance data. What do we expect to gain? A review of the evidence ' (2000 ) 283 JAMA : 1866 -1874.

    • Search Google Scholar
  • Morrison, I.: Quality information: Are consumers ready for report cards? Hosp. Health Netw., 2003, 77 , 38–38.

    Morrison I. , 'Quality information: Are consumers ready for report cards? ' (2003 ) 77 Hosp. Health Netw. : 38 -38.

    • Search Google Scholar
  • Schauffler, H. H., Mordavsky, J. K.: Consumer reports in health care: Do they make a difference? Annu. Rev. Public Health, 2001, 22 , 69–89.

    Mordavsky J. K. , 'Consumer reports in health care: Do they make a difference? ' (2001 ) 22 Annu. Rev. Public Health : 69 -89.

    • Search Google Scholar
  • Schneider, E. C., Lieberman, T.: Publicly disclosed information about the quality of health care: response of the US public. Qual. Health Care, 2001, 10 , 96–103.

    Lieberman T. , 'Publicly disclosed information about the quality of health care: response of the US public ' (2001 ) 10 Qual. Health Care : 96 -103.

    • Search Google Scholar
  • Beck, C. A., Richard, H., Tu, J. V. et al.: Administrative data feedback for effective cardiac treatment: AFFECT, a cluster randomized trial. JAMA, 2005, 294 , 309–317.

    Tu J. V. , 'Administrative data feedback for effective cardiac treatment: AFFECT, a cluster randomized trial ' (2005 ) 294 JAMA : 309 -317.

    • Search Google Scholar
  • Borzak, S.: Administrative data feedback failed to improve quality of care in acute myocardial infarction. ACP J. Club, 2005, 143 , 79–79.

    Borzak S. , 'Administrative data feedback failed to improve quality of care in acute myocardial infarction ' (2005 ) 143 ACP J. Club : 79 -79.

    • Search Google Scholar
  • Bridgewater, B., Hooper, T., Campbell, C. et al.: Publication of league tables needs to be open and accurate. BMJ, 2002, 324 , 542–543.

    Campbell C. , 'Publication of league tables needs to be open and accurate ' (2002 ) 324 BMJ : 542 -543.

    • Search Google Scholar
  • Fong, T.: Not making the grade. Mod. Healthc., 2003, 33 , 12–13.

    Fong T. , 'Not making the grade ' (2003 ) 33 Mod. Healthc. : 12 -13.

  • Davies, O., Marshall, M. N.: Public disclosure of performance data: does the public gets what the public wants? Lancet, 1999, 353 , 1639–1640.

    Marshall M. N. , 'Public disclosure of performance data: does the public gets what the public wants? ' (1999 ) 353 Lancet : 1639 -1640.

    • Search Google Scholar
  • Dranove, D., Kessler, D., McClellan, M. et al.: Is more information better? The effects of ‘report cards’ on health care providers. NBER Working Paper No. 8697*, 2002. www.nber.org/papers/w8697

  • Mullen, P. M.: Using performance indicators to improve performance. Health Serv. Manage. Res., 2004, 17 , 217–228.

    Mullen P. M. , 'Using performance indicators to improve performance ' (2004 ) 17 Health Serv. Manage. Res. : 217 -228.

    • Search Google Scholar
  • Werner, R. M., Asch, D. A., Polsky, D.: The unintended consequences of coronary artery bypass graft report cards. Circulation, 2005, 111 , 1257–1263.

    Polsky D. , 'The unintended consequences of coronary artery bypass graft report cards ' (2005 ) 111 Circulation : 1257 -1263.

    • Search Google Scholar
  • McGlynn, E. A., Cassel, Ch. K., Leatherman, Sh. T. et al.: Establishing national goals for quality improvement. Med. Care, 2003, 41 , 116–129.

    Leatherman Sh. T. , 'Establishing national goals for quality improvement ' (2003 ) 41 Med. Care : 116 -129.

    • Search Google Scholar
  • McGlynn, E. A.: Selecting common measures of quality and system performance. Med. Care, 2003, 41 , 139–147.

    McGlynn E. A. , 'Selecting common measures of quality and system performance ' (2003 ) 41 Med. Care : 139 -147.

    • Search Google Scholar
  • Krumholz, H. M., Brindis, R. G., Brush, J. E. et al.: Standards for statistical models used for public reporting of health outcomes. An American Heart Association Scientific statement from the quality of care and Outcomes Research Interdisciplinary Writing Group. Circulation, 2006, 113 , 456–462.

    Brush J. E. , 'Standards for statistical models used for public reporting of health outcomes. An American Heart Association Scientific statement from the quality of care and Outcomes Research Interdisciplinary Writing Group ' (2006 ) 113 Circulation : 456 -462.

    • Search Google Scholar
  • Leatherman, S. T., Hibbard, J. H., McGlynn, E. A.: A research agenda to advance quality measurement and improvement. Med. Care, 2003, 41 , 180–186.

    McGlynn E. A. , 'A research agenda to advance quality measurement and improvement ' (2003 ) 41 Med. Care : 180 -186.

    • Search Google Scholar
  • Scobie, S., Thomson, R., McNeil, J. J. et al.: Measurement of the safety and quality of health care. Med. J. Aust., 2006, 184 , S51–S55.

    McNeil J. J. , 'Measurement of the safety and quality of health care ' (2006 ) 184 Med. J. Aust. : S51 -S55.

    • Search Google Scholar
  • Finkelstein, B. S., Harper, D. L., Rosenthal, G. E.: Patient assessments of hospital maternity care: a useful tool for consumers? Health Serv. Res., 1999, 34 , 623–640.

    Rosenthal G. E. , 'Patient assessments of hospital maternity care: a useful tool for consumers? ' (1999 ) 34 Health Serv. Res. : 623 -640.

    • Search Google Scholar
  • Berwick, D. M., James, B., Coye, M. J.: Connections between quality measurement and improvement. Med. Care, 2003, 41 , 130–138.

    Coye M. J. , 'Connections between quality measurement and improvement ' (2003 ) 41 Med. Care : 130 -138.

    • Search Google Scholar
  • Marshall, M. N., Hiscock, J., Sibbald, B.: Attitudes to the public release of comparative information on the quality of general practice care: qualitative study. BMJ, 2002, 325 , 1278–1282.

    Sibbald B. , 'Attitudes to the public release of comparative information on the quality of general practice care: qualitative study ' (2002 ) 325 BMJ : 1278 -1282.

    • Search Google Scholar