Policy Statement Database

New Search »

Supporting and Sustaining the Practice of Quality Improvement in Public Health

Policy Date: 11/5/2013
Policy Number: 201314

Related APHA Statements

APHA Policy Statement 201210 – Promoting Health Impact Assessment (HIA) to Achieve Health in All Policies [1]

APHA Policy Statement 201119 – Encouraging Health Department Accreditation[2]

APHA Policy Statement 201015 – Securing the Long-Term Sustainability of State and Local Health Departments Policy Statement[3]

APHA Policy Statement 200911 – Public Health’s Critical Role in Health Reform in the United States[4]

APHA Policy Statement 20066 – Conduct Research to Build an Evidence-Base of Effective Community Health Assessment Practice[5]

APHA Policy Statement 200022 – Joint Resolution in Support of National Public Health Performance Standards Program (NPHPSP)[6]

APHA Policy Statement 200210 – Maternal and Child Health (MCH) Data Capacity through the National Action Agenda[7]

APHA Policy Statement 20088 – Promoting Inter-professional Education[8]

APHA Policy Statement 2005-12 – Strengthening the Public Health Work Force to Address Current and Future Challenges [9]

Abstract:

Effective quality improvement (QI) programs in public health departments have the potential to manage the public’s resources effectively and efficiently, build public trust, and improve staff morale. QI is the use of a deliberate and defined process, focused on activities that are responsive to community needs and improve population health. QI is the cornerstone of the national voluntary health department accreditation process. What’s new is: (1) the structured process to identify root causes; (2) continuously monitoring changes; and (3) conducting iterative Plan-Do-Study-Cycles. To enable greater use of QI in public health, there has to be a “culture-change” through training, applying QI methods, and sharing experiences. We describe the current assets and challenges to the adoption of QI in public health including: identifying measures of quality; training workforce; teaching students; funding resources; sharing experiences and disseminating QI results; and QI research. To support and sustain the practice of quality improvement in public health, six action steps are recommended: (1) Identify measures that can be used in quality improvement; (2) Fund opportunities for application of quality improvement; (3) Coordinate strategies to support workforce training in quality improvement; (4) Allocate a percentage of grant funds to support quality improvement and public health accreditation; (5) Support online dissemination of public health quality improvement resources that illustrate use of QI tools and the resulting improvements; and (6) Prioritize QI research agenda.
Problem Statement

Opportunities for implementing quality improvement in public health can be identified by understanding its benefits, strengths and challenges in the current environment. Quality improvement and performance management is the use of a deliberate and defined process focused on activities to community needs and improving public health. [13, 14, 15] It is different from the traditional planning-implementation-evaluation approach, in that it involves ongoing, rapid cycles of change with embedded measurements to promote immediate learning, and adapting programs to be more effective. [14] Public health practitioners struggle with this approach of rapid testing, ongoing measurement, immediate adaptation and re-testing. In schools of public health, students learn frameworks and models that planning, implementation and evaluation with before-after intervention measurements. [14]

Benefits of a quality improvement approach, based on its long history in manufacturing and health care sectors, include using resource effectively and efficiently, illustration of effective leadership, building trust, and increasing workforce morale.[16, 17] Common elements of quality improvement and performance management include: commitment from senior management to quality; employee involvement and empowerment, customer-defined quality and a focus on customer satisfaction, process focus and integration, and institutionalization of continuous improvement so it is integrated in everything an organization does.[13,18] The process or model provides the roadmap, and tools such as “cause and effect” diagrams, run charts, affinity diagrams among others provide the means for action.[17]

It is important to understand some of the key actions that influenced Quality Improvement movement in Public Health before recommending action steps. Throughout the years, there has been the intent to focus on understanding and measuring performance for action and accountability.
• Institute of Medicine (IOM) Reports and Recommendations[19] – the IOM has provided national guidance throughout the decades with reports such as the landmark 1988 report that established the core functions of public health in the “Future or Public Health”, the 1996 report “Improving health in the community: A Role for Performance Monitoring”; the 2003 “The Future of Public’s health in the 21st Century” report that included recommendations for catalyzing accreditation and increasing system partnerships, and the 2010 IOM report “For the Public’s Health: The Role of Measurement in Action and Accountability” providing recommendations on performance measurement.
• Essential Public Health Services [20] – established in 1994 and outlining essential areas of public health responsibility. This framework has set the stage for much of the subsequent work in standards and performance management.
• National Public Health Performance Standards Program [21] – begun in 1998, is intended to advance the quality and performance of public health systems through the use of public health system standards and performance improvement processes. The national performance standards for public health systems and governing bodies – which are based upon the Essential Public Health Services framework – have been used in more 1000 local jurisdictions and 30 states.
• Turning Point Performance Management model and resources [22] - the Turning Point Performance Management Collaborative provides tools and framework for implementing performance management in public health.
• Accreditation [17]– Since Fall 2011, state, tribal, local and territorial health departments are able to apply for volunteering accreditation. The goal of the program is anchored around quality improvement.
• Multi-State Learning Collaborative [23] – this Robert Wood Johnson Foundation funded initiative from 2005 until 2011 worked with 16 states to inform the development of a national accreditation program.
• Quality in Public Health [24] – the U.S. Department of Health and Human Services has led a series of efforts focused on improving quality in public health and health care. Through this effort, a Consensus Statement on Quality in the Public Health System and a report describing Priority Areas for Improvement of Quality in Public Health were developed.
• National Public Health Improvement Initiative [25] – initiated in 2010 with Affordable Care Act funding, this CDC initiative provides US state, tribal, local and territorial health departments with resources to adopt and institutionalize cross cutting performance management and quality improvement methods and approaches to improve accountability, efficiency, and effectiveness of public health programs and services.

Quality Improvement approaches can be used across the lifecourse. What are the factors behind why public health practitioners struggle with using quality improvement, and what assets can be leveraged to address the challenges? The challenges and assets are discussed under six areas. They are:

1. Improvement Measures: On the asset side is the availability of outcome indicators as in Healthy People 2020 [26] and other sites. [27, 28, 29] Also a strength is the availability of the technology to solicit, collect and share new indicators such as the National Center for Health Statistics (NCHS) online Health Indicators Warehouse [30] and similar models such as Agency for Healthcare Research and Quality (AHRQ) National Quality Measures Clearinghouse. [31] The challenge is to identify a framework and menu of improvement measures, which are a mixture of outcome and process measures for practitioners to use [32], identification of community health measures such as walkability in a community [33], and in information technology to overcome challenges related to investment, adoption and re-engineer process as faced by colleagues in the healthcare sector. [34, 35]

2. Workforce training: While there is increasing availability of training on quality improvement approaches for current public health workforce and the leadership [14, 36, 37], the need is much greater than the capacity to offer training. [13] Trained practitioners would be able utilize a process and framework for identifying QI measures.

3. Teaching quality improvement approaches in public health schools: While some coursework address quality [38] and quality improvement, it is not widespread. Ideally training for QI should be integrated into core curriculums across public health disciplines.

4. Funding allocation for quality improvement: To some extent allocation of funds for public health accreditation of local and state health departments is being allowed in some grants. Funders should support the use of quality improvement across all programs as this will encourage support for the QI approach. [39]

5. Illustration of use of quality improvement tools in public health: The recently released Public Health Quality Improvement Exchange (PHQIX), an online database of QI efforts with a forum for dialogue for practitioners may be one attempt to capture smaller-scale public health QI activities. [40] Several guidebooks on quality improvement in public health illustrate case studies such as in “Embracing Quality in Public Health: Michigan’s Quality Improvement Guidebook,”[41] Public Health Foundation’s ASQ on public health quality[42]; and GOAL/QPC’s Public Health Memory Jogger.[43] Availability of evidence-based public health resources such as the Community Guide[44], AHRQ’s evidence-based clinical practice[45], Cochrane Library[46], HIV[47], and Injury Control[48], among others help with identifying approaches for quality improvement projects. The challenge for QI databases will be balancing inclusiveness while highlight relevant use of quality improvement process and tools.

6. Practice-based evidence and dissemination: Several groups have identified research agendas. [49, 50] Researchers who have sought to review use of quality improvement practices in relation to outcomes have described intervention research as “thin”, and that a lot of the QI projects in public health do not link to outcomes. [51, 52] Out of the 18 interventions the authors studies, 9 were not linked to health outcomes. Some examples of improvement outcomes cited in a systematic review [51] are:

• Implied linkage to improved hepatitis A incidence rates and increased polio immunization for children (though there was not comparison group): Study was doing at the Los Angeles County Health Department from 2002-2004. The QI steps they practiced includes (1) accountability framework and program performance measures; (2) employee duty statements and recognition; and (3) evidence-based practice reviews.

• Increased Chlamydia treatment rates from 87% and 94% in two county health departments in Washington State in 2007 to 96% in 2009. They identified sites with highest rates of untreated Chlamydia infection and developed tailored education strategies for the sites. They identified sites with highest rates of untreated Chlamydia infection through the hospital emergency room and specific providers and developed tailored education strategies for sites.

There is work underway to standardize the definition, common metrics and taxonomy for QI projects to begin building the evidence-base; [53] development and testing of a tool to measure an organization’s QI maturity, [54, 55] and a tool to measure QI Return on Investment. [56] There are strong guidelines, Standards for QUality Improvement Reporting Excellence (SQUIRE), to share research related to QI. [57] While there were no studies we found that discussed how to share QI lessons learned with consumers who may not be part of the team, it was alluded in two studies. One study discussed the importance of separating the rhetoric versus the factual aspects of QI in communications, [58] and the other AHRQ studies the effects of communications on measures and provider behaviors from the pressure of demonstrating improvement. [59] Neither studied it from the perspective of the consumer.

QI Approaches and their impact on underserved population and promote health equity
Much of the literature on quality improvement process and disparities was found in the health care sector. Experts suggest that Public Health Systems and Services practitioners and researchers can learn from the lessons in the clinical services field. [60] The promise of QI has spurred experimentation and led to development of tools and strategies. These efforts underscore the need to carefully define, measure, and monitor changes in equity to identify which interventions are effective. Questions that remain unanswered are: (1) What types of interventions can improve equity for chronic conditions prevalent today? (2) What strategies can support and spread QI efforts to achieve equity and what are the key infrastructure needs to do so? The field of health equity has evolved from asking if disparities exist to asking why they exist. The focus is on identifying what works to reduce gaps. Diverse populations have different needs and barriers, and the same QI intervention can affect them variably.[61] Addressing disparities is complex, there are no short answers. The learning continues.

“Finding Answers: Disparities Research for Change”, a national program on healthcare for the Robert Wood Johnson Foundation (RWJF) has identified a process “Roadmap to Reduce Disparities”, and four broad recommendations. The four recommendations are: (1) Use the process roadmap to reduce disparities from beginning to the end; (2) Utilize a QI model that establishes equity as a cross-cutting dimension of every component of quality; (3) Do not get stuck on any one step, rather move forward with the equity agenda in several ways; and (4) Provide a menu of best practices and model intervention for reducing disparities.[62]

Evidence that QI is consistent with or advances the state of science

Public health is a fledgling in the QI movement. While there is promise indicated in the health care sector and manufacturing, the foundation for evidence of impact of QI is being built. Riley et al proposed a standardized case definitions, common metrics and taxonomy of QI project to begin building the evidence based for QI in public health using data from local health departments. The data was presented in three sections: (1) profile of health departments in the study; (2) nature and extent and impact of QI projects; and (3) an opportunity modes and effects analysis (OMEA) based on modification of the failure modes and effects analysis (FMEA) to identify and prioritize QI projects with the greatest potential impact. This research is in the preliminary stage.[53]

In health care, a systematic review that evaluated the effectiveness of public reporting of health care quality information as a quality improvement strategy showed that for most of the outcomes, the strength of the evidence available to assess the impact of public reporting was moderate. This was due in part to the methodological challenges researchers face in designing and conducting research on the impact of population-level interventions. Public reporting is more likely to be associated with changes in health care provider behaviors than with selection of health services providers by patients or families. Quality measures that are publicly reported improve over time. Although the potential for harms from the pressure to demonstrate improvement is frequently cited by commentators and critics of public reporting, the amount of research on harms is limited and most studies did not confirm the potential harm.[59]

Political/Resource Issues
There is a need for resources at the state and local level for data, trained workforce, programming, research, communications and funding as outlined in the challenges section of this paper. In addition, QI practitioners have to address the challenge of communicating improvements and balancing reporting of drawbacks or errors in sometime a contentious political climate. This often results in a model of evolving rhetoric and reality of quality improvement.[58]

Ethical Issues
Discussion on ethical aspects of implementing and studying the improvement suggest addressing privacy concerns, protection of participants’ physical well-being, potential conflicts of interests, and purposed explanations of how ethical concerns were addressed. [57]

Proposed Recommendations Statement
Six recommendations for supporting and sustaining the practice of quality improvement and performance management in public health are identified: (1) Identify measures that can be used in quality improvement; (2) Conducting workforce education in quality improvement; (3) Funding the application of quality improvement methods; (4) Setting aside a percentage of grant funds to support quality improvement and public health accreditation; (5) Supporting evaluation of online quality improvement resources in public health practice so that they are responsive to practitioner needs; and (6) Developing a quality improvement research agenda that uses established communication standards.

Opposing Arguments/Evidence

“Quality” is desired by organizations and individuals. However, in every institution, there is reality of improvement that is tangible, and the rhetoric of how institutional forces can distort technical reality by communicating an overly optimistic view of quality improvement. .[58] QI practitioners say that initially staff may see QI as busy work or extra work, not any different from other add-ons imposed on them. For many staff, it is not until they are part of QI efforts, and see that their input does make their work better, that they change their opinions and become advocates. Theoretical didactic training that is not integrated with hands-on application to existing organizational practices is not the best tactic for adoption. [58] QI is better learned through adult learning techniques, combining a hands-on approach with relevant modules. In the health care sector, resources spent on QI have also been seen by some as taking away from health care and new technology development, and evaluators question the consistency of definitions used for measuring the costs and benefits of QI efforts.[59] Critics and proponents of QI agree that academics and leaders should assess costs and benefits of QI efforts.

Alternative Strategies

An alternative approach to quality improvement is the planning, implementation and evaluation where the public health practitioners’ collect data, select priorities and best practice interventions, implement the interventions and then evaluates the project. The QI approach involves rapid testing, ongoing measurement, immediate adaptation and re-testing. [14] Both approaches are important, and QI is more suited to projects where the scope is specific, the outcomes clear and more successful when there are proven methods to bring about the improvement.

There are many approaches used in quality improvement, the ones most often mentioned in public health practice are Lean, Model for Improvement, Total Quality Management (TQM), Balanced Scorecard, Baldrige or its state equivalent and Turning Point Performance Management System.[14, 63]

All these models require the same broad set of minimal conditions for implementation. They include: the active engagement of key stakeholders, the active participation of middle and senior managers and the support of leadership; To move toward a culture of quality improvement, an organization needs to use of multiple-prong interventions and sustained action at different levels of the organization; the alignment of QI activities with the organization’s strategic goals; and the embedding of QI as an integral part of the everyday work of all staff rather than as the responsibility of a separate team. Moving to a culture of QI requires up-front investment in terms of staff capacity, learning methods best suited for specific aims or topics, and the value of iterative and rapid cycles of improvement.


Action Steps

1. The Department of Health and Human Services should work with state, local, tribal, and territorial governments, and other federal agencies, to identify measures of use in quality improvement efforts; support the infrastructure needed to collect data on these measures, and disseminate the measures throughout the public health system: HHS should support and strengthen federal leadership in health measurement to collaborate with state and local governments to identify measures for QI efforts and provide the infrastructure to make data on those measures readily available in real time. Innovations in measures related to both traditional public health as well as measures related to community conditions promotion health (such as walkability, graduation rates, violence rates, etc) should be supported for testing and validation for use in QI.

2. National organizations, governmental agencies, foundations must fund and support opportunities for quality improvement training of current and future public health workforce and leaders: There is a dire need for quality improvement training of public health workers and leaders on a large scale to enable the application of QI in practice. . The lessons learned over the past ten years about effective methods to diffuse QI learning and develop a culture of QI in practice settings should be used to develop programming. Institutions of higher learning should increasingly teach QI with integration into core

3. National organizations, governmental agencies, foundations and the Public Health Accreditation Board should coordinate quality improvement training of current and future public health workforce and leaders: Training of the public health workforce and students can be increased by including quality improvement in the curricula of public health academic training, use of information technology in quality improvement processes and including quality improvement competencies in graduate programs. Training leaders influences organizational culture, opening up support for continuing education on quality improvement and the use of quality improvement frameworks, and facilitates incorporating quality improvement into job descriptions and work plans.

4. Governmental agencies and funders should support quality improvement work and public health accreditation by allocating a percentage of grant funds: Quality improvement applications need resources, including information technology. Funders could encourage the use of quality improvement methods by setting aside a percentage of funds for QI and public health accreditation.

5. National organizations, governmental agencies, and funders should support quality improvement resources so they are responsive to practitioners’ needs: There is growing interest and investment in using quality improvement, performance management and related approaches. It is important to assess whether the resources are responsive to practitioners’ needs, such as linking QI approaches to public health outcome improvement or predictors; or serve as a repository of evidence-based intermediate measures; or illustrate use of quality improvement tools such as cause-effect diagrams, check sheet or run charts in addition to providing description of projects.

6. Governmental agencies should collaborate with foundations to prioritize public health quality improvement research agenda and encourage the use of communication standards: Governmental agencies collaborate with foundations to synthesize questions related to the science of improvement in public health, and to develop a research agenda. As these agencies are also funders, they should encourage public health practitioners to use communication standards such as the Standards for QUality Improvement Reporting Excellence (SQUIRE) statement or similar guidelines to report on quality improvement in journals and reporting. In addition, effective ways to communicate quality improvements lessons to the consumer should be studied.


XIV. References

1. American Public Health Association. APHA policy statement 201210," Promoting Health Impact Assessment to Achieve Health in All Policies ". Washington DC. American Public Health Association 2012. Available at: http://www.apha.org/advocacy/policy/policysearch/default.htm?id=1444 Accessed May 31, 2013.
2. American Public Health Association. APHA policy statement 2011-D6," Public Health Accreditations a Means to Strengthen Governmental Public Health Systems in the US". Washington DC. American Public Health Association 2011. Available at: http://www.apha.org/advocacy/policy/2011approvedpolicies.htm Accessed May 31, 2013.
3. American Public Health Association APHA policy statement 201015, Securing the Long-Term Sustainability of State and Local Health Departments Policy Statement," Washington, D C. American Public Health Association 2010. Available at: http://www.apha.org/advocacy/policy/policysearch/default.htm?id=1404 Accessed May 31, 2013.
4. American Public Health Association APHA policy statement 200911, Public Health's Critical Role in Health Reform in the United States" Washington DC. American Public Health Association 2009. Available at: http://www.apha.org/advocacy/policy/policysearch/default.htm?id=1386 May 31, 2013.
5. American Public Health Association APHA policy statement 20066, Conduct Research to Build a Evidence-Base of Effective Community Health Assessment Practice." Washington DC. American Public Health Association 2006. Available at: http://www.apha.org/advocacy/policy/policysearch/default.htm?id=1330 May 31, 2013.
6. American Public Health Association APHA policy statement 200022, Joint Resolution in Support of National Public Health Performance Standards Program (NPHPSP), Washington DC. American Public Health Association 2000. Available at: http://www.apha.org/advocacy/policy/policysearch/default.htm?id=229 May 31, 2013.
7. American Public Health Association APHA policy statement 200210, Maternal and Child Health (MCH) Data Capacity through the National Action Agenda, Washington DC. American Public Health Association 2002. Available at: http://www.apha.org/advocacy/policy/policysearch/default.htm?id=286 May 31, 2013.
8. American Public Health Association APHA policy statement 20088, Promoting Interprofessional Education, Washington DC. American Public Health Association 2008. Available at: http://www.apha.org/advocacy/policy/policysearch/default.htm?id=1374 Accessed May 31, 2013.
9. American Public Health Association APHA policy statement 2005-12 Strengthening the Public Health Work Force to Address Current and Future Challenges Washington DC. American Public Health Association 2005. Available at: http://www.apha.org/advocacy/policy/policysearch/default.htm?id=1306 May 31, 2013.
0. We counted word frequencies in the policy statement listings from the APHA policy database to identify words possibly relevant to quality improvement and performance management. We reviewed the titles of statements whose titles or keywords contained one or more of the words and read summaries of all 2011 policies. In addition to the keyword search, a review of policies over the last 10 years was done manually. We read policies on workforce and collaboration. We found six policies supporting quality improvement and performance management, including one policy encouraging health department accreditation, two broad policies supporting workforce training, and one specifically calling for training on data capacity.
11. American Public Health Association. Policy Statement Gaps for 2013. Available at: http://www.apha.org/advocacy/policy/policygaps.htm Accessed May 31, 2013.
12. Juran JM. The Quality Trilogy – A Universal Approach to Managing for Quality. Juran Institute. Available at: http://www.juran.com/elifeline/elifefiles/2009/09/Juran-Trilogy-Model.doc Accessed May 31, 2013.
13. Randolph GS, Lea S. Quality Improvement in Public Health: Moving from knowing the path to walking the Path. Journal of Public Health Management and Practice, 2012, 18(1) 4-8.
14. Riley WJ, Moran JW, Corso LC, Beitsch LM, Bialek R, Cofsky A. Commentary – Defining Quality Improvement in Public Health. Journal of Public Health Management and Practice, 2010, 16(1) 5-7.
15. Public Health Foundation. Turning Point. From Silos to Systems: Using Performance Management to Improve the Public's Health, 2003. Available at: http://www.turningpointprogram.org/Pages/pdfs/perform_manage/Silos_to_Sytems_FINAL.pdf Accessed May 31, 2013.
16. Tews DS, Sherry MK, Butler JA, Martin A. Embracing Quality in Local Public Health, Michigan’s Quality Improvement Guidebook, 2008. Available at: http://www.accreditation.localhealth.net/MLC-2%20website/Michigans_QI_Guidebook.pdf Accessed May 31, 2013.
17. Public Health Accreditation Board. Available at: http://www.phaboard.org/ as of May 31May 31, 2013.
18. Tague NR. The Quality Toolbox. 2005. American Society for Quality.
19. Institute of Medicine Reports Index. Available at: http://www.iom.edu/Reports.aspx as of May 31, 2013
20. Public Health Function Steering Committee. Public Health in America. Available at http://www.health.gov/phfunctions/public.htm as of January15, 2013.
21. Centers for Disease Control and Prevention. National Public Health Performance Program. Available at: http://www.cdc.gov/nphpsp/index.html as of May 31, 2013.
22. Turning Point Initiative. Available at: http://www.turningpointprogram.org/ as of May 31, 2013
23. National Network of Public Health Institutes. Accreditation and Performance Improvement. Available at: http://www.nnphi.org/program-areas/accreditation-and-performance-improvement as of May 31, 2013.
24. HHS. What is Public Health Quality. Available at http://www.hhs.gov/ash/initiatives/quality/quality/index.html as of May 31, 2013.
25. Centers for Disease Control and Prevention. National Public Health Improvement Initiative. Available at: http://www.cdc.gov/stltpublichealth/NPHII/index.html as of May 31, 2013.
26. US Department of Health and Human Services. Healthy People 2020. Topics & Objectives Index – Healthy People. Available at: http://www.healthypeople.gov/2020/topicsobjectives2020/default.aspx. Accessed May 31, 2013.
27. America’s Health Rankings. Take Action for America’s Health. Minnetonka, MN: United Health Foundation. 2011. Available at: http://www.americashealthrankings.org/. Accessed May 31, 2013.
28. US Department of Health and Human Services. Healthy People 2020. Community Health Status Indicators. Available at: http://www.communityhealth.hhs.gov/HomePage.aspx. Accessed May 31. 2013.
29. Robert Wood Johnson Foundation. County Health Rankings – Mobilizing Action Toward Community Health. Available at: http://www.countyhealthrankings.org/. Accessed May 31, 2013.
30. Health Indicators Warehouse. Propose an indicator – Governance of Health Indicators. Available at: http://www.healthindicators.gov/Indicators/Propose. Accessed May 31, 2013.
31. Agency for Healthcare Research and Quality. National Quality Measures Clearinghouse. Available at: http://qualitymeasures.ahrq.gov/. Accessed May 31, 2013.
32. Institute of Healthcare Improvement. Science of Improvement – Establishing measures. Available at: http://www.ihi.org/knowledge/Pages/HowtoImprove/ScienceofImprovementEstablishingMeasures.aspx as of May 31, 2013.
33. Institute of Medicine. For the Public's Health: The Role of Measurement in Action and Accountability.2011. Washington, DC: The National Academies Press. Available online: http://www.iom.edu/Reports/2010/For-the-Publics-Health-The-Role-of-Measurement-in-Action-and-Accountability.aspx. Accessed May 31, 2013.
34. Foldy S. Health Information Techology. Promises and Challenges for Disease Control and Prevention. Presented at the NCHHSTP Prevention through Health Care Consultation Conference on June 21 in Atlanta, Georgia. http://www.naccho.org/advocacy/healthreform/upload/Foldy_Plenary_PTHC_June21.pdf. Accessed May 31, 2013.
35. Kellermann AL, Jones SS. What it will take to achieve that as-yet-unfulfilled promises of health information technology. Health Affairs, January 2013 32(1) pp.63-68. doi:10.1377/hlthaff.2012.0693
36. MCH Navigator. Program Implemention. Available at: http://navigator.mchtraining.net/?page_id=185 as of May 31, 2013.
37. Public Health Foundation. Welcome to the Quality Improvement Quick Guide Tutorial Available at: http://www.phf.org/quickguide/LeftNavOnePanel.aspx?Page=Tutorial as of May 31May 31, 2013.
38. Written and oral communication with HHS Staff, January 2013.
39. Mathews GW and Baker EL. Looking Back From the Future: Connecting Accreditation, Health Reform, and Political Opportunities. Journal of Public Health Management and Practice. 2010, 16(4), 367-369.
40. Public Health Quality Improvement Exchange. Available at: https://www.phqix.org/. Accessed May 31, 2013.
41. Michigan Local Public Health Accreditation Program. Embracing Quality in Public Health: Michigan’s Quality Improvement Guidebook. Available at: http://www.accreditation.localhealth.net/guidebook.htm, Accessed May 31, 2013.
42. Bialek R, Moran JW, Duffy GL. The Public Health Quality Improvement Handbook. ASQ Quality Press. 2009 .
43. Brassard M, Ritter D, Oddo F, MacCausland J. The Memory Jogger 2: Tools for Continuous Improvement and Effective Planning. Goal/QPC, 2007.
44. Community Preventive Services Task Force. The Community Guide. Available at: http://www.thecommunityguide.org/index.html. Accessed May 31, 2013.
45. Agency for Healthcare Research and Quality. Evidence-Based Practice. Available at: http://www.ahrq.gov/clinic/epcix.htm. Accessed May 31, 2013.
46.The Cochrane Collaboration. Cochrane Public Health Group. Available at: http://ph.cochrane.org/finding-public-health-reviews. Accessed May 31, 2013.
47. Centers for Disease Control and Prevention. The 2009 Compendium of Evidence-based HIV Prevention Interventions. Available at: http://www.cdc.gov/hiv/topics/research/prs/evidence-based-interventions.htm. Accessed May 31, 2015.
48. Centers for Disease Control and Prevention. CDC Falls Compendium Table of Contents. Available at: http://www.cdc.gov/HomeandRecreationalSafety/Falls/compendium/0.0_toc.html. Accessed May 31, 2013.
49. Riley WJ, Lownik B, Scutchfield FD, May GP, Corso LC, Beitsch LM. Public Health Accreditation – Setting the Research Agenda. American Journal of Preventive Medicine, 2012, 42(3): 263-271.
50. Consortium from Altarum Institute; Centers for Disease Control and Prevention; Robert Wood Johnson Foundation; National Coordinating Center for Public Health Services and Systems Research. A national research agenda for public health services and systems. Am J Prev Med. 2012 May;42(5 Suppl 1):S72-8. doi: 10.1016/j.amepre.2012.01.026.
51. Dilley JA, Bekemeier B, Harris JR. Quality Improvement Interventions in Public Health Systems – A Systematic Review. American Journal of Preventive Medicine. 2012. 42(5S1):S58-S71.
52. Scutchfield FD, Howard AF, Mays GP. Public Health Services and Systems Research – An Afterword. American Journal of Preventive Medicine. 2012. 42(5S1): S84-S86.
53. Riley WJ, Lownik B, Halverson P, Parrotta C, Godsall JR, Gyllstrom E, Gearin K, Mays G. Developing a Taxonomy for the Science of Improvement in Public Health. Journal of Public Health Management and Practice, 2012, 18(6) 506-514.
54. Joly BM, Booth M, Mittal P, Shaler G. Measuring Quality Improvement in Public Health: The Development and Psychometric Testing of a QI Maturity Too. Evaluation and the Health Professions, 2012, 35(2) 119-147.
55. Gearin KJM, Gyllstrom ME, Joly BM, Frauendienst RS, Myhre J. Monitoring QI Maturity of Public Health Organizations and Systems in Minnesota: Promising Early Findings and Suggested Next Steps. Frontiers in Public Health Services and Systems Research, 2013, 2(3), Article 3. Available at http://uknowledge.uky.edu/frontiersinphssr/vol2/iss3/3. Accessed May 31, 2013.
56. Association of State and Territorial Health Officials, Return On Investment (ROI) Project Summary and Workgroup, Personal Communication, December 2012.
57. Ogrinc G, Mooney SE, Estrada C., Foster T, Golderman D, Hall LW, Huizinga MM, Liu SK, Mills P, Neily J, Nelson W, Pronovost PJ, Provost L, Rubenstein LV, Speroff T, Splaine M, Thomson R, Tomolo AM, Watts B. The SQUIRE (Standards for Quality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Qual Saf. Health Care.2008;17/Suppl 1:i13-i32 and available at: http://squire-statement.org/guidelines. Accessed May 31, 2013.
58. Zbaracki MJ. The Rhetoric and Reality of Total Quality Management. Administrative Science Quarterly. 1998. 43(3): 602-636.
59. AHRQ. Evidence Report/Technology Assessment Number 208. Public Reporting as Quality Improvement – Closing the Quality Gap: Revisiting the State of the Science. July 2012. Available at: http://www.effectivehealthcare.ahrq.gov/ehc/products/343/1199/EvidReport208_CQGPublicReporting_FinalReport_20120724.pdf. Accessed May 31, 2013.
60. Academy Health. Research Insights. Addressing Disparities through Public Health Systems and Services Research and Adaptations to Public Health Practice. 2009. Available at; http://www.academyhealth.org/files/publications/PHSRDisparitiesbrief.pdf. Accessed February 4, 2013.
61. Center for Health Professions in the University of California San Francisco. Bringing Equity into Quality Improvement: An Overview of Opportunities Ahead. 2012. Available at: http://futurehealth.ucsf.edu/Content/5436/Part%201%20_Equity%20into%20QI.pdf. Accessed February 4, 2013.
62. Marshall Hc, Clarke AR, Nocon RS, Casey AA, Goddu AP, Keesecker NM, Cook SC. A Roadmap and Best Practices for Organizations to Reduce Racial and Ethnic Disparities in Health Care. Journal of General Internal Medicine. 27(8), (2012, 992-1000, DOI: 10.1007/s11606-012-2082-9
63. Beitsch LM, Leep C, Shah G, Brooks RG, Pestronk RM. Quality Improvement in Local Health Departments: Results of the NACCHO 2008 Survey. Journal of Public Health Management & Practice. January/February 2010. 16(1):49-54.