Exploring why quality circles work in primary health care: a realist review protocol
© Rohrbasser et al.; licensee BioMed Central Ltd. 2013
Received: 8 July 2013
Accepted: 25 November 2013
Published: 9 December 2013
Quality circles (QCs) are commonly used in primary health care in Europe to consider and improve standard practice over time. They represent a complex social intervention that occurs within the fast-changing system of primary health care. Numerous controlled trials, reviews, and studies have shown small but unpredictable positive effect sizes on behavior change. Although QCs seem to be effective, stakeholders have difficulty understanding how the results are achieved and in generalizing the results with confidence. They also lack understanding of the active components of QCs which result in changes in the behavior of health care professionals. This protocol for a realist synthesis will examine how configurations of components and the contextual features of QCs influence their performance.
Stakeholder interviews and a scoping search revealed the processes of QCs and helped to describe their core components and underlying theories. After clarifying their historical and geographical distribution, a purposive and systematic search was developed to identify relevant papers to answer the research questions, which are: understanding why, how, and when QCs work, over what time frame, and in what circumstances. After selecting and abstracting appropriate data, configurations of contexts and mechanisms which influence the outcome of QCs within each study will be identified. Studies will be grouped by similar propositional statements in order to identify patterns and validation from stakeholders sought. Finally, theories will be explored in order to explain these patterns and to help stakeholders maintain and improve QC performance.
Analyzing context-mechanism-outcome (CMO) patterns will reveal how QCs work and how contextual factors interact to influence their outcome. The aim is to investigate unique configurations that enable them to improve the performance of health care professionals. Using a standardized reporting system, this realist review will allow the research questions to be answered to the satisfaction of key stakeholders and enable on-going critical examination and dissemination of the findings.
PROSPERO registration number: CRD42013004826.
KeywordsRealist synthesis Realist review Quality circle Peer review group Small group work Quality improvement
Rationale for the review
Quality circles (QCs) are small groups of 6 to 12 professionals from a similar background who meet at regular intervals to discuss and review their clinical practice. The focus of discussion is usually a critical evaluation of a key issue related to quality in health care. QCs select the issues they want to deal with themselves, decide on their method of gathering data, and determine a way of finding solutions to prioritized problems. Facilitators observe and lead the group through the circle of quality improvement [1–8]. The main purposes of QCs in primary health care are continuing professional development (CPD), quality improvement, and continuing medical education (CME) [3, 9–13].
QCs have been established predominantly in Germany, Switzerland, Austria, Sweden, Denmark, Norway, Belgium, France, the Netherlands, and Ireland . In Switzerland, they have been established as the main method of quality improvement and CPD and, currently, 80% of all physicians working in primary health care regularly attend QCs .
Numerous studies suggest that QCs improve individual and group performance in terms of costs, ordering of tests, prescription habits, and adherence to clinical practice guidelines, thus resulting in better patient outcome measures and changes in performance indicators [6, 8, 16–22]. Several systematic reviews (SRs) in the Cochrane Library show that elements of QCs such as educational materials, workshops, audit and feedback (with or without outreach visits), and local knowledge experts have a positive impact on the behavior of practitioners [23–27]. Facilitation is a key factor that has been shown in several reviews to be effective [28, 29]. Although systematic reviews of QCs provide summaries of their effectiveness, they are based on the assumption that the intervention has causal powers, and are typically unable to explain considerable variations in effectiveness because the original trials that are included in the review rarely explore the influence of surrounding contexts effectiveness .
Stakeholders in Switzerland, including practitioners, networks of health care centers, professional associations, and health insurance companies, largely recognize that QCs are effective but they have difficulty generalizing the results with confidence. They believe that QCs provide a social context for reflective practice and allow the dissemination of knowledge to influence the work practices of the participants. While stakeholders agree about the context and broad range of internal mechanisms, they are not confident in understanding how the active components of QCs prompt physicians to change their behavior .
QCs have all the properties of complex interventions. Complex interventions depend on human behavior and their active ingredients tend to enable people to do the right thing at the right time or constrain them from doing something. They combine numerous and varying components and they function in diverse contexts. Individually, QCs respond to the unique constellation of the local needs of the complex system of primary health care. They are also responsive to changes in prevailing economic and cultural circumstances [32, 33].
For these reasons, a realist review is necessary to explore how local context interacts with various mechanisms to produce more or less effective QCs [34–36]. Further, a realist review can produce important information about the relative effectiveness of various components, thereby enabling stakeholders and practitioners to make informed decisions about the best structure and process for their particular QCs.
Objectives and focus of the review
The primary aim of this synthesis is to explain the QC program by finding underlying program theories, reviewing how the theories are tested within studies, and comparing context-mechanism-outcome (CMO) configurations across studies to produce a middle-range theory explaining why, how, and when QCs work, over what time frame, and in what circumstances. The range of components that characterize QCs, their underlying mechanisms, and the local context in which QCs are conducted will be documented. The patterns within QCs, in which components act both independently and inter-dependently, will be investigated and mapped in relation to variations in underlying mechanisms and the local context. Study outcomes will be evaluated to identify optimal conditions for success, which will then inform stakeholders about strategies to manage and maintain current QCs.
How do configurations of components and their underlying mechanisms within QCs influence their outcomes?
How do contextual features surrounding QCs improve individual and/or group performance?
This protocol sets out the scope for the realist review, describes the initial program theory for QCs and relevant candidate theories, outlines the search strategy and process for sifting abstracts on the topic of interest, and proposes a theory-based framework for extracting data. Overall approaches to data analysis and synthesis are outlined, with the condition that, as with all realist reviews, the approaches are tailored to ensure they are appropriate for the available data.
Identifying candidate theories
We initially explored the QC program theory using a scoping review of published papers and grey literature together with discussions with stakeholders [2, 37–40]. By using terms such as ‘quality circle’ in the title and the abstract in Ovid MEDLINE and Embase, examining training material used in Switzerland, Austria and Germany, and studying the background literature used by stakeholders, the scoping review revealed a number of key elements that may facilitate QCs [4, 5, 41–44].
QCs are assumed to work because they bring people together to identify key issues concerning the quality of health care and they involve people in exploring solutions where there is a need for improvement. Coming from similar backgrounds, meeting in small groups at regular intervals, and, perhaps, voluntary participation are basic properties of QCs. It also seems to be important to have a trained facilitator who can engage the individuals and groups, support autonomy in terms of selection of the issue and the approach for reviewing information, and find solutions . Communication techniques such as debate, consensus discussion, brainstorming, reflective thinking, self-observation, and role play among other practices appear to keep QCs active. Educational strategies such as audit and feedback, outreach visits, workshop-like atmosphere, and use of local opinion leaders are regularly brought into play [45, 46]. Results of QCs include improved patient outcomes measured in changes of performance indicators and cost benefits .
After informal discussions with stakeholders from Switzerland and other European countries where QCs are prevalent, a set of questions was used to guide interviews with QC participants, facilitators, mentors, and regulators in Switzerland. These interviews clarified the objectives and the focus of QCs and confirmed that stakeholders have a common understanding of this important program. Stakeholders believe that QCs work because groups provide a social context for reflective practice and allow the dissemination of the findings to the participants’ working places, especially if they are embedded in a wider system. This has provided the impetus for this review.
Search for data
An initial scoping search was performed to clarify the historical development of QCs, establish time frames, and describe their geographical distribution. The terms ‘peer review group’ (PRG), ‘quality circle’ (QC), and ‘small group work’ (SGW) were all included in the search as these terms are used interchangeably in different European countries. A Web of Knowledge citation map was used to identify the earliest published paper in 1979. Therefore, this search begins in 1980 using MEDLINE, Embase, PsycINFO, and CINHAL databases without language restrictions [48–51]. A comprehensive but purposive search for literature will be conducted using search strings for terms related to the descriptors of QC program theory, quality improvement, group functions and facilitation, and primary health care [Additional file 1]. In collaboration with a librarian, we developed a strategy that was guided by a selected set of documents [Additional file 2] . At all stages, snowball strategies and access to the grey literature will be used. Authors will be contacted to clarify data when necessary. An iterative search may be necessary if new prospective theories are identified during data analysis. Results of stakeholder discussions, interviews, and consultations as well as training and conference material will serve as data.
Selection of studies
Does the study describe QCs that take place in a primary health care setting?
Does the study describe QCs that include structured SGW including a facilitator?
These questions identify appropriate examples of QCs in the setting of primary health care. When both questions can be answered in the affirmative, the next step will be to identify appropriate quantitative papers that provide an adequate description of the evaluation methods and outcomes together with appropriate qualitative studies that provide descriptions and explanations of the key elements in the preliminary program theory.
Does the study provide details about the tools and outcomes of evaluation? Or
Does the study provide qualitative data on the context in which the QC takes place? Or
Does the study present qualitative data on QC group dynamics or facilitation? Or
Does the study contain qualitative data on QCs and social learning, adult learning, learning techniques, or behavioral change theories?
Quality assessment of the studies will focus on determining relevance and credibility in relation to the research question and how the findings relate to the context of the study . Reasons why papers are included or excluded will be documented.
Data extraction and analysis
Data will be extracted as text and quantitative summaries to describe the configuration of mechanisms, context, and outcomes for each study. Author discussions of reasons for QC successes or failures will be included as data. Where studies have explicitly used a program theory, the theory will be noted.
In the first stage of analysis, context components will be extracted together with descriptions of mechanisms on a study-by-study basis. The configuration within each study will be identified; that is, how context interacts with mechanisms to either enable or constrain QCs. Propositions will be developed for each study describing the relationship between context, mechanism, and outcome .
In the second stage of analysis, studies will be grouped by similar propositional statements in order to identify patterns in an iterative process. For example, QCs that occur in similar contexts could be compared to see if these similar contexts consistently trigger the same mechanisms. Likewise, QCs that focus on the same mechanisms will also be compared across different contexts to examine if these mechanisms are consistent in terms of producing similar outcomes. Differences in contexts and configurations of components will be highlighted in which the underlying mechanisms can either be constraining or enabling. These demi-regularities will be presented as statements of which types of QC work, in which local contexts and circumstances, for which type of practitioner, and at which points in time. Swiss and European stakeholders have declared an interest in assisting with building these propositional statements as experts in local contexts. At the third stage of analysis, middle-range theories will be explored in order to explain the visible and hidden forces behind the demi-regularities. Our consultation with stakeholders and reviews of training material suggested that we may find explanatory theories from psychological, social, or economic sciences that offer some explanation about the complex social interactions within a QC. The search on theories has revealed three different theoretical fields focusing on factors that influence professional behavior change: 1) theories of adult learning, social learning, and problem-based learning, which can be used to explain improvement in observed practice and changes in physician behavior [54, 55]; 2) theories on individual practitioner behavior change, which can be mapped to general domains of behavior change ; and 3) theories related to implementation research in health settings and translating knowledge into action [57, 58].
Over the past ten years, the theoretical domains that influence practitioner behavior change have been systematically identified [59, 60]. The Theoretical Domains Framework (TDF) was developed as a theory-based classification of terms labeling behavior change across 14 key explanatory domains . Several SRs and randomized studies have demonstrated that using theoretical domains helps to identify relevant theories to explain clinical practice and problems with implementation [62–64]. For this review, the TDF will be used as an initial organizing framework for categorizing the process of behavior change across QCs. It is recognized that most realist reviews aim to identify potential middle-range theories at an early stage of the review, but the scoping revealed a diversity in terms of types of theories. Therefore, we have chosen the approach used by Jagosh et al. (2013) of first sorting the literature and then immersion in the data, at which stage a considered approach can be taken to reviewing goodness of fit with the theoretical areas identified during preliminary theory scoping .
We aim to synthesize the findings so that they are of use to practitioners when designing or modifying QCs. The stakeholders’ main interest in a realist review lies in learning more about which ingredients lead to better outcomes and lower costs and which contexts are necessary to achieve these goals over a longer period of time. Fine-tuning the synthesis of theories underlying each of the original review questions should provide stakeholders with relevant answers [34, 53, 68].
Theories in comparative settings will reveal which contextual features are of importance. Questioning program integrity will reveal which combinations of contexts or interventions at different levels, be it group, organization, or primary health care, will support the QC process. The assessment of rival program theories will identify middle-range theories and demi-regularities that can optimize QC performance. This will result in specific recommendations of optimal variations to the program in specific contexts, thus enabling stakeholders to implement successful patterns of contextual features and effective QC components .
Reporting and dissemination of the findings
Transparent reporting of the process and findings is important for creating an audit trail. The standards of realist and meta-narrative evidence synthesis (RAMESES) provide publication criteria for a review of this type . The authors will follow the RAMESES statement as they explore and answer the research questions in language acceptable to all stakeholders. An academic article will be written for publication in an international journal specializing in implementation. Findings will be disseminated through consultations with stakeholders in Switzerland, who will be able to critically evaluate the results and then, ultimately, implement them. Further, the results of the review will be presented at the meeting of the European Society for Quality and Safety in Family Practice (EQuiP) in 2015. This will allow European stakeholders to evaluate current practice and discuss further steps for enhancing QCs across Europe.
The authors have already completed the protocol, and plan in December 2013 to start the review, which has been registered with PROSPERO, the international prospective register of systematic reviews: CRD42013004826.
A review of this kind does not require approval from an ethics committee because it is not primary research. However, it will follow the relevant standards of utility, usefulness, feasibility, propriety, accuracy, and accountability [70, 71].
This realist review has two major limitations. Firstly, it is dependent on the transparency and adequacy of reporting by original authors. To address this, authors will be contacted for clarification and related reports will be examined. Additionally, there is the risk of the selective bias of choosing underlying theories and synthesizing them in an ad hoc manner. This will be addressed through the creation of an overarching framework to guide systematic theory development and through an iterative process of communication with the stakeholders and consultation of the literature.
Realist synthesis is a methodology under development and this protocol makes explicit the processes of using stakeholders as a key source of information to clarify the scope of the review. We developed separate search strategies for understanding the scope, finding underlying theories, and identifying primary studies. We explicitly described the data extraction and analysis process across three levels of investigation, and have planned the final stage of synthesizing data and drawing of conclusions for theory development. The stakeholders will receive on-going reports for checking and ensuring policy relevance and for safeguarding future policy development.
AR: DPhil, Department of Primary Health Care Sciences, University of Oxford, Oxford, UK; santémed Health Care Centres; member of EQuiP; and delegate for Quality in Medicine on behalf of the Swiss Association of Family Medicine. SM: Senior Researcher, Centre for Evidence Based Medicine, Department of Primary Health Care Sciences, University of Oxford, Oxford, UK. JH: Senior Lecturer and Course Director, MPH in International Health Management and Leadership, School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK; and Co-Convenor, Cochrane Qualitative Research Methods Group.
Behavior change wheel
Continuing medical education
Continuing professional development
European Society for Quality and Safety in Family Practice
Promoting action on research implementation in health services
Peer review group
Realist and meta-narrative evidence synthesis
Small group work
Theoretical domains framework.
Nia Roberts at the Department of Primary Health Care Sciences, Bodleian Libraries, University of Oxford, has contributed substantially to the search strategy of this work.
This paper was written without any funding as part of AR’s DPhil studies at the Department of Primary Health Care Sciences, University of Oxford.
- Ishikawa K: How to Operate QC Circle Activities. 1985, Tokyo: QC Circle Headquarters, Union of Japanese Scientists and EngineersGoogle Scholar
- Tutorengruppe für die Ausbildung von Qualitätszirkel-Moderatoren: Definition medizinischer Qualitätszirkel – ein Vernehmlassungstext. Prim Care. 2005, 5 (16): 370-372.Google Scholar
- Grol R, Lawrence M: Quality Improvement by Peer Review. 1995, Oxford: Oxford University PressGoogle Scholar
- Tross O: Qualitätszirkel als Form der Arbeitsorganisation: Planung und Gestaltung von Qualitätszirkelnals Variante der Teamarbeit in Unternehmen. 2003, Munich: GRIN VerlagGoogle Scholar
- Verstappen WH, van Merode F, Grimshaw J, Dubois WI, Grol RP, van der Weijden T: Comparing cost effects of two quality strategies to improve test ordering in primary care: a randomized trial. Int J Qual Health Care. 2004, 16 (5): 391-398. 10.1093/intqhc/mzh070.View ArticlePubMedGoogle Scholar
- Watkins C, Timm A, Gooberman-Hill R, Harvey I, Haines A, Donovan J: Factors affecting feasibility and acceptability of a practice-based educational intervention to support evidence-based prescribing: a qualitative study. Fam Pract. 2004, 21 (6): 661-669. 10.1093/fampra/cmh614.View ArticlePubMedGoogle Scholar
- Wensing M, Broge B, Riens B, Kaufmann-Kolle P, Akkermans R, Grol R, Szecsenyi J: Quality circles to improve prescribing of primary care physicians. Three comparative studies. Pharmacoepidemiol Drug Saf. 2009, 18 (9): 763-769. 10.1002/pds.1778.View ArticlePubMedGoogle Scholar
- Grol R, Baker R, Wensing M, Jacobs A: Quality assurance in general practice: the state of the art in Europe. Fam Pract. 1994, 11 (4): 460-467. 10.1093/fampra/11.4.460.View ArticlePubMedGoogle Scholar
- Gerlach FM, Beyer M, Römer A: Quality circles in ambulatory care: state of development and future perspective in Germany. Int J Qual Health Care. 1998, 10 (1): 35-42. 10.1093/intqhc/10.1.35.View ArticlePubMedGoogle Scholar
- Ennis K, Harrington D: Quality management in Irish health care. Int J Health Care Qual Assur Inc Leadersh Health Serv. 1999, 12 (6–7): 232-243.View ArticlePubMedGoogle Scholar
- O'Riordan M: Continuing medical education in Irish general practice. Scand J Prim Health Care. 2000, 18 (3): 137-138. 10.1080/028134300453313.View ArticlePubMedGoogle Scholar
- Munck A, Damsgaard J, Hansen DG, Bjerrum L, Sondergaard J: The Nordic method for quality improvement in general practice. Qual Prim Care. 2003, 11 (1): 73-78.Google Scholar
- Beyer M, Gerlach FM, Flies U, Grol R, Krol Z, Munck A, Olesen F, O'Riordan M, Seuntjens L, Szecsenyi J: The development of quality circles/peer review groups as a method of quality improvement in Europe. Results of a survey in 26 European countries. Fam Pract. 2003, 20 (4): 443-451. 10.1093/fampra/cmg420.View ArticlePubMedGoogle Scholar
- Meyer-Nikolic VA, Hersperger M: Qualitätsentwicklung in der ambulanten Medizin CH: Q-Monitoring-Resultate schaffen Übersicht. Schweizerische Ärztezeitung. 2012, 93: 1036-1038.Google Scholar
- Verstappen WH, van der Weijden T, Sijbrandij J, Smeele I, Hermsen J, Grimshaw J, Grol RP: Effect of a practice-based strategy on test ordering performance of primary care physicians: a randomized trial. JAMA. 2003, 289 (18): 2407-2412. 10.1001/jama.289.18.2407.View ArticlePubMedGoogle Scholar
- Verstappen WH, van der Weijden T, Dubois WI, Smeele I, Hermsen J, Tan FE, Grol RP: Improving test ordering in primary care: the added value of a small-group quality improvement strategy compared with classic feedback only. Ann Fam Med. 2004, 2 (6): 569-575. 10.1370/afm.244.View ArticlePubMedPubMed CentralGoogle Scholar
- Niquille A, Ruggli M, Buchmann M, Jordan D, Bugnon O: The nine-year sustained cost-containment impact of Swiss pilot physicians-pharmacists quality circles. Ann Pharmacother. 2008, 44 (4): 650-657.View ArticleGoogle Scholar
- Welschen I, Kuyvenhoven MM, Hoes AW, Verheij TJ: Effectiveness of a multiple intervention to reduce antibiotic prescribing for respiratory tract symptoms in primary care: randomised controlled trial. BMJ. 2004, 329 (7463): 431-10.1136/bmj.38182.591238.EB.View ArticlePubMedPubMed CentralGoogle Scholar
- Stevenson K, Baker R, Farooqi A, Sorrie R, Khunti K: Features of primary health care teams associated with successful quality improvement of diabetes care: a qualitative study. Fam Pract. 2001, 18 (1): 21-26. 10.1093/fampra/18.1.21.View ArticlePubMedGoogle Scholar
- Schneider A, Wensing M, Biessecker K, Quinzler R, Kaufmann-Kolle P, Szecsenyi J: Impact of quality circles for improvement of asthma care: results of a randomized controlled trial. J Eval Clin Pract. 2008, 14 (2): 185-190. 10.1111/j.1365-2753.2007.00827.x.View ArticlePubMedPubMed CentralGoogle Scholar
- Bugnon O, Jotterand S, Niquille Charrière A, Ruggli M, Herzig L: Physicians-pharmacists quality circles: shared responsibility of the freedom of prescription. Rev Med Suisse. 2012, 8 (341): 1044-1048. 1042Google Scholar
- O'Brien M, Rogers S, Jamtvedt G, Oxman Andrew D, Odgaard-Jensen J, Kristoffersen Doris T, Forsetlund L, Bainbridge D, Freemantle N, Davis DA, Haynes RB, Harvey EL: Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2007, 4: CD000409-PubMedGoogle Scholar
- Forsetlund L, Bjorndal A, Rashidian A, Jamtvedt G, O'Brien MA, Wolf F, Davis D, Odgaard-Jensen J, Oxman AD: Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009, 2: CD003030-PubMedGoogle Scholar
- Baker R, Camosso-Stefinovic J, Gillies C, Shaw Elizabeth J, Cheater F, Flottorp S, Robertson N: Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2010, 3: CD005470-PubMedGoogle Scholar
- Flodgren G, Parmelli E, Doumit G, Gattellari M, O'Brien Mary A, Grimshaw J, Eccles Martin P: Local opinion leaders: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2011, 8: CD000125-PubMedGoogle Scholar
- Ivers N, Jamtvedt G, Flottorp S, Young Jane M, Odgaard-Jensen J, French Simon D, O'Brien Mary A, Johansen M, Grimshaw J, Oxman Andrew D: Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012, 6: CD000259-PubMedGoogle Scholar
- Dogherty EJ, Harrison MB, Graham ID: Facilitation as a role and process in achieving evidence-based practice in nursing: a focused review of concept and meaning. Worldviews Evid Based Nurs. 2010, 7 (2): 76-89.PubMedGoogle Scholar
- Baskerville NB, Liddy C, Hogg W: Systematic review and meta-analysis of practice facilitation within primary care settings. Ann Fam Med. 2012, 10 (1): 63-74. 10.1370/afm.1312.View ArticlePubMedPubMed CentralGoogle Scholar
- Shepperd S, Lewin S, Straus S, Clarke M, Eccles MP, Fitzpatrick R, Wong G, Sheikh A: Can we systematically review studies that evaluate complex interventions?. PLoS Med. 2009, 6 (8): e1000086-10.1371/journal.pmed.1000086.View ArticlePubMedPubMed CentralGoogle Scholar
- Campbell NC, Murray E, Darbyshire J, Emery J, Farmer A, Griffiths F, Guthrie B, Lester H, Wilson P, Kinmonth AL: Designing and evaluating complex interventions to improve health care. BMJ. 2007, 334 (7591): 455-459. 10.1136/bmj.39108.379965.BE.View ArticlePubMedPubMed CentralGoogle Scholar
- Rickles D, Hawe P, Shiell A: A simple guide to chaos and complexity. J Epidemiol Community Health. 2007, 61 (11): 933-937. 10.1136/jech.2006.054254.View ArticlePubMedPubMed CentralGoogle Scholar
- Shiell A, Hawe P, Gold L: Complex interventions or complex systems? Implications for health economic evaluation. BMJ. 2008, 336 (7656): 1281-1283. 10.1136/bmj.39569.510521.AD.View ArticlePubMedPubMed CentralGoogle Scholar
- Pawson R, Greenhalg T, Haervey G, Walshe K: Realist synthesis: an introduction. Research Methods: An ESRC Research Programme. 2004, Manchester: ESRC Research Methods Programme, University of ManchesterGoogle Scholar
- Rycroft-Malone J, McCormack B, Hutchinson A, DeCorby K, Bucknall T, Kent B, Schultz A, Snelgrove-Clarke E, Stetler C, Titler M, Wallin L, Wilson V: Realist synthesis: illustrating the method for implementation research. Implement Sci. 2012, 7 (1): 33-10.1186/1748-5908-7-33.View ArticlePubMedPubMed CentralGoogle Scholar
- Astbury B, Leeuw FL: Unpacking black boxes: mechanisms and theory building in evaluation. Am J Eval. 2010, 31 (3): 363-381. 10.1177/1098214010371972.View ArticleGoogle Scholar
- Leeuw FL: Reconstructing program theories: methods available and problems to be solved. Am J Eval. 2003, 24 (1): 5-20.View ArticleGoogle Scholar
- Die Argomed-Qualitätszirkel.http://www.argomed.ch/qualitaetszirkel.html,
- Nicholson JAH: Quality circles and participative management. Farm Manag. 1987, 6 (8): 335-341.Google Scholar
- Langmaak B, Braune-Krickau M: Wie die Gruppe laufen lernte. 1989, Berlitz Verlag: WeinheimGoogle Scholar
- Gerlach FM: Qulaitätssicherung durch hausärtzliche Qualitätszirkel: Strategien zur Etablierung. 1994, Berlin: Ullstein MosbyGoogle Scholar
- Lampe G, Szecsenyi J, Weiss-Plumeyer M: Moderatorentraining. 1994, Göttingen: AQUA-VerlagGoogle Scholar
- Lawrence M, Schofield T: Medical Audit in Primary Health Care. 1993, Oxford: Oxford University PressGoogle Scholar
- Marinker M: Medical Audit and General Practice. 1995, London: BMJ Publishing GroupGoogle Scholar
- Fraser R, Mayur L, Baker R: Evidence-Based Audit in General Practice. 1999, Oxford: Butterworth HeinemannGoogle Scholar
- Rendall E: Quality circles - a ‘third wave’ intervention. Train Dev J. 1981, 35 (3): 28-Google Scholar
- Turban E, Kamin JY: Cost-benefit analysis of quality circles. Eng Costs Prod Econ. 1984, 8 (3): 199-209. 10.1016/0167-188X(84)90037-5.View ArticleGoogle Scholar
- Schmele JA, Allen ME, Butler S, Gresham D: Quality circles in the public health sector: implementation and effect. Public Health Nurs. 1991, 8 (3): 190-195. 10.1111/j.1525-1446.1991.tb00754.x.View ArticlePubMedGoogle Scholar
- Bilawka E, Craig BJ: Quality assurance in health care: past, present and future. Int J Dent Hyg. 2003, 1 (3): 159-168. 10.1034/j.1601-5037.2003.00035.x.View ArticlePubMedGoogle Scholar
- McMahon T, Ward PR: HIV among immigrants living in high-income countries: a realist review of evidence to guide targeted approaches to behavioural HIV prevention. Syst Rev. 2012, 1: 56-10.1186/2046-4053-1-56.View ArticlePubMedPubMed CentralGoogle Scholar
- Pawson R: Evidence Based Policy: A Realist Perspective. 2006, London: SAGEView ArticleGoogle Scholar
- Principles of Adult Learning.http://www.qotfc.edu.au/resource/?page=65375,
- Elwyn G, Greenhalg T, Macfarlane F: Groups: A Guide to Small Group Work in Healthcare, Management, Education and Research. 2004, Oxford: Radcliffe Medical PressGoogle Scholar
- Michie S, van Stralen M, West R: The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011, 6 (1): 42-10.1186/1748-5908-6-42.View ArticlePubMedPubMed CentralGoogle Scholar
- Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N: Lost in knowledge translation: time for a map?. J Contin Educ Health Prof. 2006, 26 (1): 13-24. 10.1002/chp.47.View ArticlePubMedGoogle Scholar
- Grimshaw JM, Schunemann HJ, Burgers J, Cruz AA, Heffner J, Metersky M, Cook D: Disseminating and implementing guidelines: article 13 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report. Proc Am Thorac Soc. 2012, 9 (5): 298-303.View ArticlePubMedGoogle Scholar
- Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A: Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005, 14 (1): 26-33. 10.1136/qshc.2004.011155.View ArticlePubMedPubMed CentralGoogle Scholar
- Leeman J, Baernholdt M, Sandelowski M: Developing a theory-based taxonomy of methods for implementing change in practice. J Adv Nurs. 2007, 58 (2): 191-200. 10.1111/j.1365-2648.2006.04207.x.View ArticlePubMedGoogle Scholar
- Cane J, O'Connor D, Michie S: Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012, 7 (1): 37-10.1186/1748-5908-7-37.View ArticlePubMedPubMed CentralGoogle Scholar
- Francis JJ, Stockton C, Eccles MP, Johnston M, Cuthbertson BH, Grimshaw JM, Hyde C, Tinmouth A, Stanworth SJ: Evidence-based selection of theories for designing behaviour change interventions: using methods based on theoretical construct domains to understand clinicians' blood transfusion behaviour. Br J Health Psychol. 2009, 14 (Pt 4): 625-646.View ArticlePubMedGoogle Scholar
- Group A, McSherry L, Dombrowski S, Francis J, Murphy J, Martin C, O'Leary J, Sharp L: ‘It’s a can of worms': understanding primary care practitioners' behaviours in relation to HPV using the theoretical domains framework. Implement Sci. 2012, 7 (1): 73-10.1186/1748-5908-7-73.View ArticleGoogle Scholar
- Francis J, O'Connor D, Curran J: Theories of behaviour change synthesised into a set of theoretical groupings: introducing a thematic series on the theoretical domains framework. Implement Sci. 2012, 7 (1): 35-10.1186/1748-5908-7-35.View ArticlePubMedPubMed CentralGoogle Scholar
- Jagosh J, Pluye P, Wong G, Cargo M, Salsberg J, Bush PL, Herbert CP, Green LW, Greenhalgh T, Macaulay AC: Critical reflections on realist review: insights from customizing the methodology to the needs of participatory research assessment. 2013, Methods: Res Synth, doi: 10.1002/jrsm.1099Google Scholar
- Kitson A, Rycroft-Malone J, Harvey G, McCormack B, Seers K, Titchen A: Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges. Implement Sci. 2008, 3 (1): 1-10.1186/1748-5908-3-1.View ArticlePubMedPubMed CentralGoogle Scholar
- Straus SE, Tetroe J, Graham I: Defining knowledge translation. Can Med Assoc J. 2009, 181 (3–4): 165-168.View ArticleGoogle Scholar
- Greenhalgh T, Wong G, Westhorp G, Pawson R: Protocol–realist and meta-narrative evidence synthesis: evolving standards (RAMESES). BMC Med Res Methodol. 2011, 11 (1): 115-10.1186/1471-2288-11-115.View ArticlePubMedPubMed CentralGoogle Scholar
- Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R: RAMESES publication standards: meta-narrative reviews. BMC Med. 2013, 11 (1): 20-10.1186/1741-7015-11-20.View ArticlePubMedPubMed CentralGoogle Scholar
- Bislimi R, Bischofberger I, Drewe J, Galeazzi R, Kesselring A, Kind C, Salathé M, Rehmann-Sutter C, Sprumont D: Forschung mit Menschen: ein Leitfaden für die Praxis. 2009, Schwabe: BaselGoogle Scholar
- Yarbrough DB, Shulha LM, Hopson RK, Caruthers FA: The Program Evaluation Standards: A Guide for Evaluators and Evaluation Users. 2010, Thousand Oaks, CA: SAGE, 3Google Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.