This realist synthesis will follow the steps and procedures outlined RAMESES Publication Standards for Realist Synthesis [28] and associated training materials [30] in an iterative manner. Realist principles will be embedded in all stages of the review process. A PRISMA-P checklist has been completed and is available as an additional file (see Additional file 1).
Defining the scope of the review
Clinical learning environments in postgraduate medical education and training is a broad topic. Initial rough searches of the literature suggest that there is a substantial published literature in this area. Focussing the review question will be an iterative process consisting of exploration of the literature and relevant programme theories as well as consultation with experts and stakeholders. We envisage that focussing of the review will be guided by the evidence as it is discovered and the need to ensure a manageable volume of literature for synthesis.
We will identify programme and substantive workplace learning theories which will support the identification of the key areas on which to focus and the most relevant literature to consider. There are several theories of learning which can inform our understanding of the processes and outcomes of doctors’ workplace learning and how clinical environments might impact these. Communities of practice theory [32] emphasises the importance of participation in practice and connection, recognition and belonging within a community. Cognitive apprenticeship theory [33, 34] provides an account of how people learn from each other, through observation, modelling and reflection. The frameworks for workplace learning developed by Billett [35] and Teunissen [36] may also prove relevant to this review. An initial programme theory for workplace learning in postgraduate medical education and training will be developed and expressed in realist terms. This theory will be refined as the review progresses and becomes more focussed.
Regular meetings will be held with the wider project team to discuss and define the key aspects of the review, to ensure consensus on review focus. Additionally, the scope of the review will be informed by another study into clinical learning environment being undertaken by the group. This is a consensus building study which will use group concept mapping to capture the perspectives of multiple stakeholders such as trainers, trainees, allied health professionals and hospital management on priority areas for improvement within clinical learning environments. The findings of this study will help to direct the focus of the realist synthesis.
Search strategy
Unlike the search strategy of traditional systematic reviews, searches in realist synthesis do not aim to uncover every published paper addressing the topic, but rather to balance comprehensiveness with theoretical saturation. Searching is iterative and as synthesis and theory refinement occur further searches may be necessary to test the emerging theory. Initially, we will perform an electronic search in the following databases: Academic Search Complete, Australian Education Index, British Education Index, CINAHL, ERIC, MEDLINE, PsycInfo and SocIndex. Search terms will be developed and trialled iteratively, and in discussion within the wider research team and a librarian. Both MeSH (medical subject headings) and free text will be employed to ensure breadth and depth of coverage. The search will be adjusted as required in each database. We will supplement searches by reviewing the reference lists of included studies and relevant review articles. We will also search the following journals by hand: Academic Medicine, Advances in Health Sciences, Graduate Medical Journal, Medical Education, Medical Teacher and Postgraduate Medical Journal. We will check the validity of our search by contacting experts in the field of workplace learning to identify the key papers. Additional rounds of searching may be added throughout the review process to further explore particular areas of interest. Searching will cease when sufficient evidence has been found to demonstrate the coherence and plausibility of the refined programme theory.
Our core inclusion criteria will be (1) papers related to postgraduate medical education and training in the clinical setting, (2) quantitative, qualitative and mixed-method studies, (3) papers published in English and (4) papers published between 1995 and 2016. Exclusion criteria include (1) non-empirical papers including commentaries, letters, editorials and reviews, (2) papers related to undergraduate medical education and (3) research on simulation or other non-clinical interventions.
Study selection criteria
Titles and abstracts will be imported into EndNote and screened by the core research team using the criteria outlined above. Full texts of articles deemed potentially relevant will be retrieved and evaluated for inclusion in the data extraction stage. Inclusion and exclusion decisions will be based on whether the findings can contribute to theory testing and refinement and whether the methods used to generate findings are credible and trustworthy.
The questions that will guide us selecting papers based on relevance are the following: Does the study relate directly to the clinical learning environment of medical trainees? Is the study rich enough in information on context, mechanism and outcomes? These questions will ensure that the sources identified allow the team to make sense of the subject area, in order to develop, refine and test theories, and to support inferences made about mechanisms. Reasons for exclusion of papers will be thoroughly documented to ensure transparency. Searching will be iterative and will discontinue when sufficient evidence is found that it is reasonable to claim that the programme theory is coherent and plausible.
Quality of the papers selected for data extraction will be assessed, and methodological limitations will be identified and taken into account during the data synthesis phase. Realist synthesis is an interpretative approach to the literature and the RAMESES Realist Synthesis Training Materials do not recommend using a strict checklist approach to quality, as this can lead to exclusion of relevant papers early in the process. Our assessment of quality will involve the use of checklists, for example CASP, as sensitising influences, but will lean towards inclusion of data from relevant studies with some methodological shortcomings.
Data extraction
In realist review, data extraction may include descriptions and explanations of how and why the programme theory may have worked in particular contexts [28]. We will use two approaches to extract data from selected studies. Firstly, an electronic data extraction sheet will be used in order to record study identification details (authors, title, publication, etc.), geographical area in which the study was conducted, specific population and methodology used. Comments on the rigour and trustworthiness of the study will also be included. Secondly, we will import the selected papers into NVivo and code the results and discussions sections in order to identify context, mechanism, outcome configurations in the findings. Three members of the research team will undertake data extraction, and cross checking will be undertaken to identify any inconsistencies, inaccuracies or oversights. Any discrepancies will be discussed and resolved among the core research team with reference to the wider research team if necessary.
Data analysis and synthesis
Analysis and synthesis will proceed in NVivo as we identify recurring relationships between contexts, mechanisms and outcomes in the selected papers. This process will be guided by programme and substantive theories. We will look for predictable patterns (demi-regularities) to determine how similar mechanisms act in different contexts to generate outcomes. Emerging findings will be challenged and contrary examples will be sought in the data and in theory. This process will allow information on outcomes that differ in comparable circumstances, for contradictory outcomes to occur in particular contexts, and for judgements of the strength/weaknesses of research methods to be integrated into the synthesis. These findings will feedback into theory refinement. The following conceptual tools will be applied during this phase [30];
-
juxtaposing (“for instance, when one study provides the process data to make sense of the outcome pattern noted in another”)
-
reconciling (identifying differences which explain apparently contradictory sets of findings)
-
adjudicating between studies (based on the quality of research);
-
consolidating (building ‘multi‐faceted explanations of success’)
-
situating (‘this mechanism in context A, that one in context B”)
We will adopt an iterative and explanatory approach to synthesis of the data. The findings will be synthesised to be of practical use and will be reported according to the RAMESES reporting standards for realist syntheses [28].