- Open Access
A review protocol on research partnerships: a Coordinated Multicenter Team approach
Systematic Reviews volume 7, Article number: 217 (2018)
Research partnership approaches, in which researchers and stakeholders work together collaboratively on a research project, are an important component of research, knowledge translation, and implementation. Despite their growing use, a comprehensive understanding of the principles, strategies, outcomes, and impacts of different types of research partnerships is lacking. Generating high-quality evidence in this area is challenging due to the breadth and diversity of relevant literature. We established a Coordinated Multicenter Team approach to identify and synthesize the partnership literature and better understand the evidence base. This review protocol outlines an innovative approach to locating, reviewing, and synthesizing the literature on research partnerships.
Five reviews pertaining to research partnerships are proposed. The Coordinated Multicenter Team developed a consensus-driven conceptual framework to guide the reviews. First, a review of reviews will comparatively describe and synthesize key domains (principles, strategies, outcomes, and impacts) for different research partnership approaches, within and beyond health (e.g., integrated knowledge translation, participatory action research). After identifying commonly used search terminology, three complementary scoping reviews will describe and synthesize these domains in the health research partnership literature. Finally, an umbrella review will amalgamate and reflect on the collective findings and identify research gaps and future directions. We will develop a collaborative review methodology, comprising search strategy efficiencies, terminology standardization, and the division of screening, extraction, and synthesis to optimize feasibility and literature capture. A series of synthesis and scoping manuscripts will emerge from this Coordinated Multicenter Team approach.
Comprehensively describing and differentiating research partnership terminology and its domains will address well-documented gaps in the literature. These efforts will contribute to and improve the quality, conduct, and reporting of research partnership literature. The collaborative review methodology will help identify and establish common terms, leverage efficiencies (e.g., expertise, experience, search and protocol design, resources) and optimize research feasibility and quality. Our approach allows for enhanced scope and inclusivity of all research user groups and domains, thereby contributing uniquely to the literature. This multicenter, efficiency and quality-focused approach may serve to inspire researchers across the globe in addressing similar domain challenges, as exist in this rapidly expanding field.
Research partnership approaches, in which researchers and stakeholders work together collaboratively on a research project, are an important component of research, knowledge translation, and implementation [1,2,3,4]. These approaches are becoming increasingly popular, as efforts to ensconce stakeholder engagement within health care research, implementation, and improvement work converge, and are prioritized by health care systems, research funders, government, and other organizations . In particular, the active integration of patients and patient-identified priorities into the research process [5,6,7] has become much more frequent and, in many cases, is now a mandated expectation of research teams [5, 8,9,10]. Research partnership approaches align well with efforts to enhance participant empowerment , elevate disenfranchised voices , and engage in real-world solution finding  to improve research relevance and impact [14,15,16,17].
Over the last half of a century or more, research partnership approaches have evolved within multiple research domains. A number of these approaches can be differentiated by important similarities and differences (e.g., integrated knowledge translation (IKT), participatory research, co-production, participatory action research (PAR), engaged scholarship, Mode 2 knowledge production) [14, 15, 17,18,19]. This variability of approach and terminology presents considerable challenges for synthesis research in the field of research partnerships, particularly in the sub-field of IKT. The dispersion and variation of relevant literature is daunting, both scientifically and logistically, and in many cases precludes attempts at more exhaustive reviews [3, 15, 20, 21]. Research partnership terminology [22,23,24] and definitions [15, 25] vary significantly by discipline and are still actively evolving, making IKT and other research partnership approaches difficult to capture conceptually [1,2,3, 21, 25,26,27,28,29,30,31,32,33].
IKTFootnote 1 , in particular, has been compared and contrasted to other types of research partnership approaches. To illustrate, Salsberg and Merati , Salsberg , Jull and colleagues , and Bowen  highlight important comparisons between IKT and participatory health research: IKT and PAR, IKT and community-based participatory research (CBPR), and engaged scholarship and participatory research, respectively. However, syntheses conducted in this area to date highlight considerable limitations and challenges, such as the use of scope control techniques and the amenability of reported data for extraction and synthesis [1, 2, 21, 35,36,37].
In preparing this protocol, we were unable to identify a single synthesis that located, described, compared, or evaluated the literature pertaining to both IKT and related partnership research approaches within the health domain, or beyond. We identified a single review examining empirically evaluated IKT studies , and several syntheses focused on individual types of partnerships and relevant domains [29, 37,38,39]. No synthesis that comparatively described principles, strategies, outcomes, and impacts in different types of research partnership approaches was identified. The implications of these findings are significant. The co-existence of multiple, potentially relevant evidence domains is well-recognized; yet, this evidence remains largely disconnected and is often viewed superficially, or within disciplines alone. We believe that extending the comparative analysis to other domains (principles, strategies, outcomes, impacts) may help researchers more deliberately apply and rigorously evaluate research partnership approaches in future. Comparative analytics examining how and why IKT and other research partnership approaches work, the key domains (principles, strategies, outcomes, impacts), and the contextual conditions under which these approaches function may allow more deliberate and efficient stakeholder engagement [1, 21] and would represent a major step forward in the design, conduct, assessment, and impact of IKT and other research partnership approaches, in real-world settings.
This protocol describes the work plan of a newly established Coordinated Multicenter Team, focused on optimizing the quality and efficiency of IKT and other research partnership syntheses. The team has a specific interest in IKT [20, 40, 41] and applied this lens in the design of the proposed studies. Using a collaborative approach, we will build consensus strategies to address common challenges (e.g., terminology, definitions, conceptual similarities/differences, evidence volume and dispersion, logistics/resource and feasibility issues) faced by researchers attempting to synthesize the research partnership literature, including the sub-field of IKT. The three main aims of this study are to:
Systematically scope the literature and comparatively describe and synthesize principles, strategies, outcomes, and impacts reported in different types of research partnership approaches within and beyond health;
Describe and synthesize the principles, strategies, outcomes, and impacts and the accompanying research methods and tools reported in different types of health research partnership studies; and
Amalgamate and reflect on the collective findings and identify research gaps and future directions.
This review protocol describes a Coordinated Multicenter Team approach to reviewing and synthesizing the key domains in different types of research partnership approaches. This work will contribute to broadening and deepening the the evidence base for research partnerships and practice.
Coordinated Multicenter Team
A Coordinated Multicenter Team approach to plan, execute, assess, and report the proposed research syntheses will be used. The team is spread geographically and comprises clustered, multicenter teams working on complementary themes and projects. This approach will create resource and time efficiency, high productivity, and effectiveness and enhance methodological, logistical, and reporting quality within this area of the research literature. The Coordinated Multicenter Team comprises currently nine individuals (KJM, FH, KMS, TN, MVD, CJN, LKC, IDG, HG) across six academic and healthcare centers (University of Calgary, Alberta Health Services, University of British Columbia Okanagan, University of Manitoba, Ottawa Hospital Research Institute, University of Ottawa). Our work is embedded within an international integrated knowledge translation research network  established to systematically study and advance what is known and documented about IKT.
Engagement of stakeholders in the proposed studies
Our Coordinated Multicenter Team will both study and employ an IKT approach in the proposed research . Team members work with several stakeholder groups who have a vested interest in improving the science of IKT, including patients, in particular. A steering committee, consisting of a diverse representation of stakeholders (e.g., patients, policy and decision-makers, healthcare professionals, researchers), will be established for each individual review (see Appendix 1). Committee members will be actively involved in reviews according to their needs and preferences and according to the specific needs of each review [44,45,46]. At a minimum, the Coordinated Multicenter Team will engage its stakeholders in the following research phases:
Conceptual design and formulation of the research questions
Before starting data extraction
Data analysis, interpretation, and dissemination of results
Scoping practices outlined by Arksey, O’Malley, and other colleagues [47,48,49,50,51] guide our work to identify and describe the research questions, identify and select studies, abstract, collate, synthesize, and validate findings. Given the diversity of terminology and the dispersion of this literature, we will synthesize in three steps (Fig. 1). First, we will start broadly by conducting a review of reviews to comparatively describe and synthesize key domains (principles, strategies, outcomes, and impacts) for different research partnership approaches, within and beyond health (step 1). In this first step, we will identify the research partnership terminology and research scope in different practice domains in order to optimize our search strategies for subsequent steps. Secondly, using a more refined set of search strategy terms informed by the review of reviews, we will conduct several scoping reviews to describe and synthesize each key domain further in the health research partnership literature (step 2). Finally, we will amalgamate and reflect on the findings of all reviews conducted in the previous steps, using an umbrella review, to draw overarching conclusions, describe our collaborative approach and future directions, and contribute to the research agenda (step 3). A series of scoping and synthesis manuscripts will emerge a review of reviews (1a), three scoping reviews (2a–c), and one overarching umbrella review (3a).
The planning, execution, evaluation, and reporting of all reviews and findings will be guided by the Cochrane Collaboration Handbook of Systematic Reviews , the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) , Preferred Reporting Items for Systematic Review and Meta-Analysis Equity (PRISMA-E) , and the emergent PRISMA-ScR for Scoping Reviews , and recent guidance from Pollock and colleagues for the conduct of overviews of reviews . Details about the compliance with the PRISMA-P guidelines are described in Additional file 1.
Guiding conceptual framework
As part of protocol planning, the Coordinated Multicenter Team developed a consensus-based focused conceptual framework to guide its work (Fig. 2). Three authors (KJM, FH, HG) developed a first draft of the guiding framework based on early research questions and the PICOS for each individual review. The content of the framework was discussed with all members of the Coordinated Multicenter Team at several team meetings and revised iteratively until consensus was reached. The framework defines the topic of interest, describes key domains of research partnerships, and captures the overall intended scope and outcomes of the Coordinated Multicenter Team research agenda. This includes four key domains (principles, strategies, outcomes, impacts), and each of these domains will be assessed in terms of their research methods, methodologies and/or tools. Finally, both functionally and conceptually, we anticipate that the nature of each of the proposed domains will be heavily influenced by context.
For the purpose of this review, we will use the following operational terms and definitions:
Research partnerships: “individuals, groups, or organizations engaged in collaborative research activity involving at least one researcher (e.g., individual affiliated with an academic institution) and any stakeholder actively engaged in any part of the research process (e.g., decision or policy maker, health care administrator or leader, community agency, charities, network, patients etc.)” [1, 57]. Examples of research partnership approaches include, but are not limited to, IKT, participatory research, and participatory action research.
Principles: “fundamental norms, rules, or values that represent what is desirable and positive for a person, group, organization, or community and help it in determining the rightfulness or wrongfulness of its actions. Principles are more basic than policy and objectives and are meant to govern both” .
Strategies: “observable actions designed to achieve an outcome” .
Outcomes: “a planned, a priori assessment described in the study methods that is used to determine a change in status as a result of interventions, can be measured or assessed as a component of the study, and is not something of futuristic benefit”. (Adapted from the University of Waterloo Research Ethics—Definition of Outcome) .
Impact: “identifiable benefit to, or positive influence on, the economy, society, public services, health, the environment, quality of life, or academia” .
Method: “the techniques or procedures used to gather and analyze data related to some research question or hypothesis” .
Methodology: “the strategy, plan of action, process, or design underlying behind the choice and use of particular methods and the choice and use of methods to the desired outcomes” .
Tools: “an instrument (e.g., survey, measures, assessments, questionnaire, inventory, checklist, metrics, indicators, list of factors, subscales, or similar) that can be used to assess/evaluate the elements or domains of an IKT or health research partnership” .
Context: defined as “the physical, organizational, institutional, and legislative structures that enable and constrain, and resource and realize, people and procedures” .
Facilitators: “single or multilevel factors that are positively associated with or enhance IKT or research partnership and/or its definition, conceptualization, establishment, or conduct, design, assessment, or impact” 
Barriers: “single or multilevel factors that are negatively associated with or hinder IKT or research partnership and/or its definition, conceptualization, establishment, or conduct, design, assessment, or impact” .
1a) What differences and similarities can be identified in reported principles, strategies, outcomes, and impacts among different health and non-health research partnership approaches?
The primary research questions for scoping reviews 2a–c are as follows:
2a) What principles and strategies are used to guide the different types of health research partnerships?
2b) What are the reported outcomes and impacts of the different types of health research partnerships, and what are the available measurement tools for assessing the outcomes and impact?
2c) What research methodologies and methods have been used to explicitly study or evaluate the partnering process underpinning health research partnership?
The primary research questions for the overarching umbrella review 3 are as follows:
3a) What do we currently know about principles, strategies, outcomes, and impacts in the context of research partnership approaches? What are the research gaps in the literature on research partnership approaches? What are the next steps that should be taken in the field of research partnerships?
Secondary research questions for each individual scoping review are described in Appendix 2.
Step 1: Review of reviews
In consultation with our collaborating academic librarians (MVD, CJN), the Coordinated Multicenter Team developed a search strategy centered on capturing the following key concepts: partnership research, participatory research, knowledge translation, and knowledge transfer. We opted not to use controlled vocabularies given that preliminary inquiries confirmed poor capture of this literature by traditional health indexing. Controlled vocabulary would adversely impact precision and inflate recall [22, 23, 64].
There is great diversity in the terminologies used to express concepts associated with different types of research partnership. However, we will work on the assumption that review papers, as they synthesize existing knowledge, will use standardized terminology in their titles, abstracts, and keywords, allowing for a less complex, but still comprehensive strategy. The search will be piloted in four health databases (MEDLINE, Embase, CINAHL, and PsycINFO) to evaluate scope and feasibility. Appendix 3 describes an example of the search strategy in MEDLINE. Further refinement of this strategy will be based on our findings during an initial screening process. The refined search strategy will then be used to search a wider range of disciplines. Ultimately, the review will describe the terminology required for our subsequent reviews and allow us to identify a list of “gold standard” articles, ensuring scoping review search strategies are effective. The final search strategies for all individual databases will be available via the Open Science Framework [65, 66].
Electronic data sources
We will search for review papers within and beyond the health domain using the following electronic databases: MEDLINE, Embase, CINAHL, PsycINFO, ERIC, Education Source, Social Services Abstracts, Sociological Abstracts, Sociology Database, Applied Social Sciences Index and Abstracts, Web of Science Core Collection, and JSTOR.
Screening process and data extraction
The search will be executed by our academic librarian (MVD), and the results managed using Endnote™ X.7.5.3. De-duplication of search findings will be done according to Bramer’s method . De-duplicated results will be imported into Rayyan, a web-based tool designed to facilitate the screening process of literature reviews . Prior to the actual screening process, we will choose a random sample (5%) of citations to conduct calibration screening. Two members of the Coordinated Multicenter Team (FH, KJM) will take part and review the same set of citations independently. We will calculate inter-rater agreement using the kappa statistic and start the screening process once a kappa ≥ 0.6 is achieved. Where discrepancies arise, these will be discussed and resolved by consensus or failing agreement, referred to a third team member for a final decision.
The screening process will be conducted in three separate rounds. In the first round, both members will screen all citations on the title only, independently and in duplicate. Citations included by at least one team member will pass title screening and will be stored in a new database for the next screening round. In the second round, both members will screen all citations on title and abstract, independently and in duplicate, guided by the following eligibility criteria: included articles must (a) describe a literature overview of research partnerships according to our definition, (b) describe a systematic search of the literature including search terms and databases, and (c) be published in the English language. Articles that (a) do not meet our definition of research partnership and/or (b) describe a review of a method or tool instead of literature overview will be excluded and the reasons for exclusion categorized. The development of search strategies for each subsequent scoping review will be informed by the classifications formulated by this review (step 2). The third round will involve gathering full-text versions of all citations meeting eligibility criteria. Using the previously described screening calibration process, the same two team members will screen full-text review papers independently and in duplicate, discussing discrepancies to consensus or failing consensus, referring them to a third team member for a final decision. Full-text screening will then be performed based on the specific eligibility criteria aligned with our research questions (see Table 1 and Appendix 2 for more details). Once a final set of eligible review papers is generated, data extraction from full-text review papers will proceed, independently, and in duplicate, using a pre-tested data extraction tool in MS Excel. The extractable data (e.g., principles, strategies, outcomes, impacts) will be summarized for different types of research partnership approaches (e.g., IKT, CBPR, PAR). Strategies to determine the risk of bias and the methodological quality of the included review papers will be developed using published guidance [56, 69, 70].
Step 2: Scoping reviews
Building on the findings from the review of reviews (step 1), the Coordinated Multicenter Team will refine the health research partnership search strategy. The review of reviews will be used to identify relevant terminology and definitions used in different types of health research partnership, allowing for the development of a high-quality, evidence-informed search strategy for each scoping review. The review of reviews will also supplement the creation of a “gold standard” list of articles to test subsequent searches against. This approach will require intense collaborative effort and multiple strategy alignments; we anticipate the output will generate a comprehensive, well-defined body of literature amenable to multiple reviews that are focused on specific aspects of different types of health research partnerships. The search process will be facilitated by the team’s academic librarian (MVD). For each scoping review (step 2, reviews 2a–c), three individual search strategies will be developed, to ensure reproducibility and feasibility. Search strategies will consist of two parts: (1) an overarching segment to identify different types of health research partnerships and (2) protocol-specific part(s) identifying and/or modifying the focus of each scoping review as per domains of the guiding framework (e.g., principles, strategies, outcomes, impacts). The first part of each search strategy will be the same for every scoping review (identification of research partnerships search terms), and the second part will be customized to match review-specific research questions. Strategies will be piloted in MEDLINE to determine scope and feasibility limitations and anticipate resource requirements. To optimize search quality and comprehensiveness and to refine the balance between search sensitivity and scope feasibility, draft search strategies will be scrutinized by a second academic librarian using the Peer Review of Electronic Search Strategies (PRESS) checklist [71, 72]. The team will review and consider the suggestions and make final edits to the strategy as necessary. All final search strategies for the individual databases will be available through Open Science Framework .
Electronic data sources
All scoping reviews will search for articles by using the following four electronic health databases: MEDLINE, Embase, CIHNAL, and PsycINFO. Decisions regarding additional refinements to data sources (e.g., time span, grey literature) will be specific to each scoping review and informed by the review of reviews findings.
The Coordinated Multicenter Team search will be executed by an academic librarian (MVD), and results managed and de-duplicated using Endnote™ X.7.5.3, as described previously. Title and abstract screening will be performed for each scoping review separately using a pre-tested MS Excel screening tool. Screening calibration will be undertaken in two stages, by title and abstract, and then in full text, using the methods described previously (step 1), and will be carried out independently and in duplicate by two team members. Discrepancies will be discussed and resolved by screener consensus or referred to a third investigator for final resolution.
To maximize quality and ensure comparability within a very large volume of literature, we formulated general eligibility criteria for all scoping reviews for use with title and abstract level screening. We will combine these general eligibility criteria with review-specific criteria. We will include citations that involve research partnerships in the health domain. We will exclude articles that do not meet the definition of research partnership. For all excluded studies, we will track primary reasons for exclusion. After title and abstract screening, each scoping review team will proceed with the full-text screening process. Once a final set of eligible papers is generated, data extraction from full-text papers will proceed.
Screening calibration within each scoping review will be undertaken at each screening level as described previously. A priori agreement on common terms and definitions was achieved and will be applied during all study levels.
Aspects related to study records (risk of bias) and data (synthesis, meta-bias, confidence in cumulative evidence) will be tailored to each individual scoping review paper. Details about these aspects will be described in the individual review papers.
Step 3: Overarching umbrella review
Finally, two researchers (HG and KS) will synthesize and aggregate findings from the review of reviews and the three scoping reviews using an umbrella review, in collaboration with review leads (FH, KJM, TN). In accordance with published guidelines for developing, conducting, and reporting umbrella reviews , this review will synthesize findings from the multiple reviews into one accessible and useable document. 
This review protocol outlines our Coordinated Multicenter Team approach to reviewing and synthesizing research partnerships using an innovative and collaborative review methodology. Our approach will result in a series of review manuscripts describing specific aspects of different types of research partnerships and attempt to address the documented gaps, with a specific focus and interest in IKT [20, 40, 41]. By documenting our Coordinated Multicenter Team approach, we hope to provide guidance to and inspire researchers in the same or other fields to tackle evidence bases that are challenged by scope, terminology, dispersion, and volume.
Our Coordinated Multicenter Team approach is centralized around three key aspects. First, we will optimize research quality by sharing knowledge and expertise among all team members. Our team currently consists of nine individuals with different backgrounds and expertise (e.g., KT, implementation, IKT, behavioral science, research partnerships, knowledge synthesis), working in six different organizations across Canada. This provides a unique opportunity to learn in collaboration and raise the quality and integrity of multiple reviews. All papers published by the Coordinated Multicenter Team will use common terms and language based on our consensus-driven and literature-based guiding framework and related terms and definitions and reporting criteria , unless otherwise noted (Fig. 2). The Coordinated Multicenter Team papers may be used as a template for future research to conduct studies and report on different types of research partnership (e.g., we know of at least two similarly structured systematic reviews that will cascade from this first proposed set of reviews). Our search strategies will be publicly available [65, 66], giving other researchers the opportunity to use our searches and build upon our work in refining terms and in locating and describing the nature of the evidence base for IKT and other types of research partnership approaches. In this way, we hope that our work will enhance research quality and transparency in the field of IKT and other research partnership approaches by creating a common language for reporting and planning.
Second, we will increase capacity by maximizing synthesis team efficiency in all stages of the review processes (e.g., search strategy development, screening process, procedural alignments for screening, extraction, dissemination). For example, we will use our findings from the review of reviews to develop an overarching research partnership-focused search strategy that will be applied in all our scoping reviews. We expect that a search strategy built upon terms and definitions comprising the breadth of the literature pertaining to research partnership will result in more focused results and will improve feasibility with well-justified search strategy controls. Syntheses can pose significant time, resource, and volume challenges  in fields where there is a high diversity of terminology, procedures, and literature dispersion. Our efficiency- and quality-focused collaborative review methodology offers potential strategies to overcome these challenges and may therefore contribute to the literature on review methodologies addressing efficiency and quality improvement [76,77,78]. Moreover, this approach allows for an enhanced scope for each review and enhances inclusivity of all research user groups and domains, thus contributing uniquely to the literature and reducing the potential for duplication of efforts.
Third, we hope to maximize the impact of our work by ensuring that our projects are relevant and usable to a broad audience by using an IKT approach tailored to each individual review project [43,44,45,46]. We will establish steering committees consisting of a diverse group of stakeholders for each individual review and engage them throughout the review processes (Appendix 1). Moreover, we will reflect on our own IKT approach and will share the lessons learned in the overarching umbrella review. Our Coordinated Multicenter Team approach will, therefore, meet the needs of our partners and ensure that both researchers and stakeholders can benefit from our work.
In summary, our protocol paper provides a methodological design template for future researchers to construct their own reviews or research. It contributes to the methodological refinement of review processes using multi-site collaborative teams, in which design, workflow, scientific and logistical strategy, and other efficiencies are leveraged to optimize research quality. Ultimately, we hope our efforts will contribute to and improve the quality, conduct, and reporting of the research partnership literature. Our Coordinated Multicenter Team approach may serve to inspire researchers across the globe in addressing similar domain challenges, as exist in this rapidly expanding field.
We operationalize IKT using a recent iteration of an earlier definition  as:
“…a way of approaching research to increase the chances that the results will be applicable to the population under study. [IKT] is a paradigm shift that focuses on engagement with end users and the context in which they work. Essentially, it is a collaborative way of conducting research that involves researchers and knowledge-users, sometimes from multiple communities (e.g. clinicians, managers, policy makers, patients, [among others]) working together as partners in the research process.” (Graham, Tetroe & MacLean, 2014, p.11) .
Community-based participatory research
Integrated knowledge translation
Participatory action research
Drahota A, Meza RD, Brikho B, Naaf M, Estabillo JA, Gomez ED, Vejnoska SF, Dufek S, Stahmer AC, Aarons GA. Community-academic partnerships: a systematic review of the state of the literature and recommendations for future research. Milbank Q. 2016;94:163–214.
Camden C, Shikako-Thomas K, Nguyen T, Graham E, Thomas A, Sprung J, Morris C, Russell DJ. Engaging stakeholders in rehabilitation research: a scoping review of strategies used in partnerships and evaluation of impacts. Disability & Rehabilitation. 2015;37:1390–400. https://doi.org/10.3109/09638288.2014.963705.
Jagosh J, Macaulay AC, Pluye P, Salsbert J, Bush PL, Henderson J, Greenhalgh T. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Millbank Quarterly. 2012;90:311–46.
Goodman MS, Sanders Thompson VL. The science of stakeholder engagement in research: classification, implementation and evaluation. Translational Behavioral Medicine. 2017;7:486–91. https://doi.org/10.1007/s13142-017-0495-z.
Canadian Institute for Health Research. Strategy for Patient Oriented Research (SPOR). Ottawa: Canadian Institutes for Health Research; 2018. cited 2018 March 7]. Available from: http://www.cihr-irsc.gc.ca/e/41204.html
Sofolahan-Oladeinde Y, Mullins CD, Baquet CR. Using community-based participatory research in patent-centered outcomes research to address health disparities in under-represented communities. Journal of Comparative Effectiveness Research. 2015;4:515.
World Health Organization. World Health Organization Alma Ata Declaration. Geneva: World Health Organization; 1978.
Patient-Centered Outcomes Research Institute (PCORI). Engagement: influencing the culture of research. Washington: PCORI; 2018 [March 21, 2018]. Available from: https://www.pcori.org/engagement.
Alberta Innovates Health Solutions (AIHS). Funding opportunities and programs. Edmonton: Alberta Innovates Health Solutions; 2018 [cited 2018 March 20, 2018]. Available from: http://www.aihealthsolutions.ca/funding/health-research-funding/.
Auckland S. BRC Guidance: involving users in research. London: Guy's and Thomas' NHS Foundation Trust; 2010.
World Health Organization. Ninth futures forum on health systems governance and public participation. Copenhagen: World Health Organization; 2006.
O'Mara-Eves A, Brunton G, McDaid D, Oliver S, Kavanaugh J, Jamal F, Matosevic T, Harden A, Thomas J. Community engagement to reduce inequalities in health: a systematic review, meta-analysis and economic analysis. Public Health Research. 2013;1:4. https://doi.org/10.3310/phr01040.
Mockford C, Staniszewska S, Griffiths F, Herron-Marx S. The impact of patient and public involvement on UK NHS health care: a systematic review. International Journal of Quality in Health Care. 2012;24:28–38.
Bowen S, Graham ID. Backwards design or looking sideways? Knowledge translation in the real world: comment on “a call for a backward design to knowledge translation”. International Journal of Health Policy and Management. 2015;4:545–7.
Jull J, Giles A, Graham ID. Community-based participatory research and integrated knowledge translation: advancing the co-creation of knowledge. Implement Sci. 2017;12:1–9.
Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, Suleman R. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect. 2012;17:637–50.
Rycroft-Malone J, Burton CR, Bucknall T, Graham ID, Hutchinson AM, Stacey D. Collaboration and co-production of knowledge in healthcare: opportunities and challenges. International Journal of Health Policy and Management. 2016;5:221–3.
Salsberg J, Maccaulay AC, Parry D. Chapter 2: guide to integrated knowledge translation research: researcher and knowledge-user collaboration in health research. In: Graham JMT ID, Pearson A, editors. Turning knowledge into action: practical guidance on how to do integrated knowledge translation research. Lippincott-Joanna Briggs Institute Synthesis Science in Healthcare Series: Book 21. Philadelphia: Lippincott Williams & Wilkins; 2014.
Salsberg J, Merati N. Participatory Health Research in North America: from community engagement to evidence-informed practice. In: Kongats K, Michael TW, editors. Participatory Health Research: voices from around the world (in press). San Fransisco: Springer; 2018.
Gagliardi AR, Kothari A, Graham ID. Research agenda for integrated knowledge translation (IKT) in healthcare: what we know and do not yet know. J Epidemiol Community Health. 2017;71:105–6. https://doi.org/10.1136/jech-2016-207743.
Gagliardi A, Berta W, Kothari A, Boyko J, Urquhart R. Integrated knowledge translation (iKT) in health care: a scoping review. Implement Sci. 2016;11:1–12. https://doi.org/10.1186/s13012-016-0399-1.
McKibbon KA, Lokker C, Wilczynski NL, Ciliska D, Dobbins M, Davis DA, Straus SE. A cross-sectional study of the number and frequency of terms used to refer to knowledge translation in a body of literature in 2006: a tower of babel? Implement Sci. 2010;5:16. https://doi.org/10.1186/1748-5908-5-16.
McKibbon KA, Lokker C, Wilczynski NL, Haynes RB, Ciliska D, Dobbins M, Davis DA, Straus SE. Search filters can find some but not all knowledge translation articles in MEDLINE: an analytic survey. J Clin Epidemiol. 2012;65:651–9.
Graham ID, Tetroe, JM., Robinson, N., Grimshaw, J., and the International Funders Study Research Group,. An international study of health research funding agencies' support and promotion of knowledge translation. Academy of Health Annual Research Meeting; Boston 2005.
Granner ML, Sharpe PA. Evaluating community coalition characteristics and functioning: a summary of measurement tools. Health Education Research - Theory and Practice. 2004;19:514–32.
Israel BA, Schulz AJ, Parker EA, Becker AB. Review of community-based research: assessing partnership approaches to improve public heath. Annu Rev Public Health. 1998;19:173–202.
Zakocs RE, Edwards EM. What explains community coalition effectiveness? A review of the literature. Am J Prev Med. 2006;30:351–61.
Seaton CL, Holm N, Bottorff JL, Jones-Bricker M, Errey S, Caperchione CM, Lamont S, Johnson ST, Healy T. Factors that impact the success of interorganizational health promotion collaborations: a scoping review. Am J Health Promot. 2017:1–15.
Voorberg WH, Bekkers VJ, Tummers LG. A systematic review of co-creation and co-production: embarking on the social innovation journey. Public Management Review. 2015;17:1333–57.
Walters SJ, Stern C, Robertson-Malt S. The measurement of collaboration within healthcare settings: a systematic review of measurement properties of instruments. JBI Database of Systematic Reviews and Implementation Reports. 2016:138–97. https://doi.org/10.11124/JBISRIR-2016-2159.
LeClercq T, Hammedi W, Poncin I. Ten years of value cocreation: an integrative review. Rech Appl Mark. 2016;31:26–60.
Stolp S, Bottorff JL, Seaton CL, Jones-Bricker M, Oliffe JL, Johnson ST, Errey S, Medhurst K, Lamont S. Measurement and evluation practices of factors that contribute to effective health promotion collaboration functioning: a scoping review. Evaluation and Program Planning. 2016;61:38–44.
Gradinger G, Britten N, Wyatt K, Froggatt K, Gibson A, Jacoby A, Lobban F, Mayes D, Snape D, Rawcliffe T, Popay J. Values associated with public involvement in health and social care research: a narrative review. Health Expect. 2013;18:661–75.
Bowen S. The relationship between engaged scholarship, knowledge translation and participatory research. In: Liamputtong GHAP, editor. Participatory qualitative research methodologies in health. Los Angeles: SAGE; 2015. p. 183–99.
Greenhalgh T, Jackson C, Shaw S, Janamian T. Achieving research impact through co-creation in community-based health services: literature review and case study. Milbank Q. 2016;94:392–429.
Smith KE, Bambra C, Joyce KE, Perkins N, Hunter DJ, Blenkinsopp EA. Partners in health? A systematic review of the impact of organizational partnerships on public health outcomes in England between 1997-2008. J Public Health. 2009;31:210–21.
Roussos ST, Fawcett SB. A review of collaborative partnerships as a strategy for improving community health. Annu Rev Public Health. 2000;21:369–402.
Varda D, Shoup J, Miller S. A systematic review of collaboration and network research in the public affairs literature: implications for public health practice and research. Research and Practice. 2012;102:564–71.
Rycroft-Malone J, Burton C, Wilkinson J, Harvey G, McCormack B, Baker R, Dopson S, Graham I, Staniszewska S, Thompson C, Ariss S, Melville-Richards L, Williams L. Collective action for knowledge mobilisation: a realist evaluation of the Collaborations for Leadership in Applied Health Research and Care. NIHR Journals Library: Southhampton; 2015.
Kothari A, McCutcheon C, Graham ID, for the iKT Research Network. Defining integrated knowledge translation and moving forward: a reponse to recent commentaries. International Journal of Health Policy and Management. 2017;6:1–2.
Graham ID, Kothari A, McCutcheon C, and the Integrated Knowledge Translation Research Network Project Leads. Moving knowledge into action for more effective practice, programmes and policy: protocol for a research programme on integrated knowledge translation. Implementation Science. 2018;13. https://doi.org/10.1186/s13012-017-0700-y.
Graham ID. Moving knowledge into action for more effective practice, programs and policy: A research program focusing on integrated knowledge translation (Foundation Scheme: 2014 1st Live Pilot). [Grant Application]. In press 2015.
Canadian Institutes of Health Research (CIHR). Guide to knowledge translation planning at CIHR: integrated and end-of-grant approaches. Ottawa: Her Majesty the Queen In Right of Canada; 2012.
Domecq JP, Prutsky, G., Elriayah, T., Wang, Z., Habhan, M., Shippee, N., Brito, J.P., Boehmer, K., Hasan, R., Firwana, B., Erwin, P., Eton, D., Sloan, J., Montori, V., Asi, N., Abu Dabrh, AM., Hassan Murad, M. Patient engagement in research: a systematic review. BMC Health Serv Res 2014;14:1–9. doi: https://doi.org/10.1186/1472-6963-14-89.
Hyde C, Dunn KM, Higginbottom A, Chew-Graham CA. Process and impact of patient involvement in a systematic review of shared decision making in primary care consultations. Health Expect. 2017;20:298–308. https://doi.org/10.1111/hex.12458.
Harris J, Croot L, Thompson J, Springett J. How stakeholder participation can contribute to systematic reviews of complex interventions. J Epidemiol Community Health. 2015;70:207–14. https://doi.org/10.1136/jech-2015-205701.
Arksey H, O'Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology: Theory and Practice. 2005;8:19–32.
Daudt HM, van Mossel C, Scott SJ. Enhancing the scoping study methodology: a large, inter-professional team's experience with Arksey and O'Malley's framework. BMC Med Res Methodol. 2013;13:1–9.
Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5:1–9.
Colquhoun HI, Levac D, O'Brien KK, Straus S, Tricco AC, Perrier L, Kastner M, Moher D. Scoping reviews: time for clarity in defintion, methods and reporting. J Clin Epidemiol. 2014;67:1291–4. https://doi.org/10.1016/j.jclinepi.2014.03.013.
Tricco AC, Lillie E, Zarin W, O'Brien K, Colquhoun H, Kastner M, Levac D, Ng C, Pearson Sharpe J, Wilson K, Kenny M, Warren R, Wilson C, Stelfox HT, Straus SE. A scoping review on the conduct and reporting of scoping reviews. BMC Med Res Methodol. 2016;16:1–10. https://doi.org/10.1186/s12874-016-0116-4.
Higgins JP, & Green, S. Cochrane handbook for systematic reviews of interventions. 2011 [updated march 2011; cited 2015 November 5]. Available from: http://handbook.cochrane.org.
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, and PRISMA-P Group. Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 Statement. Systematic Reviews. 2015;4:1–9. https://doi.org/10.1186/2046-4053-4-1.
Welch V, Petticrew M, Tugwell P, Moher D, O'Neill J, Waters E, White H, and the PRISMA-Equity Bellagio group. PRISMA-Equity 2012 extention: reporting guidelines for systematic reviews with a focus on health equity. PLOS Medicine. 2012;9:e1001333. https://doi.org/10.1371/journal.pmed.1001333.
Tricco AC. Preferred Reporting Items for Systematic Reviews and Meta-Anaysis extension for Scoping Reviews (PRISMA-ScR) Toronto, Canada2015 [cited 2015 September 11, 215]. Available from: http://www.prisma-statement.org/Extensions/InDevelopment.aspx.
Pollock M, Fernandes RM, Becker LA, Featherstone R, Hartling L. What guidance is available for researchers conducting overviews of reviews of healthcare interventions? A scoping review and qualitative metasummary. Systematic Reviews. 2016;5:1–15.
Mrklas KJ. A scoping review of available tools for assessing integrated knowledge translation research or health research partnership impact. [PhD Dissertation]: University of Calgary; 2017.
Business Dictionary. Definition of principles: business dictionary; 2017 [cited 2017 retrieved December 5, 2017]. Available from: http://www.businessdictionary.com/definition/principles.html.
Oxford Dictionary. 2018. Definition for "strategies".
University of Waterloo. Research ethics: definition of a health outcome. Waterloo, ON: University of Waterloo; 2018 [cited 2018 March 7]. Available from: https://uwaterloo.ca/research/office-research-ethics/research-human-participants/pre-submission-and-training/human-research-guidelines-and-policies-alphabetical-list/definition-health-outcome.
Higher Education Funding Council for England: Research Excellence Framework 2014. Assessment framework and guidance on submissions 2011. Bristol, UK2014 [cited 14 Nov 2017]. Available from: http://www.ref.ac.uk/2014/media/ref/content/pub/assessmentframeworkandguidanceonsubmissions/GOS%20including%20addendum.pdf.
Crotty M. Introduction: the research process. In: (Ed). MC, editor. The foundations of social research: meaning and perspective in the research process. London: Sage; 1998. p. 1–17.
May C, Finch T, Mair F, Ballini L, Dowrick C, Eccles M, Gask L, MacFarlane A, Murray E, Rapley T, Rogers A, Treweek S, Wallace P, Anderson G, Burns J, Heaven B. Understanding the implementation of complex interventions in health care: the normalization process model. BMC Health Serv Res. 2007;7:1–7.
Lokker C, McKibbon KA, Wilczynski NI, Haynes RB, Ciliska D, Dobbins M, Davis DA, Straus SE. Finding knowledge translation articles in CINAHL. Studies in Health Technology & Informatics. 2010;160:1179–83.
Foster ED, Deardorff A. Open Science Framework (OSF). Journal of the Medical Library Association (JMLA). 2017;105:203–6. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5370619/pdf/jmla-105-203.pdf.
Hoekstra F, Mrklas KJ, Sibley K, Nguyen T, Vis-Dunbar M, Neilson C, Crockett L, Gainforth H, Graham ID. Understanding collaborative approaches to research: a synthesis of the research partnership literature; 2018. https://doi.org/10.17605/OSF.IO/GVR7Y.
Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in endnote. Journal of the Medical Library Association (JMLA). 2016;104:240–3. https://doi.org/10.3163/1536-5050.104.3.014.
Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan - a web and mobile app for systematic reviews. Systematic Reviews. 2016;5. https://doi.org/10.1186/s13643-016-0384-4.
Ballard M, Montgomery P. Risk of bias in overviews of reviews: a scoping review of methodological guidance and four-item checklist. Res Synth Methods. 2017;8:92–108. https://doi.org/10.1002/jrsm.1229.
Pollock A, Campbell P, Brunton G, Hunt H, Estcourt L. Selecting and implementing overview methods: implications from five exemplar overviews. Systematic Reviews. 2017;6:145. https://doi.org/10.1186/s13643-017-0534-3.
McGowan J, Sampson M, Lefebvre C. An evidence based checklist for the peer review of electronic search strategies (PRESS EBC). Evidence Based Library and Information Practice. 2010;5:149–54. https://doi.org/10.18438/B8SG8R.
McGowan J, Sampson M, Salzwedel D, Cogo E, Foerster V, Lefebvre C. Guideline statement: PRESS peer review of electronic search strategies 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6.
Aromataris E, Fernandez R, Godfrey CM, Holly C, Khalil K, Tungpunkom P. Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. International Journal of Evidence-Based Healthcare. 2015;13:132–40. https://doi.org/10.1097/XEB.0000000000000055.
Grant M, Booth J. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Inf Libr J. 2009;26:91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x.
Petticrew M, Roberts H. Systematic reviews in the social sciences: a practical guide. Malden: Blackwell Publishing Co.; 2006.
Petticrew M. Time to rethink the systematic review catechism? Moving from 'what works' to 'what happens'. Systematic Reviews. 2015;4:1–6. https://doi.org/10.1186/s13643-015-0027-1.
Uttley L, Montgomery P. The influence of the team in conducting a systematic review. Systematic Reviews. 2017;6:1–4. https://doi.org/10.1186/s13643-017-0548-x.
Tsertsvadze A, Chen Y, Moher D, Sutcliffe P, McCarthy N. How to conduct systematic reviews more expeditiously? Systematic Reviews. 2015;4:1–6. https://doi.org/10.1186/s13643-015-0147-7.
Canadian Institutes of Health Research. Knowledge translation at the CIHR. Ottawa: Canadian Institutes of Health Research; 2000 [cited 2018 March 8]. Available from: http://www.cihr-irsc.gc.ca/e/29418.html#2.
Graham ID, Tetroe JM, McLean RK. Chapter 1: Some Basics of Integrated Knowledge Translation Research. In: Graham ID, Tetroe JM, Pearson A, editors. Turning knowledge into action: practical guidance on how to do integrated knowledge translation research. Philadelphia: Lippincott Williams & Wilkins; 2014. p. 196.
This research is supported in part by the IKTR Network (Canadian Institutes of Health Research Foundation Grant (FDN #143237), a Canadian Institutes of Health Research Project Grant (iKT Project grant: 156372), a Michael Smith Foundation for Health Research Scholar Award (Scholar Award #16910), and by the International Collaboration on Repair Discoveries (F17-01540). KMS is supported by a Canada Research Chair in Integrated Knowledge Translation in Rehabilitation Sciences. TN holds a Postdoctoral Fellowship from the Canadian Institutes of Health Research (2016–2019). This manuscript is an amalgamation of research proposals funded by different funding bodies. None of the funding bodies had a role in writing this manuscript or will have a role in the collection, analysis, or interpretation of the data.
Availability of data and materials
Not applicable. However, the final search strategies for the individual databases that will be executed for each individual review will be available through the Open Science Framework. Other relevant information about the data screening and data extraction processes will be published as a supplementary material to future publications and/or will be available through the Open Science Framework.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix 1—Steering committees
A steering committee, consisting of a diverse representation of stakeholders (e.g., patients, policy and decision-makers, healthcare professionals, researchers), will be established for each individual review. This appendix provides an overview of the stakeholders who will be a member of one of the steering committees.
The following partners will take part in one of the steering committees:
Spinal Cord Injury Ontario
Spinal Cord Injury BC
Ontario SCI Solutions Alliance
Northern American Spinal Cord Injury Consumer Consortium
Rick Hansen Institute
International Collaboration on Repair Discoveries
Spinal Cord Injury Canada
Michael Smith Foundation for Health Research
Mrklas, KJ—PhD Supervisory Committee, Department of Community Health Sciences, Cumming School of Medicine, University of Calgary
Clinicians from Hamilton Health Sciences, Toronto Rehabilitation Institute—University Health Network, Seine River Physiotherapy
Patient research partners from the BC SUPPORT Unit, University of Calgary
Appendix 2—Compendium of Review Protocols
Review of reviews 1a—Comparing and synthesizing research partnership approaches
This review of reviews aims to identify differences and similarities in principles, strategies, outcomes, and impacts reported in different health and non-health research partnership approaches in order to improve the differentiation among different research partnership approaches.
Primary research question
What differences and similarities can be identified in reported principles, strategies, outcomes, and impacts among different health and non-health research partnership approaches?
Secondary research questions
What kind of health and non-health research partnership approaches can be distinguished?
What definitions are used for research partnerships?
What principles, strategies, outcomes, and impacts are reported in different health and non-health research partnership approaches?
What are the differences and similarities between research partnership within and beyond the health domain?
What theories, models, and frameworks are used to guide review papers on health and non-health research partnership approaches?
Scoping review 2a—Principles and strategies
This scoping review aims to identify key principles and strategies for different health research partnerships.
Primary research question
What principles and strategies are used to guide the different types of health research partnerships?
Secondary research questions
How can the identified strategies be linked to the identified principles?
What theories, models, and frameworks are used to guide the different types of health research partnerships? How do they relate to the guiding principles and strategies?
How do the guiding principles and strategies differ between stages in the research process (e.g., data collection, data analysis, interpretation of findings, disseminating of findings)?
How do the guiding principles and strategies differ between settings?
How do the guiding principles and strategies differ between populations?
How do the guiding principles and strategies differ between groups of research users (e.g., patients, practitioners, decision-makers, health organizations)?
What facilitators and barriers are associated with the guiding principles and strategies?
Scoping review 2b—Outcomes, impacts, and tools
This scoping review aims to establish the scope, nature, and location of the global literature pertaining to the reported outcomes and impacts of different health research partnerships and outcomes or impact assessment tools.
Primary research questions
What are the reported outcomes and impacts of the different types of health research partnerships?
What are the available measurement tools for assessing the outcomes and impacts of the different types of health research partnerships?
Secondary research questions
How are the different health research partnership outcomes or impact (and/or related concepts) defined and described?
What are the key characteristics and constructs of the different health research partnership outcomes or impact assessment tools?
What are the documented barriers, facilitators, and/or other documented influences on the development, assessment, or use of the different health research partnership outcome or impact assessment tools?
What emergent gaps exist in health research partnership outcome or impact assessment?
To what extent is the nature and scope of the evidence base in this area amenable to systematic review?
What future research questions arise in the literature on health research partnership outcomes or impact assessment?
Scoping review 2c—Research methodologies and methods
This scoping review aims to describe the research methodologies and methods used to study or evaluate the partnering process underpinning health research partnership.
Primary research question
What research methodologies and methods have been used to explicitly study or evaluate the partnering process underpinning health research partnership?
Secondary research questions
How is IKT defined in studies?
To what extent has the IKT approach been explicitly studied/evaluated in published literature?
In what contexts has the IKT approach been explicitly studied/evaluated?
Review 3—Overarching umbrella review
The final paper aims to synthesize the findings from the four reviews in terms of the guiding conceptual framework.
Primary research questions
What do we currently know about principles, strategies, outcomes, and impacts in the context of research partnership approaches? What are the research gaps in literature on research partnership approaches? What are the next steps that should be taken in the field of research partnerships?
Appendix 3—Search strategy
The example search strategy for the review of reviews in MEDLINE.
|1||(((partnership or participatory) adj2 research) or ((transfer or translation) adj knowledge) or ((“mode 2” or “mode two” or “mode II”) adj2 (knowledge or research)) or “linkage and exchange”).ti,ab,kw,kf.||4715|
|2||(partnership? or “community-based”).ti. and (research or assessment or measurement or outcomes or tools).ab,ti.||7484|
|3||review.pt. or (“scoping review*1” or “systematic review*1” or “review of the literature” or “literature review*1” or “umbrella review*1” or “review of reviews” or “mapping review*1” or “realist review*1” or “rapid review*1”).ab,ti.||2,447,648|
|4||1 or 2||11,025|
|5||3 and 4||983|
About this article
Cite this article
Hoekstra, F., Mrklas, K.J., Sibley, K.M. et al. A review protocol on research partnerships: a Coordinated Multicenter Team approach. Syst Rev 7, 217 (2018) doi:10.1186/s13643-018-0879-2
- Knowledge synthesis
- Multicenter study
- Collaborative research partnerships
- Integrated knowledge translation
- Community-based participatory research
- Stakeholder engagement
- Research principles and strategies
- Research outcomes and impact