Skip to main content
Download PDF

Co-production of a systematic review on decision coaching: a mixed methods case study within a review

Systematic Reviews volume 13, Article number: 149 (2024) Cite this article

This article has been updated

Abstract

Background

Co-production is a collaborative approach to prepare, plan, conduct, and apply research with those who will use or be impacted by research (knowledge users). Our team of knowledge users and researchers sought to conduct and evaluate co-production of a systematic review on decision coaching.

Methods

We conducted a mixed-methods case study within a review to describe team co-production of a systematic review. We used the Collaborative Research Framework to support an integrated knowledge translation approach to guide a team through the steps in co-production of a systematic review. The team agreed to conduct self-study as a study within a review to learn from belonging to a co-production research team. A core group that includes a patient partner developed and conducted the study within a review. Data sources were surveys and documents. The study coordinator administered surveys to determine participant preferred and actual levels of engagement, experiences, and perceptions. We included frequency counts, content, and document analysis.

Results

We describe co-production of a systematic review. Of 17 team members, 14 (82%) agreed to study participation and of those 12 (86%) provided data pre- and post-systematic review. Most participants identified as women (n = 9, 75.0%), researchers (n = 7, 58%), trainees (n = 4, 33%), and/or clinicians (n = 2, 17%) with two patient/caregiver partners (17%). The team self-organized study governance with an executive and Steering Committee and agreed on research co-production actions and strategies. Satisfaction for engagement in the 11 systematic review steps ranged from 75 to 92%, with one participant who did not respond to any of the questions (8%) for all. Participants reported positive experiences with team communication processes (n = 12, 100%), collaboration (n = 12, 100%), and negotiation (n = 10–12, 83–100%). Participants perceived the systematic review as co-produced (n = 12, 100%) with collaborative (n = 8, 67%) and engagement activities to characterize co-production (n = 8, 67%). Participants indicated that they would not change the co-production approach (n = 8, 66%). Five participants (42%) reported team logistics challenges and four (33%) were unaware of challenges.

Conclusions

Our results indicate that it is feasible to use an integrated knowledge translation approach to conduct a systematic review. We demonstrate the importance of a relational approach to research co-production, and that it is essential to plan and actively support team engagement in the research lifecycle.

Peer Review reports

Introduction

Well-conducted and rigorous research can contribute to more effective, safe, appropriate, and sustainable health services to strengthen health systems and ultimately promote healthy societies [1, 2]; however, these potential benefits are often not realized [3,4,5]. In response, there has been growing support to include those who will use or be impacted by research such as patients and families, healthcare providers, health systems managers, policy makers, and other healthcare systems users (known hereafter as “knowledge users”) in research [6, 7]. Knowledge users work with researchers to co-produce knowledge and select outcomes that reflect a knowledge democracy [8, 9].

We adopt the term “co-production” in research as a collaborative approach across the research lifecycle that responds to the needs of knowledge users [6, 10, 11]. A synthesis of frameworks about engagement of knowledge users as co-producers of knowledge in health research identified 15 concepts related to the lifecycle of research studies: prepare, plan, conduct, and apply. The study authors proposed that common to co-production processes is a form of partnered negotiation that takes place at the start and throughout research studies [6]. The assumption is that there is a link between knowledge and practice, and co-production is more likely to result in the generation of knowledge that is useful and able to be used in practice and in policy. We (authors on this paper) adopt the term “engagement” in research as an arrangement in the governance of the research process where those who influence, administer, and/or use healthcare systems partner with one another as equals on a team to co-produce knowledge [12, 13]. Knowledge users and researchers work together as equals on the team to attain co-production.

There are many terms used to describe approaches to co-production including collaborative research, action research, participatory research [14], engaged scholarship [15], mode 2 research (that is, working with end users) [16], and integrated knowledge translation [14]. The term used for co-production depends on the field in which the study is being conducted, what is being produced, and who is involved [17]. Our team chose integrated knowledge translation as the research approach to structure co-production as it supports collaboration between researchers and knowledge users to develop solutions to complex issues [18]. According to the Canadian Institutes of Health Research (CIHR), integrated knowledge translation engages knowledge users as part of the research team, from defining the research question to applying the findings [16]. It is described as “meaningful engagement of the right (members of the research team) at the right time throughout the research process” [19]. Current shortcomings of co-production are conceptual (what is meant when discussing co-production) and methodological (co-production strategies and how to measure co-production) [20, 21]. In co-production, researchers and knowledge users share decisions with mutual exchange of information and learning throughout the entire research project [6], [10, 22]. Our team co-produced a systematic review to facilitate reporting on and evaluation of research processes.

Co-production and knowledge synthesis

There is growing interest in the engagement of knowledge users in the conduct of knowledge syntheses, such as systematic reviews. Engagement of knowledge users is more likely to reduce research waste, result in evidence that addresses end-users’ needs, and improve the translation of evidence into policy and practice [23, 24]. The study of co-production processes in the conduct of a knowledge synthesis is, however, a newer field. Pollock et al. [25] assessed the conduct of systematic reviews by teams that included knowledge users across 32 studies [25]. They found that knowledge user engagement was usually reported for the initial (frame the question and plan the systematic review) and the final (interpret, publish, and disseminate findings) steps of a systematic review. It was less common to report knowledge user engagement during the conduct of the systematic review (search, screen, abstract, and analyze data) [25, 26], a finding that aligns with the general literature about engagement of knowledge users in research [6]. There is no one strategy for engagement of knowledge users with researchers in systematic reviews [27]. Devane et al. [28] identify “studies within a review” as important to address the issues related to (1) uncertainties about the evidence base for how evidence syntheses are planned, conducted, and shared, and (2) the need for high-quality evidence to inform decisions for the conduct of evidence syntheses [28]. Little is known about team members’ experiences of using a co-production approach to conduct a systematic review.

The research context

We are a team with members from eight different countries and include those who experience chronic health conditions (patient/caregiver partners, here referred to as Cochrane consumers) and researchers, many of whom hold additional roles (for example, healthcare professionals, health systems administrator, educator, role with a health advocacy organization). We hold common interests in interventions to support shared decision-making processes for people seeking healthcare for themselves or for family members. Shared decision-making is a process where the patient and their clinician(s) work together to use the best available evidence, clinical expertise, and the patient’s informed preferences to make a decision that is tailored to the individual patient needs [29, 30]. An intervention called decision coaching holds potential to facilitate shared decision-making processes [31]. We decided that a systematic review was needed to understand the effectiveness of decision coaching and to identify research gaps. In addition, we wanted to understand how to conduct the systematic review to generate evidence in ways that can be considered useful and able to be used in policy and practice and set up a study within a review. Thus, we sought to conduct and evaluate the co-production of a systematic review on decision coaching for people facing treatment and screening healthcare decisions for themselves or family members.

Our study objectives were to:

  1. 1)

    describe the co-production approach to conduct the systematic review;

  2. 2)

    explore the experiences of team members with the co-production of the systematic review;

  3. 3)

    elicit perceptions of team members about the co-production of the systematic review.

Methods

Study design

We chose to evaluate ourselves to learn from our own systematic review team members’ experiences and perceptions, that is, to conduct a self-study of belonging to a co-production research team [32, 33]. We consider our study to be an example of a study within a review [28] as there is uncertainty in the methodology for engaging knowledge users in co-production, as well as the methods for evaluation of co-production. Here, we describe an approach to the co-production of a systematic review and present an evaluation of methods we used for organizing the co-production of a systematic review. We bound our work in time and context, and report on events that occurred between August 2019 and February 2022 when we conducted a mixed methods case study in the context of an international, interdisciplinary team conducting co-production of a systematic review [34,35,36]. We structured our work with the pragmatic intent to produce knowledge that is useful and could be used in practice and policy by team members and wider networks interested in shared decision-making interventions. The University of Ottawa’s Research Ethics Board approved the self-study (#20,190,408-01H).

We used the Collaborative Research Framework [37] to support an integrated knowledge translation approach, to guide partnering of knowledge users and researchers to prepare, plan, and conduct the systematic review. We were limited to dissemination of the systematic review findings due to time and complexity of the task, and did not complete the “apply” step [6] (Fig. 1).

Fig. 1
figure 1

The co-production of a systematic review with an integrated knowledge translation approach

Full size image

The framework assumption is that facilitation of integrated knowledge translation processes is possible with the preparation of team members to engage in iterative processes of knowledge exchange and learning throughout the research study and following replicable research steps: first, establish the guiding features for co-production with a team governance structure; and second, define research actions to be operationalized by the team to support the co-production of research evidence [16]. The framework has been used previously to guide research processes that include knowledge synthesis [38,39,40,41,42].

To ensure transparency and completeness in our work, we used the Guidance for Reporting Involvement of Patients and the Public 2 (GRIPP2) to report on the systematic review team engagement as it is a checklist to report on engagement in general health and social care research [43] (Supplemental file #1). We report on our self-study with the Standards for Reporting Qualitative Research (SRQR) reporting guideline [44] (Supplementary file #2).

Setting, participants

Prior to the conduct of the systematic review, two authors on this paper (JJ, DS) initiated processes to “prepare” and “plan” the systematic review to better understand decision coaching as an intervention [31]. JJ and DS held conversations with Cochrane consumers and researchers, many who held dual roles as healthcare providers, educators, policy makers, and all with common interests on the topic of decision coaching. For example, a meeting was held with Cochrane consumers who were familiar with systematic reviews as peer reviewers, to explain the project, opportunities for involvement, and their preferences. JJ and DS then assembled a team to prepare and plan the systematic review proposal (“the team”). With one person coordinating activities (JJ), the team engaged in an iterative series of conversations to negotiate:

  1. 1)

    common language, definitions, and concepts to describe our systematic review topic (that is, decision coaching);

  2. 2)

    the systematic review question, approach, and procedures;

  3. 3)

    the strategies for working together, that is, to co-produce the systematic review.

The team described the systematic review, and plans for self-study to evaluate co-production processes, in a funding application. While we considered all team members to be of equal value, we did not expect that they all contribute in the same manner. Team members brought a range of skills that could be helpful for activities in relation to their knowledge, occupational roles, and in relation to life experiences. Throughout the review, training was offered to team members on an “as-needed” basis and often took the form of one-on-one conversations to explain concepts or review steps.

After our team received funding from the Canadian Institutes of Health Research (CIHR) (April 1, 2019), the “conduct” phase of the systematic review began. We hired a study coordinator (MC) to support the systematic review and study within a review tasks. Compensation to support the participation of Cochrane consumers was arranged according to CIHR guidelines [45]. The team disseminated systematic review findings following a plan developed for the systematic review protocol, with presentations [46, 47], and following publication of the systematic review (November 2021) [48] with plain language reports [49,50,51], and communications to network partners. The systematic review and study within a review tasks were coordinated through the University of Ottawa and Ottawa Hospital Research Institute, in Ottawa, Ontario, Canada.

We (full team) agreed to the conduct of a study within a review to learn from team members’ belonging to a co-production research team. Systematic review team members with expertise in the use of an integrated knowledge translation approach to co-production that includes a Cochrane consumer (MS, IDG, DS, JJ) formed a core group that included the study coordinator to collaborate in the development, conduct, and analysis of the study. The full team was consulted by the core group on the final plans for the self-study [52]. All 17 team members (excluding the study coordinator) of the systematic review were eligible to participate in the study. The study occurred immediately before initiation of and then after the “conduct” phase of the systematic review.

Procedures for self-study recruitment, engagement and data collection

The study data sources included surveys and documents related to the systematic review. Prior to the conduct of the systematic review, and to begin the study, the study coordinator sent an email to all team members inviting them to participate. Of those who indicated interest in participation, the study coordinator sent further information explaining the study, data management (protection and storage), and seeking their consent to participate. Team members were informed that agreement or refusal to participate would have no effect on their status as a member of the team.

The team members who provided consent were asked to participate in a survey. In the survey, they were asked to respond to open- and closed-ended questions on preferred level of engagement during the conduct of the systematic review, and their experiences and perceptions co-producing the systematic review. Participants in the study were sent an email with a link to a baseline survey. At 1 and 2 weeks after the initial request, email reminders to complete the baseline survey were sent to participants [53]. Then, the study coordinator invited participants to be involved in the various steps of the systematic review based on their preferred level of engagement reported in the baseline self-study survey; the remainder of the systematic review team members were sent a general invitation to be involved at the various steps of the systematic review. The study coordinator arranged meetings and engagement in the tasks. If participants wanted to co-lead a step of the systematic review, they were invited to join the executive committee to conduct tasks. If they wanted to participate in a particular step of the systematic review, they were provided with the resources to participate (for example, access to Covidence to screen studies). Participants were reassured that there would be no repercussions if their availability or interest in participation with the steps of the systematic review changed from the baseline survey to when the systematic review task needed to be done. Upon completion of the systematic review, an end of self-study survey was administered to participants with the same procedures. We report on our survey [54] with the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) [55] (Supplementary file #3). Members of the core group that led the conduct of the study within a review (MC, JJ) collected systematic review documents (for example, meeting notes, systematic review paperwork, researcher journal).

Survey instrument

The core group in the development and conduct of the study with the study coordinator led the development and pilot testing of the online surveys. We could not identify an instrument to evaluate how well our team worked on conducting the systematic review. Overall, the instrument consisted of seven pages of questions, with between one to 17 items per page of the survey, for a total of 39 items. The survey took between 15 and 30 min to complete and participants had the option to answer or skip questions as they preferred.

We view the process of co-production to require consideration of the people in the research partnership, and the contexts in which they are situated, and wanted our evaluation to reflect their realities as participants:

  1. 1)

    Our approach to co-production involved assessment of participants’ preferred levels of engagement (that is, role) in the systematic review steps (baseline self-study survey) (for example, reviewing the search strategy, screening citations), and confirmation of participants’ actual levels of engagement (end of self-study survey), based on Arnstein’s Ladder of Citizen Participation and the Strategy for Patient-Oriented Research – Patient Engagement Framework [56, 57] and previous research [58, 59] about engagement in research. There were options for preferred level of engagement for each systematic review step, and open-ended questions that asked for any additional comments about their level of engagement in the steps of the systematic review. At the completion of the systematic review, participants were asked questions about their satisfaction with each step of the systematic review (end of self-study survey) [40].

  2. 2)

    To explore participants’ experiences with the co-production of the systematic review, we used the 36 item Partnership Indicators Questionnaire (PIQ) instrument adapted with permission of the PIQ lead author [60,61,62]. The PIQ is designed to assess the performance of researcher-health policy maker partnerships. Three lists of indicators include the following: common partnership indicators, early partnership indicators, and mature partnership indicators. The dimensions include the following: (common partnership indicators) communication, collaboration in research, dissemination of research, (early partnership indicators) research findings, negotiation, partnership enhancement, (mature partnership indicators) meeting information needs, level of rapport and commitment. The selected PIQ questions focused on the concepts of communication, collaboration, and negotiation and included an open text option for additional comments (baseline, end of self-study surveys).

  3. 3)

    We elicited the perceptions of participants about co-production of the systematic review with closed and open-ended questions with a focus on challenges, benefits, and impacts, at baseline and upon completion of the systematic review (end of self-study survey) [40, 63].

  4. 4)

    We collected participant demographic characteristics.

Study documents

We utilized team documents that included meeting notes, systematic review paperwork, and a researcher journal, to conduct the document analysis method [64]. The aim was to report on engagement of the team members, following completion of the systematic review. We used the Authors and Consumers Together Impacting on eVidencE (ACTIVE) framework to describe and report on Cochrane consumer engagement across team activities in the steps of systematic reviews. The ACTIVE framework is used to define whether and how knowledge users are engaged in systematic reviews, for example, the method of recruitment, the approach, format, and stage of engagement [25, 26].

Analysis

The core group in the development and conduct of the study with the study coordinator conducted the analysis. The study coordinator provided frequency counts for quantitative survey data. One researcher (JJ) conducted the frequency counts of open-text responses, and content analysis. A panel of second reviewers (MS, MC, DS, IDG) confirmed the work through interpretation and discussion [65] with reflection on concepts related to engagement [6]. Survey responses to closed-ended questions were tabulated; content analysis was used to analyze responses to open-ended questions, which involved segmenting responses by topics and into categories. Each question was considered to be a topic and the responses and development of codes defined the content in each category [66]. We report the frequency counts of the type of responses with illustrative quotes.

The team documents were reviewed for evidence that the systematic review was co-produced. One researcher (JJ) with the support of a second (MS) engaged in a three-stage analysis process of skimming, reading, and interpretation [66] to reach consensus, reported using the ACTIVE framework [25], and confirmed by a panel of reviewers (MC, DS, IDG). The analysis includes the ways in which decisions were made, meetings conducted, and information communicated [25]. While data analyses were initially conducted separately, findings were interpreted by corroborating quantitative and qualitative findings [67]. As part of the mixed methods, quantitative data findings were identified as the main data source with qualitative findings used to supplement and explain the quantitative findings.

Results

Data collection commenced after we received ethics approval in August 2019 to evaluate the co-production of the systematic review and was completed in February 2022, after publication of the systematic review by the Cochrane Library [31]. First, we provide the characteristics of participants. Then, we describe the co-production approach to the conduct of the systematic review (the context for co-producing the systematic review, and participant engagement) (objective 1), explore the experiences of team members with co-production of the systematic review (objective 2), and elicit perceptions of team members about co-production of the systematic review (objective 3).

Characteristics of the participants

Of 17 team members, 14 (82% response rate) consented to participate and completed the baseline self-study survey, and of the participants 12 (86% response rate) completed the post self-study survey. We report on the 12 participants who completed both surveys.

Of 12 participants, most self-identified as women (n = 9, 75%), were born between 1950 and 1980, and all had graduate level of education (see Table 1). There were seven researchers, four researcher trainees, two clinicians, two patient/caregiver partners (Cochrane consumers), and three researchers who reported that they held additional roles (administrator in a health system, health educator, patient advocate with an organization) and team members may have belonged to more than one category. Participants reported a broad range of experiences with decision coaching, systematic review methods, and shared decision-making. All participants indicated that they would be able to act on the findings of the systematic review, and two indicated that they might also be impacted by the findings.

Table 1 Characteristics of participants (n = 12)
Full size table

Objective 1: describe the co-production approach to the conduct of the systematic review

The context for co-producing the decision coaching systematic review

We conducted all study tasks remotely, due to the pandemic. To operationalize co-production of the systematic review, we (the team) assembled and self-organized a governance structure with an executive team (MS, SK, DS, JJ), of which one was a Cochrane consumer, and a Steering Committee consisting of the remaining 13 team members. The study coordinator supported the team. The executive team was tasked with operationalizing the systematic review and the Steering Committee with providing direction to the systematic review and engagement in systematic review tasks as they preferred. In the 15 months of the systematic review, the executive team met online every 2 weeks and the Steering Committee met online four times. The team members committed to roles and processes for systematic review governance and negotiated terms of reference document based on a previously developed format [40] (Supplementary file # 4).

The team agreed on features for the conduct of the systematic review in two stages: (1) ethical guidance, theoretical perspective, and (2) research actions. The team agreed on concept definitions for shared decision-making and decision coaching [16, 31, 68]. Finally, the team agreed upon an integrated knowledge translation approach to structure research actions and engage team members in co-production with support strategies such as communication, collaboration, and negotiation [61]:

  1. 1)

    During the conduct of the systematic review, the study coordinator and executive team maintained communication with the Steering Committee through email to support engagement in systematic review team activities at the level participants had indicated they wanted to be involved (individual 1:1, and group). In addition, the study coordinator shared systematic review progress summaries monthly with the team.

  2. 2)

    The study coordinator used the information from the baseline self-study survey to provide a structured approach to collaboration, with an individualized approach to engaging team members for the systematic review steps.

  3. 3)

    The team members had the opportunity to indicate their preferred level of engagement in the systematic review throughout the study.

We adjusted our plans for assessment of preferred and actual levels of engagement with each step in the systematic review. We originally asked participants about their preferred levels of engagement for all eleven systematic review research steps. There were four steps in which we did not end up involving team members who volunteered for those steps: (1) “conduct literature search” was done by the librarian and did not engage team members; (3) “pull full text articles” was done by the study coordinator; (8) “conduct analysis” consisted of the study coordinator adding extracted outcome data to Review Manager [69] (RevMan Web 2023) and leading the conduct of meta-analyses according to the protocol with support of other team members (DS, JJ), and followed with team review and confirmation; and (9) “draft the systematic review article” was co-led by two team members (JJ, DS). For the study of the systematic review conduct, as we were only just initiating the systematic review and the baseline results (although all extremely positive) were not a relevant indicator of team function, we only report the end of systematic review survey results for satisfaction with the level of engagement.

Participant engagement

Participants’ actual roles in the systematic review steps were mostly consistent with their preferred roles but there was a slight overall move towards greater engagement (“invite me [to participate]”) (see Table 2).

Table 2 Participant preferred, actual role and satisfaction in the conduct of the systematic review (N=12)
Full size table

Eight participants provided more detailed responses to the open-ended question that asked for any additional comments about their level of engagement in the steps of the systematic review. Four participants explained their decision to engage with the systematic review tasks due to a positive experience with the team (for example, “opportunity to reach out if we had any questions,” a “collegial approach” and “wonderful experience in knowledge synthesis and IKT,” “many learning opportunities and the team made it clear that it was a mutual learning experience which was very gratifying”). Of these eight participants, three reported that their level of engagement was due to the project management of the systematic review: “great project management,” “project team leads’ efforts to engage team members…for example, email updates [newsletters] were effective for the progress of the review,” “I participated in regular team calls.” Two participants identified factors which impacted their engagement: one participant identified perceptions of their role with the systematic review process, “I was actually able to participate more than I had envisioned”; the other alluded to factors external to the systematic review that limited engagement, “I would have liked to have had the time and opportunity to prioritize having been even more involved in the project.”

Of 12 participants, 11 were satisfied with their level of engagement at each step of the systematic review and one did not respond. There were two exceptions: for the step “search grey literature” with slightly lower participant satisfaction scores with most satisfied (n = 9, 82%), one indicating they were not satisfied (9%) and one indicating “not applicable” (9%), and “conduct analysis” with most satisfied (n = 10, 90.0%) and one indicating they were not satisfied (9%). If the systematic review was to be conducted again, eight indicated wanting to have the same level of engagement and four indicated that they would want to be more engaged. Five participants indicated that they were very satisfied and seven totally satisfied, with the extent to which they were engaged in the project.

Using the ACTIVE framework to report on the systematic review team engagement, team members were recruited by the study coordinator to engage in the conduct of the systematic review using a closed (by invitation) strategy. The mode of engagement was defined as continuous, with team members engaged during all steps of the systematic review, and with leading or controlling levels of influence on the conduct of the systematic review [25] (Supplemental file #5).

Objective 2: explore the experiences of team members with co-production of the systematic review

Participants reported on team processes that supported their ability to participate in co-producing the systematic review, these processes being communication, collaboration, and negotiation. With few exceptions, self-study participants reported positive experiences, mostly strongly agreeing or agreeing with the survey statement [61]. None responded with disagree or strongly disagree (see Table 3).

Table 3 The participants’ perceptions of the co-production approach to conducting the systematic review – results from the partnerships indicator questionnaire (n = 12)
Full size table

One participant provided a comment in the open text about communication: “communication was consistent – concise and informative.” Another participant related a comment about team collaboration: “Research team provided accommodations to a team member to ensure their meaningful contributions.” While most participants strongly agreed or agreed they had positive experiences co-producing the systematic review, some participants indicated a neutral response for three items about documentation of roles and responsibilities for their participation in the study processes, systematic review terms of reference, and documentation of team member needs. In addition, one participant reported that the “In-person meeting with co-PI to negotiate my role was very helpful and set the tone for the review.”

Objective 3: elicit perceptions of team members about co-production of the systematic review

Overall, participants indicated that the integrated knowledge translation approach was successful in leading to co-production of the systematic review. Participants reported a strong positive team experience (see Table 4).

Table 4 Results of six open-ended team survey questions about experiences with an integrated knowledge translation approach to conducting a systematic review (n = 12)
Full size table

Eight participants reported that collaborative activities and engagement of team members align with a co-production approach: “We had a patient on the executive. We had team members that had various roles.” Eight participants reported that they would not change anything about the co-production approach, and of the four that indicated being unsure, two suggested more opportunities for communication and inclusion.

Challenges to co-production were reported by five participants to be due to team logistics: “We are many people and from the whole of the world.” Four participants reported not being aware of any team challenges. Eight participants described the inclusion of a range of perspectives as the benefit of the co-production approach, and four participants reported the co-production approach was a way to “optimize the relevance, the usefulness of the systematic review output.” Eleven of the 12 participants indicated that they would work on another project with the same team. When asked for additional comments five participants reported a strong positive or “good experience.”

Discussion

Our international and interdisciplinary team that included two Cochrane consumers organized study governance and agreed on research actions and strategies to support co-production of the systematic review about decision coaching. Overall, our results indicate that it is feasible to use an integrated knowledge translation approach to conduct a systematic review. Participants reported satisfaction with their engagement in the steps of the systematic review and positive experiences with and perceptions of team processes. Our study reveals that planning and actively supporting team member engagement in the research lifecycle led all team members to perceive the systematic review as co-produced.

A relational approach to research is crucial for the success of co-production

Our study demonstrates the importance of a relational approach for co-production, meaning that there must be interactive processes among team members that leads to relationship building. Our findings about the relational nature of co-production align with recent work on integrated knowledge translation guiding principles that include the following [19]: (1) partners (knowledge users with researchers) develop and maintain relationships based on trust, respect, dignity, and transparency; (2) partners share in decision-making; (3) partners foster open, honest, and responsive communication; (4) partners recognize, value, and share their diverse expertise and knowledge; (5) partners are flexible and receptive in tailoring the research approach to match the aims and context of the project; (6) partners can meaningfully benefit by participating in the partnership; (7) partners address ethical considerations; and (8) partners respect the practical considerations and constraints of all partners.

In our study, team members who indicated wanting to be involved with systematic review steps at the start of the study reported high satisfaction with their levels of engagement in the systematic review and attributed their positive experiences with the team for their levels of engagement. Some participants indicated not being satisfied with the steps for which there was limited opportunity for engagement. The engagement of knowledge users with researchers in research has distinct and important benefits that include the generation of knowledge that is useful and able to be used in practice and policy [23, 24, 70, 71]. Based on our findings, we propose that there is potential for very positive and enjoyable inter- and intrapersonal benefits for those who participate in co-production.

Team engagement in the research lifecycle of a systematic review

It is essential to plan for and actively support collaboration, communication, and opportunities to negotiate engagement of team members who want to co-produce a systematic review. We chose an integrated knowledge translation approach to create opportunities for team members to share decisions, exchange information, and learn from one another. Other studies have described teams as challenged by co-production, due to the potential for a lack of clarity about study motivations and outcomes [72]. There is also the potential for those in the research partnership to experience risks that are practical (for example, time, money), personal (for example, interpersonal conflict, stress), and professional (for example, impacts on reputation) [21, 73, 74]. The participation by team members in the conduct of the systematic review was voluntary, and our team benefitted from an experienced study coordinator. When work or personal factors had an impact on team members’ ability to participate, the study coordinator was able to manage the logistics of scheduling meetings or study tasks to accommodate team members’ changing availabilities and needs. The study coordinator also ensured that there were regular and productive communications to the team to ensure its’ progress with tasks. Team members must have clearly defined strategies for co-production to structure and attain agreed-upon outcomes [21].

Our team used a framework to guide the conduct of our work together [37], and is one example of how research teams can organize themselves to operationalize concepts important for co-producing research across the research lifecycle [6]. Teams of knowledge users and researchers are striving to explain and evaluate co-production, with a growing theoretical and practical knowledge base in support of a relational approach to research [75,76,77].

We describe a team approach to collaboration (for example, define, agree upon team governance, research activities and roles) and communication (for example, sustained, consistent) among individuals and groups, and in a range of forms (for example, written, email, spoken). Furthermore, we sought to create opportunities for team members to determine their engagement with tasks as the systematic review proceeded. We assured all team members that they could adjust their preferred levels of engagement over time, and supported people to engage in their preferred manner. Work has been done to understand what knowledge, skills, and attitudes (“competencies”) are helpful for research teams that include families/caregivers as equal partners with researchers, healthcare providers, and decision-makers. While trends in competencies are identified across the different categories of team members, common findings across the team members are attitudes demonstrating the inclination to conduct the work [78].

The strategies we applied to support meaningful engagement of all team members are described elsewhere as creating “space to talk” (team members share views and recognize one another as bringing important knowledge) and “space to change” (in response to the shared knowledge, taking action in the study and how the team works together) [79]. We worked together to meet the needs of our international and interdisciplinary team members, to design an approach to co-production that established a culture and expectation of mutual respect [80].

We note some limitations and strengths of our work. We conducted a self-study as a study within a review and there is no established approach to determine the quality of self-study or a study within a review, as there currently there are no guidelines specific to self-study [81] or studies within a review. In response, we provide clear and detailed descriptions of how we collected and reported data, used available reporting guidelines, and more than one source of data to triangulate and represent the findings of the self-study. Another potential limitation is that we adapted an instrument that was developed for use with researcher-health policy maker partnerships rather than an interdisciplinary team. Our ability to measure co-production using this adapted instrument was limited due to the lack of a validated tool which is a recognized paucity within the co-production literature [82, 83].

We did draw on previous work conducted with international and interdisciplinary teams, and with the leadership of team members who have experience in the conduct of collaborative research. As the study relied on self-reports from participants, our study may be prone to self-reporting bias that include social desirability and recall bias. We were fortunate to have a study coordinator to support participants and ensure that they could participate anonymously. In addition, we may not have fully captured the experiences of team members and a fulsome description of an international and interdisciplinary team experience may not have been conveyed. Our team and circumstances are very specific to the conduct of a systematic review; for this reason, the transferability of findings to other teams and settings may be limited. We were fortunate to have had two Cochrane consumers participate on the team, willing to bring their knowledge and perspectives to the conduct of our work.

There were some lessons learned regarding set up of the initial survey. We did not engage team members in all eleven steps as originally planned. Step 1 “Conduct the search in databases and remove duplicates” was executed according to the study protocol by the librarian. In the future, we might omit this question from the initial survey or revise it to read “Review the search strategy to be used in electronic databases.” Step 2 “Pull full text articles” was done by the study coordinator given it is a mundane task, requires access to an institutional library for access to subscription articles not otherwise publicly available, and we felt that team member expertise and resources could be better used for other tasks. In the future, we would likely omit this item from the initial survey. Step 8 “Conduct analysis” involved adding extracted outcome data to RevMan Web and conducting meta-analyses according to the protocol. We were advised by Cochrane to only allow a limited number of people to revise the report in RevMan Web; therefore, this step was done by the study coordinator and data verified by another team member. In the future, we might omit this question from the initial survey or revise it to read “Discuss the analysis.” Step 9 “Draft the systematic review article” was led by two team members only (JJ, DS) given that there are challenges with involving many people in writing the first draft of an article. In the future, we would discuss how we planned to involve team members in drafting the systematic review prior to deciding whether to include or exclude this item in the initial survey.

Conclusions

Our mixed methods case study shows that it is feasible to co-produce a systematic review. We used a framework to guide knowledge users and researchers partnering on the systematic review and the conduct of a self-study as a study within a review of a co-production process. We describe the co-production approach used to conduct a systematic review, and report on the experiences and perceptions of international and interdisciplinary team members that include Cochrane consumers. The participants in our self-study reported overall high levels of satisfaction with their engagement in co-production of the systematic review.

We confirm that a relational approach to research is important for co-production. We propose that clearly defined strategies are essential for the co-producing a systematic review such as determining common interests, reaching agreement on the systematic review parameters (study purpose and plan, team organization), and planning and providing supports for communication, collaboration, and opportunities to negotiate participation during the systematic review lifecycle. Although approaches to co-production are assumed to have an influential role in the development of applicable research evidence, high-quality evaluations of co-production processes and outcomes are limited [77, 83, 84]. Our study can contribute to a knowledge base that is a foundation for future methodological studies that investigate co-production within evidence syntheses [28]. An important contribution of our study is that it describes how to structure, report on, and evaluate research processes used in co-producing a systematic review. Further studies that are focused on rigorous evaluations, such as intervention studies, to examine approaches to co-production of systematic reviews by international and interdisciplinary team members and consider team member interests, preferences, and circumstances are needed.

Availability of data and materials

The datasets used and/or analyzed during the self-study are available from the corresponding author on reasonable request.

Change history

  • 09 June 2024

    The collaborator member's name was corrected from 'Sandra Dun' to 'Sandra Dunn'.

References

  1. Canadian Institutes of Health Research (CIHR). CIHR Strategic Plan 2021–2031: a vision for a Healthier Future. The best health for all, powered by outstanding research [cited August 28, 2023]. Available from: https://cihr-irsc.gc.ca/e/52331.html.

  2. World Health Organization. Everybody business: strengthening health systems to improve health outcomes: WHO’s framework for action. Geneva: Switzerland; 2007.

    Google Scholar 

  3. Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Washington (DC): National Academies Press (US); 2001.

    Google Scholar 

  4. Hanna TP, King WD, Thibodeau S, Jalink M, Paulin GA, Harvey-Jones E, et al. Mortality due to cancer treatment delay: systematic review and meta-analysis BMJ. 2020;371:m4087.

    PubMed  Google Scholar 

  5. Damschroder LJ, Aron DC, Keith RE, Kirsch SR, Alexandra JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Jull JE, Davidson L, Dungan R, Nguyen T, Woodward KP, Graham I. A review and synthesis of frameworks for engagement in health research to identify concepts of knowledge user engagement. BMC Med Res Methodol. 2019;19:211.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Parry D, Salsberg J, Macauley AC. Guide to researcher and knowledge-user collaboration in health research 2015. Available from: https://cihr-irsc.gc.ca/e/44954.html#s2. Cited 2023 April 17.

  8. Jull J, Giles A, Graham ID. Community-based participatory research and integrated knowledge translation: advancing the co-creation of knowledge. Implement Science. 2017;12(1):150.

    Article  Google Scholar 

  9. Felipe A, Renedo A, Marston C. The coproduction of what? Knowledges, values, and social relations in health care. PLoS Bioly. 2017;15(5):e2001403.

    Article  Google Scholar 

  10. Kothari A, Rycorft-Malone J, McCutcheon C, Graham ID. Chapter 1 Introduction. 2022. In: Research coproduction in healthcare. Wiley.

  11. Ostrom E. Crossing the great divide: coproduction, synergy, and development. World Dev. 1996;24(6):1073–87.

    Article  Google Scholar 

  12. Sheridan S, Schrandt S, Forsythe L, Hilliard TS, Paez KA. The PCORI engagement rubric: promising practices for partnering in research. Ann Fam Med. 2017;15(2):165–70.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Canadian Institutes of Health Research. Patient engagement: what is patient engagement? : Government of Canada; 2018. Available from: http://www.cihr-irsc.gc.ca/e/45851.html.

  14. Tetroe J. Knowledge Translation at the Canadian Institutes of Health Research: a primer. Focus Technical Brief. 2007;18:1–8.

    Google Scholar 

  15. Van De Ven AH, Johnson PE. Knowledge for theory and practice. Acad Manag Rev. 2006;31(4):802–21.

    Article  Google Scholar 

  16. Canadian Institutes of Health Research (CIHR). Guide to knowledge translation planning at CIHR: integrated and end-of-grant approaches 2015 Available from: http://www.cihr-irsc.gc.ca/e/45321.html.

  17. Filipe ARA, Marston C. The coproduction of what? Knowledge, values, and social relations in health care. PLoS Biol. 2017;15(5): e2001403.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Straus S, Tetroe J, Graham ID. Knowledge translation in health care: moving from evidence to practice. UK: Wiley-Blackwell Publishing Ltd; 2009.

    Book  Google Scholar 

  19. The University of Brtitish Columbia. IKT Guiding Principles UBC Available from: https://ikt.ok.ubc.ca/.

  20. Verschuere B, Brandsen T, Pestoff V. Co-production: the state of the art in research and the future agenda. VOLUNTAS. 2012;23(4):1083–101.

    Article  Google Scholar 

  21. Oliver K, Kothari A, Mays N. The dark side of coproduction: do the costs outweigh the benefits for health research? Health Res Policy Syst. 2019;17(1):33.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Kothari A, Wathen NC. A second critical look at knowledge translation. Health Policy. 2013;109(2):187–91.

    Article  PubMed  Google Scholar 

  23. SPOR Evidence Alliance. Strategy for Patient Oriented Research (SPOR) Alliance: a Canadian Modeul to Build Learning Health Systems 2023. Available from: https://sporevidencealliance.ca.

  24. Cochrane Collaboration. Cochrane framework for engagement and involvement of patients, carers, and the public 2023. Available from: https://www.cochrane.org/news/cochrane-framework-engagement-and-involvement-patients-carers-and-public.

  25. Pollock A, Campbell P, Struthers C, Synnot A, Nunn J, Hill S, Goodare H, Morris J, Watts C, Morely R. Development of the ACTIVE framework to describe stakeholder involvement in systematic reviews. J Health Serv Res Policy. 2019;24(4):245–55.

    Article  PubMed  Google Scholar 

  26. Pollock A, Ccampbell P, Struthers C, Synnot A, Nunn J, Hill S, Goodare H, Morris J, Watts C, Morley R. Stakeholder involvement in systematic reviews: a scoping review. Syst Rev. 2018;7:208.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Pollock A, Campbell P, Synnot A, Smith M, Morley R. Patient and public involvement in systematic reviews. GIN; 2021.

  28. Devane D, Burke NN, Treweek S, Clarke S, Thomas J, Booth A, Tricco AC, Saif-Ur-Rahman KM. Study within a review (SWAR). Journal of Evidence Based Medicine. 2022;15:328–32. https://doi.org/10.1111/jebm.12505.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Coulter A, Collins A. Making shared decision-making a reality: no decision about me, without me. London: The King’s Fund; 2011.

    Google Scholar 

  30. National Institute for Health and Care Excellence (NICE). Shared decision making 2024. Available from: https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance/nice-guidelines/shared-decision-making.

  31. Jull J, Kopke S, Boland L, Coulter A, Dunn S, Graham ID, Hutton B, Kasper J, Kienlin SM, Légaré F, Lewis KB, Lyddiatt A, Osaka W, Rader T, Rahn AC, Rutherford C, Smith M, Stacey D. Decision coaching for people making healthcare decisions. Cochrane Database Syst Rev. 2019;7:CD013385.

    Google Scholar 

  32. Woods J. Self-study: a method for continuous professional learning and a methodology for knowledge transfer. Qual Advance Nurs Educ. 2021;7(2):7.

    Google Scholar 

  33. Pitthouse-Morgan K. Self-study in teaching and teacher education: characteristics and contributions. Teaching Teach Educ. 2022;119:103880.

    Article  Google Scholar 

  34. Miles MB, Huberman A, Saldaña J. Qualitative data analysis: a methods sourcebook. 3rd ed. Thousand Oaks, California: SAGE; 2014.

    Google Scholar 

  35. Yin R. Case study research design: design and methods. 6th ed. Thousand Oaks, California: SAGE; 2018.

    Google Scholar 

  36. Walton JB, Plano Clark VL, Foote LA, Johnson CC. Navigating intersecting roads in a mixed methods case study: a dissertation journey. J Mixed Methods Res. 2020;14(4):436–55.

    Article  Google Scholar 

  37. Jull J, Giles A, Boyer Y, Stacey D, Minwaashin Lodge - The Aboriginal Women’s Support Centre. Development of a collaborative research framework: an example of a study conducted by and with a first nations, inuit and métis women’s community and its research partners. ACME. 2018;17(3):671–86.

    Google Scholar 

  38. Jull J, Giles A, Boyer Y, Stacey D. Cultural adaptation of a shared decision making tool with Aboriginal women: a qualitative study. BMC Med Inform Decis Mak. 2015;15:1.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Funnell S, Jull J, Mbuagbaw L, Welch V, Dewidar O, et al. Improving social justice in observational studies: protocol for the development of a global and Indigenous STROBE-equity reporting guideline. Int J Equity Health. 2023;22(1):55.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Jull J, Graham ID, Kristjansson E, Moher D, Petkovic J, Yoganathan M, Tugwell P, Welch VA, Members of the, Consort-Equity Boston Equity, Symposium. Taking an integrated knowledge translation approach in research to develop the CONSORT-Equity 2017 reporting guideline: an observational study. BMJ Open. 2019;9(7):e026866.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Jull J, Crispo J, Welch V, MacDonald H, Brascoupe S, Boyer Y, et al. Interventions for indigenous peoples making health decisions: a systematic review. Pimatisiwin. 2013;11(3):539–54.

    Google Scholar 

  42. Jull J, Fairman K, Oliver S, Hesmer B, Pullattayil AK, The Not Deciding Alone Team. Interventions for Indigenous Peoples making health decisions: a systematic review. Arch Public Health. 2023;81:174.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, Altman DG, Moher D, Barber R, Denegri S, Entwistle A, Littlejohns P, Morris C, Suleman R, Thomas V, Tysall C. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. The BMJ. 2017;358:j3453.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. O’Brien B, Harris I, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research (SRQR). Acad Med. 2014;89(9):1245–51.

    Article  PubMed  Google Scholar 

  45. Canadian Institutes of Health Research. Considerations when paying patient partners in research 2022. Available from: https://cihr-irsc.gc.ca/e/51466.html.

  46. Morley R, Smith M, Morley K, Meader N, Jull J. Ready, set, go! Consumer involvement in systematic reviews Evidence Synthesis Ireland 2021. Available from: https://evidencesynthesisireland.ie/webinar/webinarcochraneconsumerinvolvement/.

  47. Jull J, Köpke S, Rahn A, Finderup J, Boland L, Lewis K, et al. Decision coaching for people preparing to make health care decisions: a systematic review using an integrated knowledge translation approach. SMDM 2020; October 6 2020; Virtual conference2020.

  48. Jull J, Köpke S, Smith M, Carley M, Finderup J, Rahn AC, Boland L, Dunn S, Dwyer AA, Kasper J, Kienlin SM, Légaré F, Lewis KB, Lyddiatt A, Rutherford C, Zhao J, Rader T, Graham ID, Stacey D. Decision coaching for people making healthcare decisions. Cochrane Database Syst Rev. 2021;11(11):CD013385.

    PubMed  Google Scholar 

  49. Finderup J. Shared decision making-the cornerstone in person-centred care. J Ren Care. 2021;47(3):144–5.

    Article  PubMed  Google Scholar 

  50. Jull J, Köpke S, Smith M, Carley M, Finderup J, Rahn AC, et al. Cochrane consumers and communication: Cochrane library. 2021. Available from: https://cccrg.cochrane.org/news/new-review-decision-coaching-people-making-healthcare-decisions. Cited 2023.

  51. Jull J, Stacey D, Kopke S. Support and collaboration with health-care providers can help people make health decisions. The Conversation. 2021 September 16 2023. Available from: https://theconversation.com/support-and-collaboration-with-health-care-providers-can-help-people-make-health-decisions-169816.

  52. International Association for Public Participation (IAP2): Advancing the practice of public participation. IAP2; n.d. Available from: https://www.iap2.org/page/pillars

  53. Dillman DA, Smyth JD, Christian LM. Internet, phone, mail, and mixed-mode surveys: the tailored design method: John Wiley & Sons. 2014.

    Book  Google Scholar 

  54. Latour JM, Tume L. How to do and report survey results robustly: a helpful mnemonic SURVEY. Nurs Crit Care. 2021;26:313–4.

    Article  PubMed  Google Scholar 

  55. Gunther E. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res. 2004;6(3): e34.

    Article  Google Scholar 

  56. Arnstein S. A ladder of citizen participation. J Am Plann Assoc. 1969;35(4):216–24.

    Google Scholar 

  57. Canadian Institutes of Health Research. Strategy for patient-oriented research - patient engagement framework: Government of Canada; 2019. Available from: https://cihr-irsc.gc.ca/e/48413.html.

  58. PCORI. Ways of Engaging- ENgagement Activity Tool (WE-ENACT) - Patients and Stakeholders 3.0 Item Pool2016 March 2, 2023. Available from: https://www.pcori.org/sites/default/files/PCORI-WE-ENACT-3-0-Patients-Stakeholders-Item-Pool-080916.pdf.

  59. PCORI. Evaluating Our Engagement in Research Initiatives: PCORI; 2023 [Available from: https://www.pcori.org/about/evaluating-our-work/evaluating-key-aspects-our-work/evaluating-our-engagement-research-initiatives.

  60. Kothari A, MacLean L, Edwards N, Hobbs A. Indicators at the interface: managing policymaker-researcher collaboration. Knowl Manag Res Pract. 2011;9:203–14.

    Article  Google Scholar 

  61. Kothari A, Sibbald S, Wathen N. For the PreVAiL Research Network. PreVAiL Partnership Evaluation Project Phase 1 Final Report. 2013.

    Google Scholar 

  62. Kothari A, Sibbald S, Wathen N. Evaluation of partnerships in a transnational family violence prevention network using an integreated knowledge translation and exchange model: a mixed methods study. Health Res Policy Syst. 2014;12:25.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Jull J, CONSORT Equity Team. The value of an integrated knowledge translation approach for developing reporting guidelines: engaging knowledge users interested in equity-relevant decision making. Toronto: Knowledge Translation Canada Scientific Meeting; 2016.

    Google Scholar 

  64. Bowen G. Document analysis as a qualitative research method. Qual Res J. 2009;9(2):27–40.

    Article  Google Scholar 

  65. Birt L, Scott S, Cavers D, Campbell C, Walter F. Member checking: a tool to enhance trustworthiness or merely a nod to validation? Qual Health Res. 2016;26(13):1802–11.

    Article  PubMed  Google Scholar 

  66. Elo S, Kyngas H. The qualitative content analysis. Journal of Advance Nursing. 2008;62(1):107–15.

    Article  Google Scholar 

  67. Fetters M, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Serv Res. 2013;48(6):2136–56.

    Google Scholar 

  68. O’Connor AM, Tugwell P, Wells GA, Elmslie T, Jolly E, Hollingworth G, McPherson R, Bunn H, Graham I, Drake E. A decision aid for women considering hormone therapy after menopause: decision support framework and evaluation. Patient Educ Couns. 1998;33(3):267–79.

    Article  PubMed  Google Scholar 

  69. Review Manager Web (RevMan Web). Version 4.27.0. The Cochrane Collaboration, 2023. Available at revman.cochrane.org.

  70. Involve. Briefing notes for researchers: involving the public in NHS, public health and social care research INVOLVE, Eastleigh 2012. Available from: http://www.invo.org.uk/wp-content/uploads/2012/04/INVOLVEBriefingNotesApr2012.pdf.

  71. Norström AV, Cvitanovic C, Löf MF, West S, Wyborn C, Balvanera P, et al. Principles for knowledge co-production in sustainability research. Nature Sustainability. 2020;3:182–90.

    Article  Google Scholar 

  72. Jenkins N, Matthews P. When methods meet: coproduction and participatory research Scottish Graduate School of Social Science Sgoil Cheumnaichean Saidheans Sòisealta na h-Alba 2020. Available from: https://www.sgsss.ac.uk/methods-resource/co-production-and-participatory-research/.

  73. Flinders M, Woods M, Cunningham M. The politics of co-production: risks, limits and pollution. Evidence & Policy: A Journal of Research, Debate and Practice. 2016;12(2):261–79.

    Article  Google Scholar 

  74. Rycroft-Malone Burton CR, Bucknall T, Graham ID, Hutchinson AM, Stacey D. Collaboration and co-production of knowledge in healthcare: opportunities and challenges. Int J Health Policy Manag. 2016;5(4):221–3.

    Article  PubMed  Google Scholar 

  75. Graham ID, Kothari A, McCutcheon C. Moving knowledge into action for more effective practice, programmes and policy: protocol for a research programme on integrated knowledge translation. Implement Sci. 2018;13:22.

    Article  PubMed  PubMed Central  Google Scholar 

  76. NHS England. Co-production n.d. Available from: https://www.england.nhs.uk/always-events/co-production/.

  77. McLean R, Tucker J. Evaluation of CIHR’s knoweldge translation funding program: Government of Canada; 2013. Available from: https://cihr-irsc.gc.ca/e/47332.html#a3.1.

  78. Fisch N, Atherton P, Doyle-Waters MM, MacLeod M, Mallidou A, Sheane V, Ward J, Woodley J. Patient-oriented research competencies in health (PORCH) for researchers, patients, healthcare providers, and decision makers: results of a scoping review. Research Involvement and Engagement. 2020;6 (4): https://doi.org/10.1186/s40900-020-0180-0

  79. Knowles SE, Allen D, Donnelly A, Flynn J, Gallacher K, Lewis A, et al. More than a method: trusting relationships, productive tensions, and two-way learning as mechanisms of authentic co-production. Res Involve Engagem. 2021;7:34.

    Article  Google Scholar 

  80. Witteman O, Dansokho SC, Colquhoun H, Fagerlin A, Giguere AM, Glouberman S, et al. Twelve lessons learned for effective research partnerships between patients, caregivers, clinicians, academic researchers, and other stakeholders. J Gen Intern Med. 2018;33(4):558–62.

    Article  PubMed  PubMed Central  Google Scholar 

  81. Feldman A. Validity and quality in self study. Educational Researcher. 2003;32(3):26–8.

    Article  Google Scholar 

  82. Nordin A, Kjellstrom S, Robert G, Masterson D, Areskoug JK. Measurement and outcomes of co-production in health and social care: a systematic review of empirical studies. BMJ Open. 2023;13: e073808. https://doi.org/10.1136/bmjopen-2023-073808.

    Article  PubMed  PubMed Central  Google Scholar 

  83. Norton MJ. Corproduction and mental health service provision: a protocol for a scoping review. BMJ Open. 2022;12(5): e058428. https://doi.org/10.1136/bmjopen-2021-058428.

    Article  PubMed  PubMed Central  Google Scholar 

  84. Durose C, Needham C, Mangan C, Rees J. Generating ‘good enough’ evidence for co-production. Evidence & Policy: A Journal of Research, Debate and Practice. 2017;13(1):135–51.

    Article  Google Scholar 

Download references

Acknowledgements

The authors acknowledge Tamara Rader, MLIS Information Specialist Consultant, for creating the search strategies and general librarian support for the project. We thank Brittany Hesmer for her work to develop the manuscript figure.

Consortium authors list

Members of the Cochrane Decision Coaching Review Team:

Janet Jull, School of Rehabilitation Therapy, Faculty of Health Sciences, Queen’s University, Kingston, Ontario, Canada; Ottawa Hospital Research Institute, Ottawa, Ontario, Canada

Maureen Smith, Cochrane Consumer Network Executive, Ottawa, Ontario, Canada.

Meg Carley, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada.

Dawn Stacey, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada; School of Nursing, University of Ottawa, Ottawa, Ontario, Canada.

Ian D. Graham, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada; School of Epidemiology, Public Health and Preventative Medicine, University of Ottawa, Ottawa, Ontario, Canada.

Laura Boland, IKTRN, Ottawa Hospital Research Institute, Ottawa Ontario, Canada; Western University, School of Health Sciences, London, Ontario, Canada; Evidence Synthesis and Knowledge Translation Unit, Applied Research Division, Centre for Surveillance and Applied Research, Health Promotion and Chronic Disease Prevention Branch, Public Health Agency of Canada, Government of Canada.

Sandra Dunn, Children’s Hospital of Eastern Ontario Research Institute, Ottawa, Ontario, Canada; Ottawa Hospital Research Institute, Ottawa, Ontario, Canada; BORN Ontario, Ottawa, Ontario, Canada; School of Nursing, University of Ottawa, Ottawa, Ontario, Canada.

Andrew A. Dwyer, Boston College William F. Connell School of Nursing, Chestnut Hill, Massachusetts, USA; Massachusetts General Hospital – Harvard Center for Reproductive Medicine, Boston, Massachusetts, USA.

Jeanette Finderup, Department of Renal Medicine, Aarhus University Hospital, Aarhus, Denmark; Department of Clinical Medicine, Aarhus University, Aarhus, Denmark.

Jürgen Kasper, Department of Nursing Sciences and Health Promotion, Faculty of Health Sciences, Oslo Metropolitan University, Oslo, Norway.

Simone Kienlin, Faculty of Health Sciences, Department of Health and Caring Sciences, University of Tromsø, Langnes, Norway; The South-Eastern Norway Regional Health Authority, Department of Medicine and Healthcare, Hamar, Norway.

Sascha Köpke, Institute of Nursing Science, University of Cologne, Medical Faculty; University Hospital Cologne, Cologne, Germany.

France Légaré, Department of Family Medicine and Emergency Medicine, Faculty of Medicine Laval University, Québec, Canada.

Krystina Lewis, School of Nursing, University of Ottawa, Ottawa, Ontario, Canada.

Anne Christin Rahn, Institute for Social Medicine and Epidemiology, Nursing Research Unit, University of Lübeck, Lübeck, Germany.

Claudia Rutherford, Faculty of Medicine and Health, Sydney Nursing School, The University of Sydney, Sydney, NSW, Australia.

Junqiang Zhao, Waypoint Research Institute, Waypoint Centre for Mental Health Care 500 Church Street, Penetanguishene, Ontario, Canada; Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada.

Funding

The study we report on here was funded by a Canadian Institutes of Health Research (CIHR) grant #PJT-162135.

Canadian Institutes of Health Research,FRN# PJT-162135

IDG is a recipient of a CIHR Foundation Grant (FDN # 143,237).

FL is a Tier 1 Canada Research Chair in Shared Decision Making and Knowledge Mobilization.

Author information

Authors and Affiliations

  1. School of Rehabilitation Therapy, Faculty of Health Sciences, Queen’s University, Kingston, ON, Canada

    Janet Jull

  2. Ottawa Hospital Research Institute, Ottawa, ON, Canada

    Janet Jull, Meg Carley, Dawn Stacey & Ian D. Graham

  3. Cochrane Consumer Network Executive, Ottawa, ON, Canada

    Maureen Smith

  4. School of Nursing, University of Ottawa, Ottawa, ON, Canada

    Dawn Stacey

  5. School of Epidemiology, Public Health and Preventative Medicine, University of Ottawa, Ottawa, ON, Canada

    Ian D. Graham

Authors
  1. Janet Jull
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maureen Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Meg Carley
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dawn Stacey
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ian D. Graham
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

The Cochrane Decision Coaching Review Team

  • Janet Jull
  • , Maureen Smith
  • , Meg Carley
  • , Dawn Stacey
  • , Ian D. Graham
  • , Laura Boland
  • , Sandra Dunn
  • , Andrew A. Dwyer
  • , Jeanette Finderup
  • , Jürgen Kasper
  • , Simone Kienlin
  • , Sascha Köpke
  • , France Légaré
  • , Krystina Lewis
  • , Anne Christin Rahn
  • , Claudia Rutherford
  •  & Junqiang Zhao

Contributions

JJ, IDG, DS, and MS conceived and led the design of the work described in the manuscript. MC made substantial contributions to the acquisition of data. JJ with MS and JJ with IDG, DS, MS, and MC participated in and provided substantial contributions to the analysis and interpretation during development of the work described in the manuscript, with all authors (JJ, IDG, DS, MS, MC with the Cochrane Decision Coaching Review Team) reviewing and providing final confirmation of results. JJ led the writing of the first version of the manuscript and was responsible for revisions with IDG, DS, MS, and MC. All authors (JJ, IDG, DS, MS, MC with the Cochrane Decision Coaching Review Team) made contributions to the drafts of this manuscript and have reviewed and revised the manuscript for important intellectual content. All authors (JJ, IDG, DS, MS, MC with the Cochrane Decision Coaching Review Team) approved the final version of the manuscript.

Corresponding author

Correspondence to Janet Jull.

Ethics declarations

Ethics approval and consent to participate

The Ottawa Health Science Network Research Ethics Board (REB) and University of Ottawa REB approved the self-study (#20190408-01H; #A07-12–02).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jull, J., Smith, M., Carley, M. et al. Co-production of a systematic review on decision coaching: a mixed methods case study within a review. Syst Rev 13, 149 (2024). https://doi.org/10.1186/s13643-024-02563-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13643-024-02563-8

Keywords

Systematic Reviews

ISSN: 2046-4053