Skip to main content
Download PDF

Preliminary guideline for reporting bibliometric reviews of the biomedical literature (BIBLIO): a minimum requirements

Systematic Reviews volume 12, Article number: 239 (2023) Cite this article

Abstract

Background

A bibliometric review of the biomedical literature could be essential in synthesizing evidence if thoroughly conducted and documented. Although very similar to review papers in nature, it slightly differs in synthesizing the data when it comes to providing a pile of evidence from different studies into a single document. This paper provides a preliminary guideline for reporting bibliometric reviews of the biomedical literature (BIBLIO).

Methods

The BIBLIO was developed through two major processes: literature review and the consensus process. The BIBLIO started with a comprehensive review of publications on the conduct and reporting of bibliometric studies. The databases searched included PubMed, Scopus, Web of Sciences, and Cochrane Library. The process followed the general recommendations of the EQUATOR Network on how to develop a reporting guideline, of which one fundamental part is a consensus process. A panel of experts was invited to identify additional items and was asked to choose preferred options or suggest another item that should be included in the checklist. Finally, the checklist was completed based on the comments and responses of the panel members in four rounds.

Results

The BIBLIO includes 20 items as follows: title (2 items), abstract (1 item), introduction/background (2 items), methods (7 items), results (4 items), discussion (4 items). These should be described as a minimum requirements in reporting a bibliometric review.

Conclusions

The BIBLIO for the first time provides a preliminary guideline of its own kind. It is hoped that it could contribute to the transparent reporting of bibliometric reviews. The quality and utility of BIBILO remain to be investigated further.

Peer Review reports

Background

Several guidelines exist for reporting findings of different study designs. The detailed explanations and checklists for such guidelines can be found in Enhancing the Quality and Transparency of Health Research (EQUATOR) Network [1] and are available to research communities [2]. For instance, the quality of reporting of meta-analyses (QUOROM) statement for improving the quality of reporting meta-analyses of randomized controlled trials was first published in The Lancet in 1999 [3]. Consequently, the work was further improved, and it was replaced with the preferred reporting items for systematic reviews and meta-analyses (PRISMA) [4]. This guideline was published simultaneously in 6 journals in 2009 [4,5,6,7,8,9], and since then, many biomedical journals and investigators have adhered to this instruction. The instruction also was extended, and complementary versions of the guideline either are developed (such as PRISMA for Abstracts) [10] or are under development (e.g., PRISMA for children) [11]. Even the preferred reporting items for overviews of reviews (PRIOR) are proposed [12], and a recent call by Systematic Reviews (the journal) indicates that attempts to enhance the knowledge of this type of reporting are in progress [12, 13].

However, we believe there is also a need for a guideline for another type of reporting, namely, Guideline for Reporting Bibliometric Reviews of the Biomedical Literature (BIBLIO). A bibliometric or a bibliographic review of the literature is different from an overview. Recently, the literature witnessed a relatively considerable number of bibliometric analyses of the biomedical literature [14,15,16,17,18,19,20,21,22,23]. The number of publications related to various topics with bibliometric or bibliography/bibliographic in the title during the last 10 years is presented in Fig. 1. Therefore, this paper attempts to propose a preliminary version of a guideline for reporting bibliometric reviews of the literature. The guideline was developed based on all existing guidelines presented in the EQUATOR Network [1]. In addition, experiences from writing a number of bibliometric reviews [24,25,26,27,28] helped the authors to formulate this first version of the work with the courage that it could be improved further by receiving feedbacks from other scholars in the field.

Fig. 1
figure 1

Papers with bibliography/bibliographic and bibliometric in the title of publications during 2013–2022 (PubMed)

Full size image

Although BIBLIO is in its preliminary stage of development and there is no evidence of its quality and utility, it is hoped that it could contribute to the transparent reporting of bibliometric reviews. The application of bibliometric reviews enables one to analyze vast amounts of publications and their production patterns on macroscopic and microscopic levels [29]. Therefore, this study aimed to provide a guideline for reporting bibliometric reviews. The BIBLIO checklist was registered in the EQUATOR Network on 19 October 2021 [30].

History

The term bibliometric and bibliography are used interchangeably in the literature. Earlier, the term bibliography was more popular, but it was gradually replaced with the bibliometric expression (Fig. 2). The history of the statistical bibliography as reviewed by Thackray [31] indicates that the root goes back to early 1900s as this was acknowledged in a paper by Garfield [32] and a number of scholar such as Cole and Eames (1917), Hulme (1923), Lotka (1926), and Gross and Gross (1927) were listed as those who contributed to the technic of statistical analysis of the literature. However, it was Otlet in 1934 who first used the term “bibliometrie” and defined it as “the measurement of all aspects related to the publication and reading of books and documents” [33, 34]. Then in 1969, Pritchard coined the term “bibliometrics” and defined it as “all the studies which seek to quantify the processes of written communication” [35]. The detailed history since 1934 is presented in Table 1.

Fig. 2
figure 2

Trends of using bibliography/bibliographic or bibliometric in the title of publications during 2013–2022 (PubMed)

Full size image
Table 1 A chronological list of definitions of bibliometrics, based on the historical development of the term and its definitions
Full size table

Definition

Bibliometric is a type of review that can be used to look at different and important areas of investigations and obtain a general synopsis of published literature [39]. This guideline defines a bibliometric review as follows “a review of all full published papers that appear in the biomedical journals and includes all types of evidence such as descriptive studies, observational studies, experimental studies, qualitative studies, and systematic reviews in order to account for every single evidence exist. The bibliometric of the literature does not include electronic publications a head of print since the ultimate date for such publications are not known”. This definition was formulated based on chronological account of the term bibliometric and its developments [31,32,33,34,35,36,37,38].

Similarities and differences between systematic reviews and bibliometrics

Bibliometric is similar to systematic review in retrieving the literature [40], but they have low agreement rate regarding relevant literature and the purpose. While systematic reviews are seeking to respond to a very clear question based on good quality evidences, bibliometrics is rather a numeration of evidence without quality assessment. Bibliometrics often rely on the interpretation of quantitative details of publications such as main topics, authors, sources, most impactful authors, most impactful articles, and countries in a particular area in the existing literature. In this type of study, mapping techniques including graphical representations, tabulated forms, network diagrams, and so on are used to present results usually performing these with the assistance of softwares [39,40,41,42].

Development of BIBLIO

The BIBLIO was developed through two major processes: literature review and the consensus process. These are briefly described as follows:

  • 1. Literature review for item selection

    The BIBLIO started with a comprehensive review to identify potential items for including in this guideline. The databases searched included PubMed, Scopus, Web of Sciences, and Cochrane Library. The aim was to examine and review all methodological papers on the conduct and reporting of bibliometric studies up to 2021. The search was updated in January 2022 and once during the process of revisions in September 2023. Papers were retrieved using different keywords and MeSH terms including “bibliometric,” “bibliography,” and “bibliographic” in the title of papers. All potentially relevant publications were extracted and reviewed independently by two authors (AM and SM). Overall, 13,720 papers were identified. After removing duplicates and irrelevant documents, only 19 papers [40, 43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60] were found that were dealing with methodological issues. Also, we visited all reporting guidelines for review studies that are indexed in the EQUATOR [1]. The items derived from the literature are shown in Table 2.

  • 2. Consensus process

    The process followed the general recommendations of the EQUATOR Network on how to develop a reporting guideline, of which one fundamental part is a consensus process [1]. We used Delphi consensus to obtain advice on how to report a “bibliometrics.” Delphi was performed based on the conducting and reporting Delphi studies (CREDES) guideline [61]. A panel consisted of eleven experts, including bibliometrician, epidemiologist, clinician, librarian, statistician, journal editor, and a research fellow. They were invited to see the list of items derived from the previous stage and asked to identify additional items and to choose preferred options or suggest other items that should be included in the checklist. In each round of the Delphi, the feedback process allowed and encouraged the selected participants to review and assess their own initial judgments. Thus, the results of previous iterations regarding specific items were changed or modified by each member of the expert panel in later iterations based on the review and assessing the comments and feedback provided by the other Delphi panelists [62].

Table 2 Items provided from the literature review
Full size table

In the first round of the Delphi process, we used an open-ended questionnaire to solicit specific information and to add suggested items to the list of items and increase the rich of data collection. After receiving the experts’ responses, we converted the collected information into a well-structured questionnaire on a five-point scale with content analysis technique. This questionnaire was used as the survey questionnaire for the second round of data collection. Each Delphi participant received a second questionnaire and was asked to review the items summarized based on the information provided in the first round. Accordingly, we asked Delphi panelists to rate items and state the rationale concerning rating priorities. In the third round, each Delphi panelist received a questionnaire that included the items and ratings summarized in the previous round and was asked to revise their judgments. The remaining items, ratings, minority opinions, and items achieved consensus were distributed to the panelists in the final round. The fourth round provided a final opportunity for participants to revise their responses after formal feedback of the group. At last, the checklist was finalized based on the comments and answers of the panel members in four rounds. The cut-off for consensus was determined by percentage of agreement (mainly 75 to 80%). The duration of each round of Delphi was about 8 weeks, and the length of the overall study process was 8 months. Before beginning the Delphi survey, all experts were asked to disclose any conflicts of interest. The response rate was 100% for all four rounds of the Delphi process.

Scope of the guideline

BIBLIO is for use in reporting bibliometric reviews and has been designed primarily for bibliometric reviews that evaluate published papers irrespective of the design of the studies. The BIBLIO items are relevant for all types of quantitative and qualitative studies. BIBLIO can be used for reporting original bibliometric reviews and updated bibliometric reviews. BIBLIO is not to guide a bibliometric review conduct. However, familiarity with BIBLIO is helpful when planning and conducting bibliometric reviews to ensure that all recommended information is captured.

The BIBLIO checklist

The development team provided a list of items based on the literature review and presented them into the consensus process. Participants made revisions to the phrasing and format of the checklist by consolidating and eliminating items during the consensus process. Eventually, the BIBLIO checklist consisted of 20 items that should be described as a minimum requirements in reporting a bibliometric review as follows: title (2 items), abstract (1 item), introduction/background (2 items), methods (7 items), results (4 items), discussion (4 items). The full description of the items is in progress and will be available in due course. However, as an example here, we elaborate on item 15. As shown item 15 provides guidelines for reporting the results. As such four options are proposed. In the following section, we describe each option ensuring that examples given could help investigators to better summarize the findings. Since the opening part of each option is the same here the focus is on how organize the main findings:

  • Option 1: Organization based on study design and main study types

    Research design is a blueprint of a scientific study. We could summarize studies based on different designs and main study types. For instance, one might summarize main study types based on randomized trials, observational studies, study protocols, diagnostic/prognostic studies, case reports, clinical practice guidelines, and qualitative studies on a given topic.

  • Option 2: Organization based on outcome measures

    The other suggestive way to summarize the main findings is based on outcomes. For instance, a bibliometric analysis that evaluated the impact of race on postoperative outcomes and complications following elective spine surgery was classified based on outcomes providing four categories including general complications, medical complications, surgical complications, and postoperative outcomes [63].

  • Option 3: Organization based on concept

    To simplify and clarify this presentation approach, we explain this option with an example. A study on bibliometric analysis of health literacy instruments summarized the findings in four tables according to the concept behind instruments including general instruments, condition-specific health literacy instruments (disease and content), population-specific instruments, and electronic health [28]. Authors could invent such concepts or use the literature for categorizing and summarizing the findings.

  • Option 4: Organization based on different subtitles relevant to the main topic

    This presentation approach is well known and was used in many studies. One example for this option is a bibliometric study on health-related quality of life in breast cancer patients. In this study, the findings were summarized and presented according to treatments modalities and a number of classifications including surgical treatment, systemic therapies, psychological distress, supportive care, and common symptoms [26]. One should note there are many ways that we could summarize and tabulate the findings to provide a quick and at the same time a comprehensive perspectives of the studies under review. The checklist is presented in Table 3.

Table 3 The BIBLIO checklist for reporting the bibliometric reviews of the biomedical literature
Full size table

Discussion

A bibliometric review is a helpful means for accurately and reliably summarizing the evidence, specifically when a large number of papers exist on a given topic [69]. The bibliometric studies that are well done usually could help to grasp the current literature, identify knowledge gaps, derive novel ideas for investigation, and position their intended contributions to the field [43].

The bibliometric methods are quantitative and descriptive by nature but also used to make pronouncements about qualitative aspects. The principal purpose of bibliometric studies is to change intangible knowledge (scientific quality) into manageable entities [70]. Bibliometrics are not in-depth and evaluative reviews. However, they could briefly report on effectiveness and evaluations. Overall, a good bibliometric review should provide a take-home message for its readers.

A number of recommendations are proposed to improve readability of bibliometric reviews. For instance, it was proposed using easy-to-interpret metrics, as non-experts have a difficulty understanding of complex indicators. Also, it was recommended to avoid inventing the indicators, especially composite metrics that mix several indicators in a single measure. Likewise, it was suggested to avoid conscious efforts to manipulate the findings, for instance, choosing metrics that may favor your institution, certain areas, or researchers within it [44].

A bibliometric review could reveal how much effort has been made into a specific topic. In addition, presenting and summarizing the studies allows scholars to use bibliometric analysis to uncover emerging trends in article publishing, journals’ performances, collaboration patterns, and exploring the intellectual structure of a specific domain in the extant literature [71, 72]. Describing the evidence could help policymakers, managers, and other decision-makers to formulate appropriate recommendations for practice or policy [73] and help editors judge the merits of publishing reports of new studies [74]. The bibliometric also helps translate and map the cumulative scientific knowledge and evolutionary nuances of well-established fields by making sense of large volumes of unstructured data in rigorous ways [43].

The use of BIBLIO similar to other guidelines [3, 4, 75] has the potential to benefit many stakeholders. The BIBLIO provides readers with a complete understanding of evidence about the necessity of each item. We have attempted to ensure that the guideline is helpful to authors seeking guidance on what to include in a bibliometric review. We hope the BIBLIO will help increase the quality of reported and published bibliometric reviews. Peer reviewers, editors, and other interested readers might also find the BIBLIO helpful in assessing such reviews. We hope journal editors will encourage authors to include the BIBLIO checklist when submitting a bibliometric review for publication.

Finally, although we followed the general recommendations of the EQUATOR Network and used a literature review and a Delphi consensus process to develop the BIBLIO checklist, it seems that its main limitation is the fact that there is no evidence to suggest it will improve the quality of bibliometric reviews. In this regard, feedback from editors and researchers about details and overall structure can be helpful. Additionally, one should note that bibliometric reviews is not an in-depth review of the literature and rather the most important contribution of this type of reviews is to collect and summarize evidence when we witness a pile of evidence on a topic. As such it reveals that how much effort has been conducted on a topic. In addition, this approach might help investigators to create new questions to conduct more focused studies on the topic in the future [26].

Conclusion

The BIBLIO provides a reporting guideline for bibliometric reviews of the biomedical literature. We hope that the guideline could result in more transparent and accurate reporting of bibliometric reviews.

Availability of data and materials

Not applicable.

Abbreviations

EQUATOR:

Enhancing the Quality and Transparency of Health Research

PRISMA:

Preferred reporting items for systematic reviews and meta-analyses

PRIOR:

Preferred reporting items for overviews of reviews

BIBLIO:

Guideline for reporting bibliometric review of the biomedical literature

References

  1. Equator Network. Reporting guidelines for main study types. 2022. http://www.equator-network.org/.

    Google Scholar 

  2. Groves T. Enhancing the quality and transparency of health research. BMJ. 2008;337(7661):a718.

    Article  PubMed  Google Scholar 

  3. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Lancet. 1999;354(9193):1896–900.

    Article  CAS  PubMed  Google Scholar 

  4. Moher D, Liberati A, Tetzlaff J, Altman D, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151:264–9.

    Article  PubMed  Google Scholar 

  5. Moher D, Liberati A, Tetzlaff J, Altman DG, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Open Med. 2009;3(3):e123–30.

    PubMed  PubMed Central  Google Scholar 

  6. Moher D, Liberati A, Tetzlaff J, Altman DG, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62(10):1006–12.

    Article  PubMed  Google Scholar 

  7. Moher D, Liberati A, Tetzlaff J, Altman DG, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Moher D, Liberati A, Tetzlaff J, Altman DG, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg. 2010;8(5):336–41.

    Article  PubMed  Google Scholar 

  9. Moher D, Liberati A, Tetzlaff J, Altman DG, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Plos Med. 2009;6(7):e1000097.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Transparent reporting of systematic reviews and meta-analyses. 2021. http://prisma-statement.org/Extensions/.

  11. Transparent reporting of systematic reviews and meta-analyses. 2021. http://www.prisma-statement.org/Extensions/InDevelopment.

  12. Pollock M, Fernandes RM, Becker LA, Featherstone R, Hartling L. What guidance is available for researchers conducting overviews of reviews of healthcare interventions? A scoping review and qualitative metasummary. Syst Rev. 2016;5(1):190.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Hunt H, Pollock A, Campbell P, Estcourt L, Brunton G. An introduction to overviews of reviews: planning a relevant research question and objective for an overview. Syst Rev. 2018;7(1):39.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Al-Jabi SW. Arab world’s growing contribution to global leishmaniasis research (1998–2017): a bibliometric study. BMC Public Health. 2019;19(1):625.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Bullock N, Ellul T, Bennett A, Steggall M, Brown G. The 100 most influential manuscripts in andrology: a bibliometric analysis. Basic Clin Androl. 2018;28:15.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Huang YK, Hanneke R, Jones RM. Bibliometric analysis of cardiometabolic disorders studies involving NO2, PM2.5 and noise exposure. BMC Public Health. 2019;19(1):877.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Jarden RJ, Narayanan A, Sandham M, Siegert RJ, Koziol-McLain J. Bibliometric mapping of intensive care nurses’ wellbeing: development and application of the new iAnalysis model. BMC Nurs. 2019;18:21.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Lewis R, Hendry M, Din N, Stanciu MA, Nafees S, Hendry A, et al. Pragmatic methods for reviewing exceptionally large bodies of evidence: systematic mapping review and overview of systematic reviews using lung cancer survival as an exemplar. Syst Rev. 2019;8(1):171.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Mahavadi A, Shah AH, Sarkiss CA. Commentary: a bibliometric analysis of neurosurgical practice guidelines. Neurosurgery. 2020;86(5):E412–3.

    Article  PubMed  Google Scholar 

  20. Okoroiwu HU, Lopez-Munoz F, Povedano-Montero FJ. Bibliometric analysis of global Lassa fever research (1970–2017): a 47–year study. BMC Infect Dis. 2018;18(1):639.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Shi G, Liu N, Yu X, Zhang H, Li S, Wu S, et al. Bibliometric analysis of medical malpractice literature in legal medicine from 1975 to 2018: Web of Science review. J Forensic Leg Med. 2019;66:167–83.

    Article  PubMed  Google Scholar 

  22. Sweileh WM, Al-Jabi SW, Saed HZ, Sawalha AF. Outdoor air pollution and respiratory health: a bibliometric analysis of publications in peer-reviewed journals (1900–2017). Multidiscip Respir Med. 2018;13:15.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Gorraiz JI, Repiso R, De Bellis N, Deinzer G. Best practices in bibliometrics & bibliometric services. Front Res Metr Anal. 2021;6:771999.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Montazeri A, McEwen J, Gillis CR. Quality of life in patients with ovarian cancer: current state of research. Support Care Cancer. 1996;4(3):169–79.

    Article  CAS  PubMed  Google Scholar 

  25. Montazeri A, Gillis CR, McEwen J. Quality of life in patients with lung cancer: a review of literature from 1970 to 1995. Chest. 1998;113(2):467–81.

    Article  CAS  PubMed  Google Scholar 

  26. Montazeri A. Health-related quality of life in breast cancer patients: a bibliographic review of the literature from 1974 to 2007. J Exp Clin Cancer Res. 2008;27:32.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Montazeri A. Quality of life data as prognostic indicators of survival in cancer patients: an overview of the literature from 1982 to 2008. Health Qual Life Outcomes. 2009;7:102.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Tavousi M, Mohammadi S, Sadighi J, Zarei F, Kermani RM, Rostami R, Montazeri A. Measuring health literacy: a systematic review and bibliometric analysis of instruments from 1993 to 2021. PLoS One. 2022;17(7):e0271524.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Kokol P, Blazun Vosner H, Zavrsnik J. Application of bibliometrics in medicine: a historical bibliometrics analysis. Health Info Libr J. 2021;38(2):125–38.

    Article  PubMed  Google Scholar 

  30. Enhancing the Quality and Transparency of Health Research (EQUATOR). Reporting guidelines for main study types. 2021. https://www.equator-network.org/library/reporting-guidelines-under-development/reporting-guidelines-under-development-for-other-studydesigns/#BIBLIO.

  31. Thackray A. Measurement in the historiography of science. In: Elkana YJ, Lederberg JK, K.Merton R, Thackray A, Zuckerman H, editors. Toward a metric of science: the advent of science indicator. New York: John Wiley & Sons; 1978. p. 11–30.

    Google Scholar 

  32. Garfield E. The intended consequences of Robert K. Merton Scientometrics. 2004;60(1):51–61.

    Article  Google Scholar 

  33. Otlet P. Library science: forgotten founder of bibliometrics. Nature. 1934;510:218.

    Google Scholar 

  34. Wikipedia. 2022. https://en.wikipedia.org/wiki/Bibliometrics.

  35. Pritchard A. Statistical bibliography or bibliometrics. J Doc. 1969;25(4):348–9.

    Google Scholar 

  36. Hawkins DT. Bibliometrics of electronic journals in information science. Inf Res. 2001;7(1):7–11.

    Google Scholar 

  37. De Bellis N. Bibliometrics and citation analysis: from the science citation index to cybermetrics. Lanham–Toronto–Plymouth: Scarecrow Press; 2009.

    Google Scholar 

  38. Broadus RN. Toward a definition of ‘bibliometrics.’ Scientometrics. 1987;12:373–9.

  39. Kumar M, George RJ, PS A. Bibliometric analysis for medical research. Indian J Psychol Med. 2023;45(3):277–82.

  40. Lubowitz JH, Brand JC, Rossi MJ. Search methods for systematic reviews and bibliographic articles can improve: responsibilities of authorship are vast. Arthroscopy. 2023;39(6):1367–8.

    Article  PubMed  Google Scholar 

  41. Aria M, Cuccurullo C. Bibliometrix: an R-tool for comprehensive science mapping analysis. J Informetr. 2017;11(4):959–75.

    Article  Google Scholar 

  42. Rojas-Sanchez MA, Palos-Sanchez PR, Folgado-Fernandez JA. Systematic literature review and bibliometric analysis on virtual reality and education. Educ Inf Technol. 2023;28(1):155–92.

    Article  Google Scholar 

  43. Donthu N, Kumar S, Mukherjee D, Pandey N, Lim WM. How to conduct a bibliometric analysis: an overview and guidelines. J Bus Res. 2021;133:285–96.

    Article  Google Scholar 

  44. Cabezas-Clavijo A, Torres-Salinas D. Bibliometric reports for institutions: best practices in a responsible metrics scenario. Front Res Metr Anal. 2021;6:696470.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Tester AC. A method for making a bibliography. Science. 1865;1930(72):321–2.

    Google Scholar 

  46. Ayala YD, Landero C. Practical method of determining the direction of the electrical axis in the frontal plane; bibliography of literature concerning methods of determining it. Arch Cardiol Mex. 1958;28(2):235–43.

    Google Scholar 

  47. Raisig LM. Mathematical evaluation of the scientific serial: improved bibliographic method offers new objectivity in selecting and abstracting the research journal. Science. 1960;131(3411):1417–9.

    Article  CAS  PubMed  Google Scholar 

  48. Bachev VI. Method of bibliographic work in performing local historical medical research. Sov Zdravookhr. 1979;1:61–2.

    Google Scholar 

  49. Ninkov A, Frank JR, Maggio LA. Bibliometrics: methods for studying academic publishing. Perspect Med Educ. 2022;11(3):173–6.

    Article  PubMed  Google Scholar 

  50. Kho ME, Brouwers MC. The systematic review and bibliometric network analysis (SeBriNA) is a new method to contextualize evidence. Part 1: description. J Clin Epidemiol. 2012;65(9):1010–5.

    Article  PubMed  Google Scholar 

  51. Campos-Asensio C. How to develop a bibliographic search strategy? Enferm Intensiva. 2018;29(4):182–6.

    Article  CAS  Google Scholar 

  52. Chiang HS, Huang RY, Weng PW, Mau LP, Tsai YC, Chung MP, et al. Prominence of scientific publications towards peri-implant complications in implantology: a bibliometric analysis using the H-classics method. J Oral Rehabil. 2018;45(3):240–9.

    Article  PubMed  Google Scholar 

  53. Van de Laar L, De Kruif T, Waltman L, Meijer I, Gupta A, Hagenaars N. Improving the evaluation of worldwide biomedical research output: classification method and standardised bibliometric indicators by disease. BMJ Open. 2018;8(6):e020818.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Linnenluecke MK, Marrone M, Singh AK. Conducting systematic literature reviews and bibliometric analyses. Aust J Manag. 2020;45(2):175–94.

    Article  Google Scholar 

  55. Jappe A. Professional standards in bibliometric research evaluation? A meta-evaluation of European assessment practice 2005–2019. PLoS One. 2020;15(4):e0231735.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  56. Schultz F, Anywar G, Quave CL, Garbe LA. A bibliographic assessment using the degrees of publication method: medicinal plants from the rural greater Mpigi region (Uganda). Evid Based Complement Alternat Med. 2021;2021:6661565.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Moro G, Valgimigli L. Efficient self-supervised metric information retrieval: a bibliography based method applied to COVID literature. Sensors. 2021;21(19):6430.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  58. Fernández-Ananín S, Rodríguez JB, Soler EM. Then, how do I document the idea? Bibliographic search engines. Cir Esp. 2022;100(6):375–7.

    Article  PubMed  Google Scholar 

  59. Liu F. Retrieval strategy and possible explanations for the abnormal growth of research publications: re-evaluating a bibliometric analysis of climate change. Scientometrics. 2023;128(1):853–9.

    Article  PubMed  Google Scholar 

  60. Koo M, Lin SC. An analysis of reporting practices in the top 100 cited health and medicine-related bibliometric studies from 2019 to 2021 based on a proposed guidelines. Heliyon. 2023;9(6):e16780.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on conducting and reporting Delphi studies (CREDES) in palliative care: recommendations based on a methodological systematic review. Palliat Med. 2017;31(8):684–706.

    Article  PubMed  Google Scholar 

  62. Hsu CC, Sandford BA. The Delphi technique: making sense of consensus. Pract Assess Res Evaluation. 2007;12(10):2.

    Google Scholar 

  63. Akosman I, Kumar N, Mortenson R, Lans A, De La Garza Ramos R, Eleswarapu A, et al. Racial differences in perioperative complications, readmissions, and mortalities after elective spine surgery in the United States: a systematic review using AI-assisted bibliometric analysis. Glob Spine J. 2023:21925682231186759. Online ahead of print.

  64. McDougal L, Dehingia N, Cheung WW, Dixit A, Raj A. COVID-19 burden, author affiliation and women’s well-being: a bibliometric analysis of COVID-19 related publications including focus on low-and middle-income countries. eClinicalMedicine. 2022;52:101606.

    Article  PubMed  PubMed Central  Google Scholar 

  65. Henstock L, Wong R, Tsuchiya A, Spencer A. Behavioral theories that have influenced the way health state preferences are elicited and interpreted: a bibliometric mapping analysis of the time trade-off method with VOSviewer visualization. Front Health Serv. 2022;2:848087.

    Article  PubMed  PubMed Central  Google Scholar 

  66. Bodea F, Bungau SG, Negru AP, Radu A, Tarce AG, Tit DM, et al. Exploring new therapeutic avenues for ophthalmic disorders: glaucoma-related molecular docking evaluation and bibliometric analysis for improved management of ocular diseases. Bioengineering. 2023;10(8):983.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  67. Sang XZ, Wang CQ, Chen W, Rong H, Hou LJ. An exhaustive analysis of post-traumatic brain injury dementia using bibliometric methodologies. Front Neurol. 2023;14:1165059.

    Article  PubMed  PubMed Central  Google Scholar 

  68. Ramli MI, Hamzaid NA, Engkasan JP, Usman J. Respiratory muscle training: a bibliometric analysis of 60 years’ multidisciplinary journey. Biomed Eng Online. 2023;22(1):50.

    Article  PubMed  PubMed Central  Google Scholar 

  69. Szomszor M, Adams J, Fry R, Gebert C, Pendlebury DA, Potter RW, Rogers G. Interpreting bibliometric data. Front Res Metr Anal. 2021;5:628703.

    Article  PubMed  PubMed Central  Google Scholar 

  70. Wallin JA. Bibliometric methods: pitfalls and possibilities. Basic Clin Pharmacol Toxicol. 2005;97(5):261–75.

    Article  CAS  PubMed  Google Scholar 

  71. Donthu N, Kumar S, Pandey N, Lim WM. Research constituents, intellectual structure, and collaboration patterns in journal of international marketing: an analytical retrospective. J Int Mark. 2021;29(2):1–25.

    Article  Google Scholar 

  72. Verma S, Gustafsson A. Investigating the emerging COVID-19 research trends in the field of business and management: a bibliometric analysis approach. J Bus Res. 2020;118:253–61.

    Article  PubMed  PubMed Central  Google Scholar 

  73. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev. 2021;10:89.

    Article  PubMed  PubMed Central  Google Scholar 

  74. Young C, Horton R. Putting clinical trials into context. Lancet. 2005;366(9480):107–8.

    Article  PubMed  Google Scholar 

  75. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg. 2021;88:105906.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

None.

Funding

None.

Author information

Authors and Affiliations

  1. Population Health Research Group, Health Metrics Research Center, Iranian Institute for Health Sciences Research, ACECR, Tehran, Iran

    Ali Montazeri & Samira Mohammadi

  2. Department of Epidemiology and Biostatistics, Schulich School of Medicine and Dentistry, Western University, London, Canada

    Parisa M.Hesari

  3. Vali-E-Asr Reproductive Health Research Center, Family Health Research Institute, Tehran University of Medical Sciences, Tehran, Iran

    Marjan Ghaemi

  4. Department of Midwifery and Reproductive Health, School of Nursing and Midwifery, Shahid Beheshti University of Medical Sciences, Tehran, Iran

    Hedyeh Riazi

  5. Quality of Life Research Groups, Breast Cancer Research Center, Motamed Cancer Institute, ACECR, Tehran, Iran

    Zahra Sheikhi-Mobarakeh

Authors
  1. Ali Montazeri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Samira Mohammadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Parisa M.Hesari
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marjan Ghaemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hedyeh Riazi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zahra Sheikhi-Mobarakeh
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AM put forward the idea, proposed the guideline, did the literature search, and wrote the manuscript. SM performed literature search, helped in writing the manuscript, and provided the figures and tables. PMH performed literature search and provided help in writing process. MGH helped in writing process. HR contributed to writing and conducting group discussion. ZSM helped in literature search and writing process. All authors read and approved the manuscript.

Corresponding author

Correspondence to Ali Montazeri.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no conflict of interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Montazeri, A., Mohammadi, S., M.Hesari, P. et al. Preliminary guideline for reporting bibliometric reviews of the biomedical literature (BIBLIO): a minimum requirements. Syst Rev 12, 239 (2023). https://doi.org/10.1186/s13643-023-02410-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13643-023-02410-2

Keywords

Systematic Reviews

ISSN: 2046-4053