Skip to main content

Advertisement

Reliability of measurements of the fractured clavicle: a systematic review

Systematic Reviews volume 6, Article number: 223 (2017) Cite this article

Article metrics

Abstract

Background

The objective of this systematic review was to evaluate the reliability and reproducibility of measurements of shortening in midshaft clavicle fractures (MSCF) using any available imaging technique.

Methods

Electronic databases (PubMed, EMBASE, and Cochrane) were searched. The 4-point-scale COSMIN checklist was used to evaluate the methodological quality of studies.

Results

Four studies on reliability of measurement of MSCF were identified. These studies were of fair and poor quality. The reported intrarater reliability varied between none to fair, and intrarater reliability was minimal.

Conclusion

No definite conclusions could be drawn. In order to optimize future studies and the realization of comparable results, more research is necessary to identify a standardized method of imaging and measuring.

Level of Evidence III.

Open Peer Review reports

Background

Fractures of the clavicle are common, comprising up to 5% of all fractures in adults [1]. Most clavicle fractures are localized at the level of the mid-diaphyseal third [2]. Dislocation of the fracture elements in midshaft clavicle fractures (MSCF) occurs due to the actions of the sternocleidomastoid muscle, which displaces the medial fragment superiorly and posteriorly, and of the deltoid and great pectoral muscles, which shift the lateral fragment inferiorly and anteriorly. These shifts cause a malaligned fracture that may result in symptomatic malunion of the clavicle and increase the risk of a nonunion [3,4,5,6].

In the last decades, many studies have reported that a shortened clavicle can lead to worse functional outcomes, pain, loss of strength, rapid fatigue, hyperesthesia of the hand and arm, difficulty sleeping on the affected side, and esthetic complications [5,6,7,8,9,10,11,12,13,14]. Godfrey et al. [15] reported that the degree of symptomatology and occurrence of mal- and nonunion after MSCF is related to the extent of shortening and displacement of the fracture elements. Mean post-traumatic shortening of the fractured clavicle has been reported to be approximately 1.2 cm; however, shortening of up to 3 cm has been reported [16]. It has been described that there are poorer outcomes when shortening of the clavicle is more than 15–20 mm or 9.7–15% as compared to the original length [5, 7,8,9,10,11,12,13,14].

For this reason, lately, the tendency has been to surgically reduce and fixate MSCF if shortened more than 15–20 mm, or if displaced more than the diameter of the clavicle’s shaft. However, due to the unique shape of the clavicle, consisting of an S-shape in two planes, reliable and reproducible measurements of the displacement and shortening can be challenging.

Although there is a plethora of available modalities and techniques to measure shortening of the MSCF, it still remains unclear which method is most accurate, reproducible, and useful in daily practice.

Therefore, the objective of this systematic review was to evaluate the reliability and reproducibility of measurements of shortening in MSCF using any available imaging technique.

Methods

Electronic databases (PubMed, EMBASE, and Cochrane) were searched from their inception to November 2016. Keywords used to develop our search strategy were “clavicle,” “fractures,” “imaging,” “shortening,” “displacement,” and “reliability.” The detailed search strategy is described in Additional file 1. The inclusion criteria and method of analysis were specified in advance and documented in a protocol that was not registered in PROSPERO (Additional file 2).

Inclusion criteria

All titles and abstracts were screened, and study inclusion was decided on by two reviewers (PH/GH). In case of discrepancy in study, inclusion disagreements were discussed until consensus on eligibility was reached. References of retrieved eligible articles were searched for supplementary studies. Studies meeting the following criteria were included:

  • Studies aiming to assess shortening of the fractured clavicle for intrarater and interrater reliability.

  • Studies investigating methods of imaging of the fractured clavicle for intrarater and interrater reliability.

  • Only original studies were included.

  • Studies in Dutch or English.

  • Study population aged 9 years and older.

Abstracts, theses, and conference proceedings were not included.

Data extraction and quality assessment

An electronic data extraction form was created and used to record data. Data from all included studies were extracted with respect to specific characteristics, that is, number of clavicles reviewed, study design, imaging technique, method of measurements, statistical analysis, and the author’s conclusion. PH and GH extracted data independently. If disagreement persisted after discussion, consensus was met consulting AvK.

Methods and quality were independently assessed (PH and GH, any discrepancies were discussed to achieve consensus, using a third reviewer (AvK) for all included studies. The 4-point scale COSMIN checklist box B for assessment of reliability was used.

The “worst score counts” algorithm was used for the analysis [17]. Briefly, each item from COSMIN box B was rated individually as “excellent,” “good,” “fair,” or “poor,” and an overall score was given by taking the lowest score of any of the items.

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, both the PRISMA flowchart and checklist, were followed during the preparation of this review (Fig. 1 and Additional file 3).

Fig. 1
figure1

PRISMA flow diagram

Full size image

Results

In total, 184 studies were identified. After the removal of duplicates, 122 studies were selected for the screening of titles and abstracts. Reference tracing and hand searching yielded two more possibly eligible studies. After the selection of titles and abstracts, 15 studies were selected for a full-text evaluation. After full-text evaluation, four studies were included in this systematic review and were used for data extraction (Fig. 1—flow diagram). Table 1 shows the extracted data of the four studies included in this systematic review.

Table 1 Extracted data included studies
Full size table

Methodological quality of the studies

Using the 4-point-scale COSMIN checklist box B for assessment of reliability, three included studies were rated as fair and one as poor. The quality classification per study per item is described in Fig. 2.

Fig. 2
figure2

COSMIN checklist box B for assessment of reliability per included study per item

Full size image

Studies included in the systematic review

Jones et al. [18] assessed the interrater and intrarater agreement for shortening and displacement using anterior-posterior (AP) and 30° caudo-cranial X-ray views in 30 patients. The measurements were performed by 13 observers on two occasions. The amount of shortening measured on radiograph was divided into seven categories: 0–5, 5.1–10.0, 10.1–15.0, 15.1–20.0, 20.1–25.0, 25.1–30, and > 30 mm. No to weak interrater agreement was found for shortening in the different categories. Displacement was divided into three categories: 0–49, 50–99, and 100%. Interrater agreement was minimal to weak. Intrarater agreement was moderate for displacement and minimal for shortening (Table 1).

Silva et al. [19] compared two methods of measuring shortening in 30 patients (32 fractures). The first was the method of choice of the observer, and the second was a standardized method. They used AP and 15° caudo-cranial views. Measurements were performed twice by seven observers. Intraclass correlation coefficients (ICC) with confidence intervals (CI) were calculated to determine interrater agreement, and average differences between the two time points with 95% CI were calculated to determine intrarater agreement.

For method 1, the interrater agreement was 0.771 (95% CI 0.655–0.865) and 0.743 (95% CI 0.604–0.851) at the two time points for fair agreement. The intrarater agreement for method 1 was 2.62 mm (95% CI 2.24–3.00) average difference between the two time points. For method 2, the interrater agreement was 0.741 (95% CI 0.629–0.842) and 0.685 (95% CI 0.554–0.805) at the two time points for fair and poor agreement, respectively. The intrarater agreement for method 2 was 3.34 mm (95% CI 2.88–3.80) average difference between the two time points.

Smekal et al. [20] assessed different modalities and views to determine the most accurate method compared to the CT in 30 patients. They used a standardized method of measuring. Measurements were performed by four observers on two occasions. A paired t test or a nonparametric Wilcoxon signed-rank test for determination of differences of mean values in paired samples was performed. The Kolmogorov-Smirnov test was used for determination of the distribution form. For the assessment of repeatability between occasions 1 and 2, the repeatability coefficient according to Bland and Altman was used. The differences among measurements on the four plain radiographs and CT scans were not significant. Also, there was no significant difference shown in measurements on both occasions. Repeatability coefficients were comparable for CT measurements, the posteroanterior thorax radiographs, and the 15° caudo-cranial anteroposterior panorama radiographs of the shoulder girdle. Repeatability coefficients for the clinical measurements and measurements on 15° caudo-cranial radiograph of the clavicle were markedly higher indicating lower repeatability.

Archer et al. [21] aimed to identify a correlation between plain AP film and computed tomography (CT) measurement of displacement and the inter- and intraobserver reliability of repeated radiographic measurements. Six observers (three orthopedic surgeons and three residents) measured the clavicles of 22 patients with an interval of 2 weeks. Shortening was assessed using the contralateral unfractured side as a reference. Participants were not instructed on what specific points within the fracture should be measured to estimate shortening and was therefore not standardized. The limits of agreement calculated using the Bland-Altman repeatability coefficient revealed a mean of ± 3.48 cm. The error inherent in plain film measurements in this study is 6.96 cm. Intraobserver agreement calculated with the paired t tests demonstrating a p > 0.05 in five of six observers. The authors conclude that plain AP film measurements of acute MSCF do not reliably predict shortening.

Discussion

In this systematic review, we evaluated the reliability and reproducibility of measurements of shortening in MSCF. The results of this systematic review demonstrate that the literature on this topic did yield only three fair and one poor quality studies. Since shortening plays an increasingly important role in deciding on surgical intervention of MSCF, it is important to have a reliable and accurate method of measuring. Despite the lack of high-quality studies, the available knowledge and literature should not be discarded.

Smekal et al. [20] published a paper validating the accuracy/reliability of measurements of different imaging modalities and techniques. They found that the posterior-anterior (PA) thorax approximated the measurements on CT the best. Measurements on 15° tilted caudo-cranial radiograph of the clavicle and clinical measurements showed the smallest agreement with CT measurements. However, they did not state the reproducibility of measurements. The measurements were performed in healed malunited clavicle fractures and not in the acute phase. This was done to ensure static conditions in time. This is a strong feature of the study since Plocher et al. [22] described progressive shortening in acute MSCF in time.

The PA thorax means a higher dose for the patient of 0.1 mSv compared to 0.02 mSv for a clavicle AP [23]. It also relies on the symmetry of the clavicle using the unfractured side for comparison. A study by Cunningham et al. [24] reported asymmetry of the intact clavicle of more than 5 mm in almost 30% of patients. This may mean that measuring shortening of the MSCF compared to the unfractured side may be less reliable than assumed.

Archer et al. [21] also used the assumption of symmetry which may compromise reliability. They found a limit of agreement of 3.48 cm indicating that plain AP film of the fractured clavicle is not reliable in the prediction of the shortening measured on the CT scan. However, they found an ICC of 0.90. The statistical method for calculating intrarater variability using the paired t test may be debatable but they report no significant differences in measurements in five of six observers.

Jones et al. [18] reported weak to no agreement in inter- and intrarater agreement for radiological shortening using AP and 30° caudo-cranial views. They did not report a standardized method of measuring the shortening on these views. In addition, they also reported minimal to moderate interrater agreement for displacement and comminution. Intrarater agreement was strong for comminution, moderate for displacement, and minimal for shortening.

In contrast to current standard practice in which AP and 15° caudo-cranial views are made, papers have been published that support the use of a 15–30° cranio-caudal AP or PA or PA thorax view as being the most accurate in measuring the shortening of MSCF. [20, 25,26,27]. Although commenting on accuracy, these studies did not report the reproducibility of these views. Silva et al. [19] proposed a standardized mode of measuring shortening in MSCF. Their paper focused on adolescents, not adults, and also did not report the imaging modality or technique used. After contacting the corresponding author, it was verified that measurements were performed on standard AP and 15° caudo-cranial views. They reported no difference in a standardized measurement or method of choice concerning inter- and intraobserver variability. More recent studies find both a moderate and excellent interrater agreement using a standardized method of measuring [28, 29].

Two studies were not included in the review because these studies did not meet the inclusion criteria as only interrater agreement and not intrarater agreement was reported. However, we believe these studies are worth mentioning here. Stegeman et al. [29] found an intraclass correlation coefficient of 0.97 (CI 0.95–0.99) between two observers measuring shortening in a standardized way on 32 AP X-rays of the fractured clavicle. Interestingly, they found only a moderate agreement (0.45 CI 0.12–0.69) for measuring absolute shortening on the AP panoramic view after consolidation indicating that the imaging technique may be influential on the reliability of measurements as well. Malik et al. [28] report an ICC of 0.926 (CI 0.909–0.941) between four observers using a standardized method of measuring shortening of the fractured clavicle in 196 AP chest X-rays. These images were made with the patient varying between supine, semi-upright, and upright positions. The goal of this study was to evaluate differences in measured shortening between the different positions of the patients. No additional information on statistical analysis or interrater agreement per subgroup was reported.

Other factors reported to influence reliable and reproducible measurements are variation in magnification due to X-ray positioning and possibly positioning of the patient [18, 28, 30]. Backus et al. [30] reported a statistically significant difference between upright and supine patient positioning concerning shortening and displacement. Malik et al. [28] found a significant step-wise progression of measured shortening between supine, semi-upright, and upright positioning of the patient.

Some limitations of this study have to be discussed. First, there is only limited available literature on the topic of measuring the fractured clavicle. Since four studies were included and none of them were rated as good or excellent quality according to the COSMIN checklist, it was not possible to draw definite conclusions or make definite recommendations. Second, although the COSMIN checklist is considered the best available option to evaluate the methodological quality of studies on measurement properties, the “worst score counts” algorithm might underestimate the overall quality of a paper (e.g., one poor score out of a total of 11 items results in a poor overall score). For that reason, we provided the scores for all items using the 4-point scale. Other limitations of this study include the possibility of publication bias and language restrictions. Third, the inclusion criteria used might have been too strict. Two papers that did not meet the inclusion criteria were identified but yet could be of value on the topic. Including these papers [28, 29], however, does not influence the final conclusion pertaining the lack of evidence on the subject.

In order to optimize future studies and the realization of comparable results, a standardized method of imaging and measuring is of great importance. When considering the optimal method of imaging and measuring the fractured clavicle, one should consider the following: Imaging modality and technique, patient positioning, radiation exposure, costs and the method for measuring shortening, and/or displacement. To identify a standardized method, a compromise between these factors should be made based on further research.

CT scans and PA thorax seem more accurate, but the first is more expensive and both expose the patient to a much higher radiation dose. Supine positioning of the patient may underestimate the actual shortening and displacement, which in turn can negatively influence the decision to surgically reduce and fixate the MSCF. Calibrated views will prevent magnification errors while measuring. Although not proven better, it might be a consideration to optimize consistency by measuring shortening and displacement in a standardized and possibly proportional way as proposed by other authors. [9, 13, 19, 30, 31]

Conclusion

The objective of this systematic review was to evaluate the reliability and reproducibility of measurements of shortening in MSCF using any available imaging technique.

We identified four studies on reliability of measurement of MSCF. Since these studies were only of fair and poor quality, it was impossible to draw definite conclusions. Shortening is one of the reasons to surgically treat the fractured clavicle, so further research is needed to identify the most effective, reproducible, and reliable method of imaging and measuring. In order to optimize future studies and the realization of comparable results, a standardized method of imaging and measuring is of great importance.

Abbreviations

AP:

Anterior-posterior

CI:

Confidence interval

CT:

Computed tomography

ICC:

Intraclass correlation coefficient

MSCF:

Midshaft clavicle fractures

PA:

Posterior-anterior

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

References

  1. 1.

    Nowak J, Holgersson M, Larsson S. Can we predict long-term sequelae after fractures of the clavicle based on initial findings? A prospective study with nine to ten years of follow-up. J Shoulder Elb Surg. 2004;13:479–86.

  2. 2.

    Robinson CM. Fractures of the clavicle in the adult. Epidemiology and classification. J Bone Joint Surg Br. 1998;80:476–84.

  3. 3.

    Canadian Orthopaedic Trauma S. Nonoperative treatment compared with plate fixation of displaced midshaft clavicular fractures. A multicenter, randomized clinical trial. J Bone Joint Surg Am. 2007;89:1–10.

  4. 4.

    Jorgensen A, Troelsen A, Ban I. Predictors associated with nonunion and symptomatic malunion following non-operative treatment of displaced midshaft clavicle fractures—a systematic review of the literature. Int Orthop. 2014;38:2543–9.

  5. 5.

    Ledger M, Leeks N, Ackland T, Wang A. Short malunions of the clavicle: an anatomic and functional study. J Shoulder Elb Surg. 2005;14:349–54.

  6. 6.

    McKee MD, Wild LM, Schemitsch EH. Midshaft malunions of the clavicle. J Bone Joint Surg Am. 2003;85-A:790–7.

  7. 7.

    Hill JM, McGuire MH, Crosby LA. Closed treatment of displaced middle-third fractures of the clavicle gives poor results. J Bone Joint Surg Br. 1997;79:537–9.

  8. 8.

    Lazarides S, Zafiropoulos G. Conservative treatment of fractures at the middle third of the clavicle: the relevance of shortening and clinical outcome. J Shoulder Elb Surg. 2006;15:191–4.

  9. 9.

    De Giorgi S, Notarnicola A, Tafuri S, Solarino G, Moretti L, Moretti B. Conservative treatment of fractures of the clavicle. BMC Res Notes. 2011;4:333.

  10. 10.

    Eskola A, Vainionpaa S, Myllynen P, Patiala H, Rokkanen P. Outcome of clavicular fracture in 89 patients. Arch Orthop Trauma Surg. 1986;105:337–8.

  11. 11.

    Jubel A, Schiffer G, Andermahr J, Ries C, Faymonville C. Shortening deformities of the clavicle after diaphyseal clavicular fractures: influence on patient-oriented assessment of shoulder function. Unfallchirurg. 2016;119:508–16.

  12. 12.

    McKee MD, Pedersen EM, Jones C, Stephen DJ, Kreder HJ, Schemitsch EH, Wild LM, Potter J. Deficits following nonoperative treatment of displaced midshaft clavicular fractures. J Bone Joint Surg Am. 2006;88:35–40.

  13. 13.

    Postacchini R, Gumina S, Farsetti P, Postacchini F. Long-term results of conservative management of midshaft clavicle fracture. Int Orthop. 2010;34:731–6.

  14. 14.

    Thormodsgard TM, Stone K, Ciraulo DL, Camuso MR, Desjardins S. An assessment of patient satisfaction with nonoperative management of clavicular fractures using the disabilities of the arm, shoulder and hand outcome measure. J Trauma. 2011;71:1126–9.

  15. 15.

    Godfrey J, Hamman R, Lowenstein S, Briggs K, Kocher M. Reliability, validity, and responsiveness of the simple shoulder test: psychometric properties by age and injury type. J Shoulder Elb Surg. 2007;16:260–7.

  16. 16.

    Inman VT, Saunders JB. Observations on the function of the clavicle. Calif Med. 1946;65:158–66.

  17. 17.

    Terwee CB, Mokkink LB, Knol DL, Ostelo RW, Bouter LM, de Vet HC. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012;21:651–7.

  18. 18.

    Jones GL, Bishop JY, Lewis B, Pedroza AD, Group MS. Intraobserver and interobserver agreement in the classification and treatment of midshaft clavicle fractures. Am J Sports Med. 2014;42:1176–81.

  19. 19.

    Silva SR, Fox J, Speers M, Seeley M, Bovid K, Farley FA, Vanderhave KL, Caird MS. Reliability of measurements of clavicle shaft fracture shortening in adolescents. J Pediatr Orthop. 2013;33:e19–22.

  20. 20.

    Smekal V, Deml C, Irenberger A, Niederwanger C, Lutz M, Blauth M, Krappinger D. Length determination in midshaft clavicle fractures: validation of measurement. J Orthop Trauma. 2008;22:458–62.

  21. 21.

    Archer LA, Hunt S, Squire D, Moores C, Stone C, O’Dea F, Furey A. Plain film measurement error in acute displaced midshaft clavicle fractures. Can J Surg. 2016;59:311–6.

  22. 22.

    Plocher EK, Anavian J, Vang S, Cole PA. Progressive displacement of clavicular fractures in the early postinjury period. J Trauma. 2011;70:1263–7.

  23. 23.

    Liu H, Zhuo W, Chen B, Yi Y, Li D. Patient doses in different projections of conventional diagnostic X-ray examinations. Radiat Prot Dosim. 2008;132:334–8.

  24. 24.

    Cunningham BP, McLaren A, Richardson M, McLemore R. Clavicular length: the assumption of symmetry. Orthopedics. 2013;36:e343–7.

  25. 25.

    Axelrod DSO, Axelrod T, Whyne C, Lubovsky O. Fractures of the clavicle: which X-ray projection provides the greatest accuracy in determining displacement of the fragments? J Orthop Trauma. 2013;13:3.

  26. 26.

    Bhattacharyya RLS, Finn P, Campbell R. Clavicular length measurement following trauma. Injury Extra. 2004;35:22.

  27. 27.

    Sharr JR, Mohammed KD. Optimizing the radiographic technique in clavicular fractures. J Shoulder Elb Surg. 2003;12:170–2.

  28. 28.

    Malik A, Jazini E, Song X, Johal H, O’Hara N, Slobogean G, Abzug JM. Positional change in displacement of midshaft clavicle fractures: an aid to initial evaluation. J Orthop Trauma. 2017;31:e9–e12.

  29. 29.

    Stegeman SA, de Witte PB, Boonstra S, de Groot JH, Nagels J, Krijnen P, Schipper IB. Measurement of clavicular length and shortening after a midshaft clavicular fracture: spatial digitization versus planar roentgen photogrammetry. J Electromyogr Kinesiol. 2016;29:74–80.

  30. 30.

    Backus JD, Merriman DJ, McAndrew CM, Gardner MJ, Ricci WM. Upright versus supine radiographs of clavicle fractures: does positioning matter? J Orthop Trauma. 2014;28:636–41.

  31. 31.

    Stegeman SA, Fernandes NC, Krijnen P, Schipper IB. Reliability of the Robinson classification for displaced comminuted midshaft clavicular fractures. Clin Imaging. 2015;39:293–6.

Download references

Acknowledgements

None

Funding

Not applicable.

Availability of data and materials

The detailed search strategy for this systematic review is available in Additional file 1. The review protocol adhered to by the authors is available in Additional file 2. The PRISMA flowchart and PRISMA checklist are available in Fig. 1 and Additional file 3, respectively.

Disclaimer

None

Author information

Affiliations

  1. Department of Orthopedics, Radboud University Medical Center, P.O. Box 9101, 6500 HB, Nijmegen, The Netherlands

    • Paul Hoogervorst
    • , Gerjon Hannink
    •  & Albert van Kampen

  2. Onze Lieve Vrouwe Gasthuis, Amsterdam, The Netherlands

    • Arnoud R. van Geene

Authors

  1. Search for Paul Hoogervorst in:

  2. Search for Gerjon Hannink in:

  3. Search for Arnoud R. van Geene in:

  4. Search for Albert van Kampen in:

Contributions

PH contributed to the conception and design of the study, acquisition and analysis of the data, and drafting and critical revision of the manuscript. GH contributed to the conception and design of the study, analysis of the data, and critical revision of the manuscript. AG contributed to the conception and design of the study and critical revision of the manuscript. AK contributed to the conception and design of the study and critical revision of the manuscript. All authors read and approved the final version of the manuscript.

Corresponding author

Correspondence to Paul Hoogervorst.

Ethics declarations

Ethics approval and consent to participate

The need for approval by the ethics committee and consent to participate was waived by our institutional review board (CMO Arnhem-Nijmegen).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Search query PubMed, EMBASE, and Cochrane. (DOCX 24 kb)

Additional file 2:

Nonregistered PROSPERO review protocol. ReviewProtocol_MeasurementsClavicle. (PDF 60 kb)

Additional file 3:

PRISMA 2009 checklist. PRISMA 2009 checklist. (PDF 65 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hoogervorst, P., Hannink, G., van Geene, A.R. et al. Reliability of measurements of the fractured clavicle: a systematic review. Syst Rev 6, 223 (2017) doi:10.1186/s13643-017-0614-4

Download citation

Keywords

  • Clavicle
  • Fracture
  • Imaging
  • Shortening
  • Reliability
  • Reproducibility

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.

Systematic Reviews

ISSN: 2046-4053