Skip to main content

Table 2 Adapted GRRAS tool for quality assessment of measurement comparison studies

From: Availability and performance of image/video-based vital signs monitoring methods: a systematic review protocol

Section Original guidelines from GRRAS Scoring
Title and abstract Identify in title or abstract that interrater/intrarater reliability or agreement was investigated N/Aa  
Introduction Name and describe the diagnostic or measurement device of interest explicitly The image-based vital signs monitor is described +
No description
Specify the subject population of interest Specifies the subject population of interest +
No population specified
Specify the rater population of interest (if applicable) N/A  
Describe what is already known about reliability/agreement and provide a rationale for the study Describes what is already known about the reliability of image-based monitoring method and provide a rationale for the study +
No statement on the current knowledge of the method and no rationale stated
Methods Explain how the sample size was chosen. State the determined number of raters/subjects/objects/replicate observations Explains how the sample size was chosen and/or state the determined number of subjects/replicate observationsb +
No explanation of sample size. Number of subjects/replicate observations not stated
Describe sampling method Describes sampling method +
No description of sampling method
Describe the measurement/rating process (e.g. time interval between repeated measurements, availability of clinical information, blinding) Describes the measurement process +
No description of measurement process
State whether measurements/ratings were conducted independently Two (or more) methods of measurements conducted independently +
Measurements not conducted independently
Describe the statistical analysis Describes the statistical analysis planned +
No description of statistical analysis planned
Results State the actual number of raters and subjects/objects which were included and the number of replicate observations which were conducted States the actual number of subjects who were included and number of observations (e.g. duration of recording) +
No statement of the number of subjects/observations
Describe the sample characteristics of raters/subjects N/A  
Report estimates of reliability and agreement including measure of statistical uncertainty Reports estimates of reliability and agreement including measure of statistical uncertainty using gold standard measures +
Reports estimates of reliability and agreement including measure of statistical uncertainty using acceptable standard measures ?
Reports estimates of reliability and agreement including measure of statistical uncertainty but using inappropriate measures/no estimates of reliability/agreement given
Discussion Discuss the practical relevance of results Discusses the practical relevance of results +
No discussion of practical relevance of results
Auxillary material Provide detailed results if possible (e.g. online) N/A  
  1. N/A not applicable
  2. aStudies will not be penalised for not stating the gold standard/reference method of monitoring in the title/abstract as we aim to include studies where this comparison was performed with an alternative primary aim (e.g. improve image analysis protocol)
  3. bPilot studies will not be penalised for not stating how the sample size was chosen