### Addressing heterogeneity in random-effects meta-analysis

The RE model assumes differences in the treatment effects {\theta}_{i} across *k* studies. Hence, the estimation and presentation of the average effect and its CI alone are insufficient. It is also important to quantify the heterogeneity between the effect sizes. The following measures are often used for this purpose: the between-study variance {\tau}^{2}, which can be estimated by various methods [10, 11]; the *Q* statistic, which is a measure of weighted squared deviations; or *I*
^{2}, which describes the proportion of the total variance of the study effects due to heterogeneity [1, 12, 13]. One way to present the dispersion of the study effects graphically is to add the PI to the forest plot of RE meta-analyses.

Under the assumption that both the RE and the estimated average effect are approximately normally distributed, that is:

{\theta}_{i}\sim N\left(\theta ,{\tau}^{2}\right),\stackrel{\wedge}{\theta}\sim N\left(\theta ,SE{\left(\stackrel{\wedge}{\theta}\right)}^{2}\right)\text{,}

(1)

Higgins *et al*. [7] suggest that the PI is:

\left[\stackrel{\wedge}{\theta}-{t}_{1-\alpha /2;k-2}\sqrt{{\stackrel{\wedge}{\tau}}^{2}+\stackrel{\wedge}{SE}}{\left(\stackrel{\wedge}{\theta}\right)}^{2};\theta +{t}_{1-\alpha /2;k-2}\sqrt{{\stackrel{\wedge}{\tau}}^{2}+\stackrel{\wedge}{SE}}{\left(\stackrel{\wedge}{\theta}\right)}^{2}\right],

(2)

where {t}_{1-\alpha /2;k-2} is the (1 − α/2) quantile of the *t*-distribution with *k-2* degrees of freedom, and \stackrel{\wedge}{\tau} and \stackrel{\wedge}{SE}\left(\stackrel{\wedge}{\theta}\right) denote the estimated between-study variation and the standard error of \stackrel{\wedge}{\theta} respectively. Applying a *t*-distribution instead of a normal distribution reflects the uncertainty resulting from the estimation of *τ*.

However, the assumption that the true effects are normally distributed may not be justified. In these situations the choice of a different distribution [14] may be appropriate, leading to a different PI.

In contrast to the commonly presented CI, which quantifies the precision of the estimated average effect, the PI includes the effect of an individual study, with the level of confidence (1 − α). It is important to note that the PI provides no information on the statistical significance of \stackrel{\wedge}{\theta}.

The PI should be presented graphically in the forest plot of the RE meta-analyses. In such an extended forest plot, the degree of heterogeneity is illustrated, and a clear visual distinction is made between the results of the FE and the RE meta-analyses.

### Modified extension of the forest plot

Forest plots are a graphical presentation of the results derived from a meta-analysis. They allow a rapid overview of the potential heterogeneity of the studies analyzed. In conventional forest plots, the effect measures of the *k* studies with the corresponding CI are represented by a square with horizontal lines, in which the size of the squares reflects the weight that each study contributes to the meta-analysis. Below the results of the individual studies, the average estimate and its CI are displayed as a diamond, whose centre (vertical line) indicates the point estimate and whose width indicates the CI.

To date, the PI has not been part of the common layout of forest plots: However, some proposals to include PIs have been made. Figure 1a shows the proposal by Higgins *et al*. [7], in which the PI is illustrated as a hollow diamond. Riley *et al*. [8] suggest a different presentation in which the confidence and PIs are ‘merged’ (Figure 1b). The point estimate and CI are shown in the typical form of a diamond, and are then extended by lines on both sides representing the width of the PI. Borenstein *et al*. [9] displayed the PI in the same way and, for explanatory purposes, added a truncated bell-shaped curve based on the assumption of a normal distribution.

We propose an alternative graphical approach based on the original suggestion by Skipka [6], which can be considered a mixture of the approaches described. As described previously, the row ‘total expectation (95% CI)’ represents the point and interval estimates for *θ* in the form of a diamond. We have added a new row, ‘95% prediction interval’, to the forest plot, illustrating the corresponding interval in an easily distinguishable way in the form of a rectangle (Figure 1c).