Original link:tecdat.cn/?p=3060
Original source:Tuo End number according to the tribe public number
introduce
In the systematic evaluation of the accuracy of diagnostic tests, the statistical analysis section aims to estimate the average (cross-study) sensitivity and specificity of tests and their variability, as well as other measures. There is often a negative correlation between sensitivity and specificity, indicating the need for relevant data models. Because of the user, the analysis is statistically challenging
- Processing two summary statistics,
- The correlation between sensitivity and specificity must be considered,
- The heterogeneity of sensitivity and specificity in studies must be considered
- Inclusion of covariables should be allowed.
This tutorial introduces and demonstrates a layered hybrid model for meta-analysis of diagnostic accuracy studies. At the first level of the hierarchy, given the sensitivity and specificity of each study, two binomial distributions were used to describe changes in the number of true positive and true positive samples in sick and healthy individuals, respectively. At the second level, we use a binary distribution to model unobserved sensitivity and specificity. Although a hierarchical model was used, the meta-analysis focused on pooled mean values in studies, which were few in a given study estimate.
Using data sets from two previously published meta-analyses to demonstrate these methods:
- Diagnostic accuracy of telomerase in urine as a tumor marker for the diagnosis of primary bladder cancer, because it is a problematic data set, its correlation parameter is estimated to be -1 and there is no covariable causing convergence problems (Glas et al.2003)
- Comparing the sensitivity and specificity of viral tests (using HC2 tests) with repeat cytology to classify women with cervical lesions to detect potential precervical lesions (Arbyn et al., 2013). The second dataset is used to prove a meta-regression with a single covariable that can be naturally extended to include several covariables.
Statistical methods for meta-meta-analysis
Reasoning framework and software
Due to its flexibility and use of MCMC simulations, complex modeling can often be implemented more easily within a Bayesian framework. Bayesian inference can avoid the identification problem by controlling the prior distribution, while numerical approximation algorithms in frequency inference without prior distribution may get into trouble due to the identification problem. However, Bayesian methods generally require statistical expertise and patience because MCMC simulations are computationally intensive. Instead, the most frequent methods have been included in standard “procedures” that require less statistical knowledge and programming skills. In addition, the frequency-theoretic approach is optimized by maximum likelihood estimation (MLE) and has a shorter running time than MCMC simulation.
JAGS (Plummer et al. 2003) is another scalable general-purpose sampling engine for Stan. Extending JAGS requires knowledge of the C++ dynamic link library (DLL) module. From experience, configuring and building modules is a difficult and tedious task, especially in Windows operating systems. These shortcomings plus the fact that Stan tends to converge with fewer iterations even if it starts from poor initial values.
Model diagnosis
To evaluate the model convergence and stationarity of the chain, it is necessary to examine potential scaling factors, effective sample sizes (ESS), MCMC errors, and tracking graphs of parameters. When all the chains reach the target posterior distribution, the estimated posterior square difference is close to the chain variance, making the ratio of the two close to 1, indicating that the chain is stable and the target distribution may have been reached. An effective sample size represents how much information is actually available about a parameter. When samples are automatically correlated, the information about the posterior distribution of desired parameters is less than that when samples are independent. Because the posterior distribution is simulated, the approximate value may deviate from a certain amount. MCMC error close to 0 indicates that the target distribution may have been reached.
Model comparison and selection
The Watanabe-Alkaike Information Criterion (WAIC) (Watanabe 2010) is a recent model comparison tool for measuring the prediction accuracy of fitted models in a Bayesian framework for comparing models. WAIC can be seen as an improvement on Deviance Information Criterion (DIC) which, despite its popularity, has had some problems (Plummer 2008). WAIC is a fully Bayesian tool, very close to Bayesian cross validation, invariant to reparameterization, and can be used for simple as well as layered and mixed models.
The data set
Telomerase data
(Glas et al., 2003) systematically reviewed the sensitivity and specificity of cytology and other markers (including telomerase) for the initial diagnosis of bladder cancer. They reported that telomerase had a sensitivity and specificity of 0.75, and 0.86, respectively. They concluded that telomerase was not sensitive enough for everyday use.
Load the data into the R environment and generate the following outputCopy the code## ID TP TN FN FP ## 1 1 25 25 8 1 ## 2 2 17 11 4 3 ## 3 3 88 31 16 16 ## 4 4 16 80 10 3 ## 5 5 40 137 17 1 ## 6 6 38 24 2 ## 9 9 14 29 3 3 ## 10 10 37 7 7 22Copy the codeID is the study identifier, DIS is the number of patients, TP is the number of true positive NonDis, is the number of healthy TN, is the number of true negative.
ASCUS categorizes data
A Cochrane review of the accuracy of HPV testing was performed (Arbyn et al., 2013), and cytological analysis was repeated for cervical smears to diagnose cervical precancerous lesions. They SAS METADAS 10 studies using BRMA models, in which two tests were used. Their absolute sensitivity for HC2 and repeat cytology was 0.909 [0.857, 0.944] and 0.715 [0.629, 0.788], respectively. This data is used to demonstrate how to extend the intercept model in regression Settings. Load the data into the R environment and generate the following output
## Test StudyID TP FP TN FN
## 1 RepC Andersson 2005 6 14 28 4
## 2 RepC Bergeron 2000 8 28 71 4
## 3 RepC Del Mistro 2010 20 191 483 7
## 4 RepC Kulasingam 2002 20 74 170 6
## 5 RepC Lytwyn 2000 4 20 26 2
## 6 RepC Manos 1999 48 324 570 15
## 7 RepC Monsonego 2008 10 18 168 15
## 8 RepC Morin 2001 14 126 214 5
## 9 RepC Silverloo 2009 24 43 105 10
## 10 RepC Solomon 2001 227 1132 914 40
## 11 HC2 Andersson 2005 6 17 25 4
## 12 HC2 Bergeron 2000 10 38 61 2
## 13 HC2 Del Mistro 2010 27 154 566 2
## 14 HC2 Kulasingam 2002 23 115 129 3
## 15 HC2 Lytwyn 2000 4 19 33 1
## 16 HC2 Manos 1999 58 326 582 7
## 17 HC2 Monsonego 2008 22 110 72 2
## 18 HC2 Morin 2001 17 88 253 2
## 19 HC2 Silverloo 2009 34 65 81 2
## 20 HC2 Solomon 2001 256 1050 984 11
Copy the codeTest is an explanatory variable showing the type of classified tests, StudyID is the study identifier, TP is the number of true positives, TN is the number of true negatives, and FN is the number of false negatives.
Intercept model
By default, Chains = 3 ‘ ‘cores = 3. From the code above, take 1000 samples from each of the 3 chains, discard the first sample, and then make each chain have 900 “pre-burn periods” before taking. The seed value seed = 3 specifies the random number generator to allow reproducibility of the results, and cores = 3 allows parallel processing of chains by using three cores, one for each chain.
The trace plot below shows the chain and convergence.
Next, the following model outline estimates are obtained
## Parameter Mean Lower Median Upper n_eff Rhat ## MUse[1] Sensitivity 0.756762 6.904E-01 0.756036 0.99958 0.99989 1.001 ## ktau[1] Correlation-0.820176-9.861e-01-0.876343-0.33334 269.2 1.015 ## Varse[1] Var(Sens) 0.006198 8.321E-06 0.005047 ## Varsp[1] Var(Spec) 0.048111 1.357e-02 0.041060 0.12204 169.5 1.007 ## ## ## Model features ## ## Copula function: gauss, sampling algorithm: NUTS(diag_e) ## ## Formula(1): MUse ~ 1 ## Formula(2): MUsp ~ 1 ## Formula(3): Omega ~ 1 ## 3 chain(s)each with iter=28000; warm-up=1000; thin=30. ## post-warmup draws per chain=900; Total post-Warmup Draws =2700. #### Model prediction accuracy #### Log Point-wise Predictive density (LPPD): ## Watanabe-Akaike Information Criterion (WAIC): 91.2828Copy the codeFrom the output above, the meta-analysis sensitivity MUse[1] and specificity MUsp[1] are 0.7568 [0.6904, 0.8166] and 0.7983 [0.6171, 0.9064]. The sensitivity and specificity were 0.0062 [0, 0.0195] and 0.0048 [0.0136, 0.1220], respectively. Kendall tau correlation between sensitivity and specificity was estimated to be -0.8202 [-0.9861, -0.3333].
The following command generates a series of forest maps.
## $G1
Copy the code
##
## $G2
Copy the code
##
## $G3
## Warning: Removed 2 rows containing missing values (geom_errorbar).
Copy the code
$G1 is a graph that studies specific sensitivity and specificity (magenta dots) and their corresponding 95% confidence intervals (black lines). $G2 is a plot of posterior study sensitivity and specificity and their corresponding 95% confidence intervals (black line).
$G3 is a plot of posterior study sensitivity and specificity and their corresponding 95% confidence intervals (black lines). Sensitivity and specificity (magenta dot) and corresponding 95% confidence interval (thick gray line) for study specificity are also given.
As shown in the figure above, there is a “contraction” in the overall mean sensitivity and specificity: the estimate of a posterior study depends on the global estimate, and therefore on all other studies.
Next, prepare the data by creating the following list
In the data block, the dimension and name of variables in the data set are specified, where Ns indicates the number of studies in the data set. The Parameters block introduces unknown parameters to be estimated. etarho; A scalar representing the form of the correlation parameter of Fisher’s transformation, the sensitivity and specificity of the MUL representation is the logarithmic mean of the central observation, where the random effect is vector zero.
Transform the parameters further in the Transformed Parameters block. Define prior distributions of all parameters and data likelihood in the Model block. Finally, in generated chunks, Loglik is the logarithmic likelihood vector required to calculate WAIC.
Next, Stan calls a function to convert the code to C++, compile the code and extract a sample from the posterior distribution, extract the parameter estimates as shown below, and further check for chain convergence and autocorrelation using the following code
## Inference for Stan model: 61572683b29d52354783115614fab729. ## 3 chains, each with iter=5000; warmup=1000; thin=10; ## post-warmup draws per chain=400, Total post-warmup Draws =1200. ## ## mean se_mean SD 2.5% 50% 97.5% n_eff Rhat ## MU[1] 0.7525 0.0018 0.0517 0.6323 0.7562 0.8415 796 0.9999 ## MU[2] 0.7908 0.0034 0.1095 0.5273 0.8094 0.9539 1045 1.0008 ## MU[1] 0.7668 0.0013 0.0388 0.6869 0.7688 0.8369 891 0.9990 ## mu[2] 0.8937 0.0027 0.0753 0.6943 0.9115 0.9825 789 0.9992 ## ho-0.9311 0.0070 Sigma[1,1] 0.3376 0.0091 0.2918 0.0579 0.2554 0.9851 1026 1.0023 ## Sigma [1, 2] 1.2291 0.0272 0.8765 3.4195 1.0031 0.2724 0.9991 # 1040 # Sigma (2, 1] 1.2291 0.0272 0.8765 3.4195 1.0031 -0.2724 1040 0.9991 ## Sigma[2,2] 5.6827 0.1282 4.1931 1.4720 4.6330 16.9031 1070 1.0002 ## ## Samples were drawn using NUTS(diag_e) at Mon Oct 09 09:19:55 2017. ## For each parameter, n_eff is a crude measure of effective sample size, ## and Rhat is the potential scale reduction factor on split chains (at ## convergence, Rhat=1).Copy the codeThe meta-analysis sensitivity (MU[1]) and specificity (MU[2]) and 95% confidence interval are 0.7525[0.6323, 0.8415] and 0.7908[0.5273, 0.9539]. This is different from an article published by the author in two ways (0.75 [0.66,0.74] and 0.86 [0.71,0.94]). The authors fit the sensitivity and specificity values of the standard bivariate normal distribution into the Logit transformation, allowing heterogeneity between studies in the study and ignoring higher-level stratified models. Therefore, the authors had to use a continuity correction of 0.5, a problem not encountered in the layered model.
The figure below shows that most of the fitting models, except the Clayton Copula model, have satisfactory chain mixing with little autocorrelation.
The average sensitivity and specificity of all fitted distribution estimates are shown in the table below.
## Warning: ## We recommend trying loo(). ## Model Parameter Mean Lower Median Upper n_eff ## 1 Gaussian Sensitivity 0.756762 6.904E-01 0.756036 8.166E-01 1196.857 ## 2 Gaussian Specificity 0.798289 6.171E-01 0.813517 9.064E-01 704.379 ## 3 Gaussian Correlation - 0.820176-9.861E-01-0.876343-3.333E-01 269.179 ## 4 Gaussian Var(Sens) 0.006198 8.321E-06 0.005047 1.947E-02 165.705 ## 5 Gaussian Var(Spec) 0.048111 1.357E-02 0.041060 1.220E-01 169.508 ## 9 C90 Sensitivity 0.751379 6.913E-01 0.753546 8.098E-01 25.638 ## 7 C90 Specificity 0.807051 6.549E-01 0.821119 9.069E-01 119.897 ## 8 C90 correlation-0.528340-9.766E-01-0.725178-4.020E-18 4.111 ## 9 C90 Var(Sens) 0.004885 3.400e-04 0.003297 1.955E-02 11.615 ## 10 Var(Spec) 0.045694 1.556e-02 0.038049 1.020e-01 137.149 # 9 C270 Sensitivity 0.757528 6.877E-01 0.761163 8.210E-01 273.236 # 9 C270 Specificity 0.803502 6.328E-01 0.811740 # 6 C270 Var(Sens) # 7 C270 Var(Sens) # 7 C270 Var(Sens) C270 Var(Spec) 0.044767 1.268e-02 0.037922 1.112e-01 1098.815 ## 16 FGM Sensitivity 0.759407 6.891E-01 0.761931 8.174E-01 2475.208 ## 17 FGM Specificity 0.802588 6.453E-01 0.812498 9.045E-01 2293.332 ## 18 FGM Correlation - 0.174538-2.222E-01-0.222221 2.222E-01 785.016 ## 19 FGM Var(Sens) 0.005390 7.425e-07 0.004181 1.813e-02 1019.633 ## 20 FGM Var(Spec) 0.041890 1.177e-02 0.036671 9.997e-02 2479.371 ## 21 Frank 7.425e-07 0.004181 1.813e-02 1019.633 ## 20 FGM Var(Spec) 0.041890 1.177e-02 0.036671 9.997e-02 2479.371 ## 21 Frank Sensitivity 0.756683 6.855E-01 0.758340 8.152E-01 2686.631 ## 22 Frank Specificity 0.808239 6.472E-01 0.818777 9.110E-01 1910.561 ## 23 Frank Var(Sens) 0.706819-8.550e-01-0.692019 1.000e+00 2700.000 ## 24 Frank Var(Sens) 0.006678 5.896e-04 0.005280 2.140e-02 2699.766 ## 25 Frank Var(Spec) 0.042067 1.201E-02 0.035908 1.039e-01 1937.653 ## 26 BRMA Sensitivity 0.752531 6.323E-01 0.756181 8.415E-01 796.037 ## 27 BRMA Specificity 0.790796 5.273E-01 0.809420 9.539E-01 1044.902 ## 29 BRMA Var(lSens) 1044.902 ## 29 BRMA Var(lSens) 1044.902 ## 28 BRMA Var(lSens) 0.337556 BRMA Var(lSpec) 5.682692 1.472e+00 4.632967 1.690e+01 1070.481 ## Rhat WAIC ## 1 1.0014 91.28 ## 2 1.0044 91.28 ## 3 1.0154 91.28 ## 4 1.0072 91.28 ## 5 1.0069 91.28 ## 6 1.1047 91.40 ## 7 1.0304 91.40 ## 8 1.3707 91.40 ## 9 1.1005 91.40 ## 10 1.0311 91.40 ## 11 1.0096 90.75 ## 12 1.0001 90.75 ## 13 1.0407 90.75 ## 14 1.0024 90.75 ## 15 0.9999 90.75 ## 16 0.9998 97.37 ## 17 0.9996 97.37 ## 18 1.0070 97.37 ## 19 1.0034 97.37 ## 20 0.9999 97.37 ## 21 0.9994 90.55 ## 22 0.9997 90.55 ## 23 NaN 90.55 ## 24 0.9990 90.55 ## 25 0.9992 90.55 ## 26 0.9999 86.76 ## 27 1.0008 86.76 ## 28 1.0218 86.76 ## 29 1.0023 86.76 ## 30 1.0002 86.76Copy the codeThe results are graphically shown below
Model to compare
As shown above, the correlation between the BRMA model and the Gaussian Copula bivariate β estimates is more extreme. The other extreme is model FGM copulas connect bivariate beta estimates, this is because the FGM copulas connect associated parameters, in which value within 2/9 | |.
In G1 in the figure above, the marginal mean sensitivity and specificity of the five bivariate β distributions are comparable to the slight differences in the 95% confidence interval, despite differences in the relevant structures.
In the absence of estimation difficulties, the above table shows a correlation of -0.8224[-0.9824, -0.3804] estimated by Pearson. This is because the Bayesian method is not affected by sample size and is therefore able to handle small sample sizes with fewer problems.
Basically, all six models are equivalent in the first level hierarchy, and the prior distributions of sensitivity and specificity differ in specifying “study specificity”. Therefore, models should have the same number of parameters, in which case it makes sense to compare the predicted densities. At the time of examination, the logarithmic predicted densities from the five Copula-based models were virtually equivalent (min=-38.77, Max =-37.89) but the effective numbers of parameters were slightly different (min=7.25, Max =9.92).
meta-regression
The AScus dataset has Test as a covariable. It makes sense to use covariates to study their effect on the combined distribution of sensitivity and specificity, including correlation. The following is a fitting of the copula-based binary beta binomial distribution to asCUS data.
The following figure shows tractions for all six models applicable to the ASCUS data, where all parameters (including related parameters (except BRMA)) are modeled as functions of covariables. There is proper chain mixing and convergence, except in the case of the Clayton Copula-based bivariate β.
From the bivariate β distribution based on Copula, it is clear that the correlation between sensitivity and specificity in HC2 and repeat cytology is different.
## Warning: P_waic estimates greater than 0.4. ## We recommend trying loo() instead. ## Median Upper ## 9 Gaussian HC2 Correlation - 0.43812-0.9984-6.959E-01 9.847E-01 ## 10 Gaussian Repc Correlation -0.91991-0.9997-9.643E-01-6.103E-01 ## 23 C90hC2 Correlation -0.06588-0.7610-1.039e-17-7.624e-19 ## 24 C90 Repc Correlation-0.85157-0.9804-9.120e-01-4.906e-01 ## 37 C270 HC2 correlation-0.03038-0.6452-7.000E-18-1.782E-18 ## 38 C270 Repc correlation-0.77847-0.9757-7.058E-01-5.394E-01 ## 51 FGM HC2 correlation-0.07618-0.22222-2.21501 2.222E-01 ## 52 FGM Repc Correlation 0.19819-0.2222-2.222E-01 1.894E-01 ## 65 Frank HC2 Correlation 0.48806-0.8140 4.497 e-01 1.000 e+00 # # 66 Frank Repc Correlation - 0.73784-0.8627-7.275 e-01 1.000 e+00 BRMA # # 81 to Both the Correlation -0.84808-0.9839-8.98001-4.497E-01 ## n_eff Rhat WAIC ## 9 154.238 1.0066 236.4 ## 10 24.342 1.0542 236.4 ## 23 30.089 1.0690 235.7 ## 24 8.532 1.1128 235.7 ## 37 76.410 1.0326 227.5 ## 38 2.945 1.4613 227.5 ## 51 2422.935 1.0007 245.1 ## 52 2550.145 0.9997 245.1 ## 65 2700.000 NaN 238.3 ## 66 2700.000 NaN 238.3 ## 81 102.983 1.0254 233.7Copy the codeThe Clayton90 model had the lowest WAIC.
Therefore, this example shows that it is crucial to check the adequacy of the fit and rationality of the model, rather than blindly relying on information criteria to select the criteria that best fit the data.
From the posterior relative sensitivity and specificity plotted below, all convergent models are generally considered to be less sensitive to repeat cytology than HC2, while specificity is not significantly lost.
discuss
Copula-based models offer great flexibility and ease of use, but they need to be used with caution. Although copulas used in this paper are attractive because they are mathematically tractable, (Mikosch 2006) and (Genest and Remillard 2006) point out that estimating copulas from data can be difficult. In addition, the concepts behind the Copula model are slightly more complex and therefore require statistical expertise to understand and program because they are not yet standard procedures in statistical software.
In this paper, several METAT statistical models for diagnostic accuracy are briefly discussed.
There are some differences between models in assessing the sensitivity and specificity of meta-analyses, as well as correlations. Therefore, it is necessary to further investigate the influence of certain parameters, such as study number, sample size and joint distribution specified on the estimation of the meta-analysis.
conclusion
The proposed Bayesian model uses Copula to construct binary β distributions, which estimate the sensitivity and specificity of a particular study, with specific random effect values.
In the ASCUS classification data, the conclusion based on the fitting model was consistent with the authors’ conclusion that HC2 was more sensitive but less severe than repeat cytology and was not significantly lower than specific Pap smears in the diagnosis of cervical precancerous lesions in women.
Although BRMA had the lowest WAIC for both datasets, we still recommend using bivariate β distributions for modeling sensitivity and specificity, as they can directly provide meta-analysis estimates.
Thank you very much for reading this article, please leave a comment below if you have any questions!
Most welcome insight
1. Apriori association algorithm in Python – market basket analysis
2. The R language survival curve drawing estimate | | how to survive R for survival analysis graph
3. Use association rule data mining to explore the rules of drug compatibility
4. Mining association rules using the Apriori algorithm in Python
5. Use association rule data mining to explore the rules of drug compatibility
6. SPSS Modeler Web complex network was used to analyze all acupoints
7. How to calculate IDI and NRI indices in survival analysis and COX regression for R language
8. How does R language find indicators with differences in patient data? (PLS-DA analysis)
9. Survival analysis of R language in 4 patients with advanced lung cancer