Abstract
Introduction
The aim of the study was a comparative evaluation of in-house real-time PCR and commercial real-time PCR (Fast Track Diagnostics (FTD), ampliCube/Mikrogen) targeting enteropathogenic bacteria from stool in preparation of Regulation (EU) 2017/746 on in vitro diagnostic medical devices.
Methods
Both 241 stool samples from patients and 100 samples from German laboratory control schemes (“Ringversuche”) were used to comparatively assess in-house real-time PCR, the FTD bacterial gastroenteritis kit, and the ampliCube gastrointestinal bacterial panels 1&2 either with the in-house PCRs as gold standard and as a test comparison without gold standard applying latent class analysis. Sensitivity, specificity, intra- and inter-assay variation and Cohen’s kappa were assessed.
Results
In comparison with the gold standard, sensitivity was 75–100% for strongly positive samples, 20–100% for weakly positive samples, and specificity ranged from 96 to 100%. Latent class analysis suggested that sensitivity ranges from 81.2 to 100% and specificity from 58.5 to 100%. Cohen’s kappa varied between moderate and nearly perfect agreement, intra- and inter-assay variation was 1–3 to 1–4 Ct values.
Conclusion
Acceptable agreement and performance characteristics suggested replaceability of the in-house PCR assays by the commercial approaches.
Background
With impeding enactment of Regulation (EU) 2017/746 on in vitro diagnostic medical devices, in-house diagnostic assays will have to be replaced by commercially available assays unless an appropriate performance level cannot be met by assays available on the market. At present, well-established molecular diagnostic assays are available for many target organisms with relevance for the microbiological laboratory. However, not all assays are accredited for all diagnostic platforms and nucleic acid extraction strategies, making customized diagnostic solutions challenging if stockpiling of a broad variety of different diagnostic platforms within the diagnostic laboratory shall be avoided.
For the diagnosis of infectious enteritis, broad-spectrum multiplex real-time PCR assays have been developed and evaluated [1]. While such assays were formerly applied for screenings or outbreak investigations in settings, where traditional culture-based diagnostic approaches were unfeasible due to logistic reasons [2], they have nowadays been established in the routine diagnostic laboratory for cost saving purposes. If only PCR-positive samples are subjected to labour-intensive diagnostic culture in hospital laboratories, the workload of skilled and thus expensive laboratory personnel can be reduced. This strategy is not unusual in hospital laboratories.
For example, at the Tropical Microbiology Subdepartment of the Bundeswehr Hospital Hamburg, Germany, an in-house real-time PCR targeting Salmonella spp., Shigella spp./enteroinvasive Escherichia coli (EIEC), Campylobacter jejuni, and Yersinia spp. [3] next to a Phocid Herpes virus DNA-base internal control [4] on RotorGene 6000/RotorGene Q cyclers (Qiagen, Hilden, Germany) after standardized nucleic acid extraction from stool samples applying the QIAamp DNA Stool Mini Kit (Qiagen) has been used for more than a decade for the screening of stool samples provided by military returnees from the tropics. This in-house assay has a number of disadvantages. Associated with low melting temperatures of the oligonucleotides, high limits of detection of 104 to 105 bacterial copies have been recorded [3]. In comparison, commercial assays targeting Shigella spp./EIEC have shown considerably lower cycle threshold (Ct) values when applied with identical samples [2]. However, a previous evaluation under diagnostic real-life conditions had indicated acceptable sensitivity and specificity when applied with stool from patients with gastroenteritis [3].
However, with the enactment of Regulation (EU) 2017/746, which is expected for the middle of 2022, diagnostic use of in-house diagnostic approaches will be unfeasible if commercial CE-IVD-(Conformité Européenne – in vitro diagnostics-)accredited test assays with comparable performance characteristics are available. Accordingly, studies comparing commercially available test assays with the presently applied in-house diagnostic approaches are desirable and the results of respective studies may guide decisions regarding the future use of commercial test platforms.
In the study presented here, we have assessed two commercial real-time multiplex PCR assays targeting enteropathogenic bacteria with a well-established in-house real-time PCR protocol. Aim of the assessment was the proof of at least comparable performance characteristics to allow a switch to the commercial approaches in line with Regulation (EU) 2017/746 on in vitro diagnostic medical devices.
Methods
Samples
Anonymized nucleic acid extraction residuals from stool samples assessed at the diagnostic laboratory of the Tropical Microbiology Subdepartment of the Bundeswehr Hospital Hamburg, Germany, located at the German National Reference Centre for Tropical Pathogens Bernhard Nocht Institute for Tropical Medicine Hamburg, were included in the test comparison. Those residual sample materials were either from screening assessments of German soldiers and policemen returning from tropical deployments [5, 6], from migrants travelling under poor hygiene conditions [7] or from studies conducted in resource-limited tropical settings (unpublished data) to ensure high proportions of positive samples for test comparison purposes. In addition, well-characterized sample materials from German external laboratory control schemes (“Ringversuche” by INSTAND e.V., Düsseldorf, Germany) were included. In total, 341 samples were assessed, comprising 241 patient samples and 100 “Ringversuch” samples. Details regarding the case definitions for the gold standard-based test comparison approach are presented in the “Case definitions” sub-heading below.
Nucleic acid extraction and sample storage
All diagnostic stool samples and “Ringversuch” materials had been subjected to standardized nucleic acid extraction applying the QIAamp DNA Stool Mini Kit-based nucleic acid extraction scheme as described by the manufacturer. The nucleic acid extractions had been stored frozen at −80 °C prior to the analyses.
Applied PCR schemes
The included residual materials from diagnostically assessed stool samples were assessed by the abovementioned in-house multiplex real-time PCR targeting Salmonella spp., Shigella spp./EIEC, C. jejuni, and Yersinia spp. as well as by an in-house real-time PCR targeting enterotoxigenic Escherichia coli (ETEC) on RotorGene Q cyclers (Qiagen, Hilden, Germany) as previously described [3, 8]. Both in-house PCR assays were applied with a Phocid Herpes virus DNA-based internal control [4]. In house testing was done in parallel with the commercial PCR runs to exclude effects of discrepant sample age.
Due to similar spectra of target pathogens as well as compatibility with the RotorGene cyclers and the QIAamp DNA Stool Mini Kit-based nucleic acid extraction scheme as confirmed by the manufacturers, comparatively assessed commercial real-time PCR assays comprised:
The Fast Track Diagnostics (FTD) bacterial gastroenteritis kit (Siemens, Berlin & Munich, Germany) targeting Campylobacter coli/jejuni/lari, Clostridioides difficile, enterohemorrhagic E. coli (EHEC), Salmonella spp., Shigella spp./EIEC, Yersinia enterocolitica, and an internal control.
The ampliCube gastrointestinal bacterial panel 1 (Mikrogen, Neuried, Germany) targeting Campylobacter spp., Salmonella spp., Y. enterocolitica as well as the ampliCube gastrointestinal bacterial panel 2 targeting EHEC, enterotoxigenic E. coli (ETEC), and Shigella spp./EIEC, both assays combined with an internal control.
Case definitions
For the purpose of a gold standard-based test comparison, samples from the diagnostic routine that had either been positive for any of the targets from the in-house multiplex real-time PCR targeting enteroinvasive pathogens described above [3] or for ETEC in another in-house real-time PCR [8] were included as positive controls. For the evaluation of the PCR targets C. difficile and EHEC, which had not been assessed in the diagnostic routine at our subdepartment so far, well-characterized samples from German external laboratory control schemes (“Ringversuche”) were used as positive controls.
Those well-characterized “Ringversuch” materials were also considered as negative control materials for PCR assays targeting infectious agents not abundant in the “Ringversuch” sample materials, while samples from the routine diagnostics were not considered as confirmed negative. Accordingly, only “Ringversuch” sample materials were used as negative controls in the gold standard-based test comparison.
Test comparison strategy with and without gold standard as well as statistics
Based on the abovementioned assumptions, a gold standard-based test comparison assessing diagnostic sensitivity, specificity, intra- and inter-assay variation as described previously [9] was performed. Thereby, strongly and weakly positive samples in the gold standard assessments were used for sensitivity assessments with cycle threshold (Ct) value 34 in the gold standard PCRs as the cut-off to discriminate between each other. In analogy to the detection thresholds of the in-house PCRs, copy numbers smaller than 105 were considered as weakly positive for the “Ringversuch” samples.
In addition to the gold standard-based assessment, the results from all samples were subjected to latent class analysis (LCA) [10, 11] as a test comparison without a gold standard next to calculation of Cohen’s kappa as described [12].
Ethics
As clarified previously (WF-011/19) by the Ethics Committee of the Medical Association of Hamburg, Germany, in line with German National Laws, use of anonymized residual sample materials for test comparison purposes in diagnostic laboratories is neither a “research project involving human beings” in line with § 9 chapter 2 of the “Hamburgisches Kammergesetz für Heilberufe” (Hamburg’s Association of Health Care Professions Act) nor a research project requiring ethical advice according to §15 chapter 1 of the “Berufsordnung für Hamburger Ärzte und Ärztinnen” (Professional Regulations for Physicians in Hamburg). Therefore, ethical clearance was not required for this assessment.
Results
Results of the gold standard-based test comparison
In the gold standard-based assessment, the FTD assays’ sensitivity ranged from 75 to 100% for both the strongly and weakly positive samples, while the ampliCube assays showed 87.3–100% sensitivity for strongly positive and 20–100% sensitivity for weakly positive ones. Thereby, sensitivity of weakly positive samples could not be calculated for all parameters due to lacking appropriate sample materials. Gold standard-based specificity using the “Ringversuch” samples ranged from 98.6 to 100% for the FTD assays as well as 96.0 to 100% for the ampliCube assays. Inter- and intra-assay variation for the FTD and ampliCube assays were quite similar, ranging from 1 to 3 Ct values for in-house and FTD assays and from 1 to 4 Ct values for ampliCube assays. Again, samples with Ct values >34 were quite rare, so samples slightly lower but close to this value were accepted as weakly positive samples for this assessment as well.
Focussing on the FTD platform, sensitivity of 100% was recorded for strongly and weakly positive samples with C. jejuni and Shigella spp./EIEC, respectively. Sensitivity >90% was seen for strongly positive samples with Yersinia spp. and Salmonella spp., respectively, sensitivity >80% for both strongly and weakly positive samples with EHEC and sensitivity >70% for both strongly and weakly positive samples with C. difficile. Specificity was 100% with the exception of the assays for Y. enterocolitica (99.0%) and C. difficile (98.6%).
Focussing on the ampliCube platform, sensitivity of 100% was detected for weakly positive samples with C. jejuni as well as for strongly positive samples with ETEC. Sensitivity >90% was seen for strongly positive samples with C. jejuni, Yersinia spp., and Shigella spp./EIEC, respectively, sensitivity >80% for strongly positive samples with Salmonella spp. and EHEC, sensitivity ≥50% for weakly positive samples with Shigella spp./EIEC and EHEC, and sensitivity ≥20% for weakly positive samples with ETEC. Recorded specificity was 100% in all instances with the exception of the Campylobacter spp.-specific assay (96.0%).
Details of the gold standard-based assessment are provided in Tables 1 and 2.
Sensitivity and specificity compared with the diagnostic gold standard (in-house PCR or materials from laboratory control schemes)
| Assay/target organism | Sensitivity with strongly positive samples in % (n/n) | Sensitivity with weakly positive samples in % (n/n) | Specificity in % (n/n) |
| FTD/Campylobacter coli/jejuni/lari | 100.0 (97/97) | 100 (2/2) | 100 (100/100) |
| FTD/Clostridioides difficile | 75.0 (6/8) | 75.0 (12/16) | 98.6 (73/74) |
| FTD/enterohemorrhagic E. coli (EHEC) | 88.2 (15/17) | 82.4 (14/17) | 100 (72/72) |
| FTD/Salmonella spp. | 94.9 (75/79) | n.a. | 100 (97/97) |
| FTD/Shigella spp./enteroinvasive E. coli (EIEC) | 100 (18/18) | 100 (2/2) | 100 (99/99) |
| FTD/Yersinia enterocolitica | 91.7 (11/12) | n.a. | 99.0 (99/100) |
| ampliCube/Campylobacter spp. | 97.9 (95/97) | 100 (2/2) | 96.0 (96/100) |
| ampliCube/Salmonella spp. | 87.3 (69/79) | n.a. | 100 (97/97) |
| ampliCube/Yersinia enterocolitica | 91.7 (11/12) | n.a. | 100 (100/100) |
| ampliCube/enterohemorrhagic E. coli (EHEC) | 88.2 (15/17) | 58.8 (10/17) | 100 (71/71) |
| ampliCube/enterotoxigenic E. coli (ETEC) | 100 (21/21) | 20.0 (1/5) | 100 (97/97) |
| ampliCube/Shigella spp./enteroinvasive E. coli (EIEC) | 94.4 (17/18) | 50 (1/2) | 100 (98/98) |
Intra- and inter-assay-variation. Ct = cycle threshold
| Assay/target organism | Assessed sample category | Intra-assay variation in total Ct-values | Inter-assay variation in total Ct-values |
| In-house enterotoxigenic E. coli (ETEC) | Strongly positive samples | 1 | 1 |
| Weakly positive samples | 1 | 2 | |
| Negative samples | 0 | 0 | |
| In-house Salmonella spp. | Strongly positive samples | 0 | 1 |
| Weakly positive samples | 1 | 2 | |
| Negative samples | 0 | 0 | |
| In-house Shigella spp./enteroinvasive E. coli (EIEC) | Strongly positive samples | 0 | 1 |
| Weakly positive samples | 3 | 0 | |
| Negative samples | 0 | 0 | |
| In-house Campylobacter jejuni | Strongly positive samples | 1 | 0 |
| Weakly positive samples | 1 | 3 | |
| Negative samples | 0 | 0 | |
| In-house Yersinia spp. | Strongly positive samples | 1 | 2 |
| Weakly positive samples | 2 | 2 | |
| Negative samples | 0 | 0 | |
| FTD/Campylobacter coli/jejuni/lari | Strongly positive samples | 0 | 1 |
| Weakly positive samples | 2 | 2 | |
| Negative samples | 0 | 0 | |
| FTD/Clostridioides difficile | Strongly positive samples | 0 | 1 |
| Weakly positive samples | 3 | 2 | |
| Negative samples | 0 | 0 | |
| FTD/enterohemorrhagic E. coli (EHEC) | Strongly positive samples | 0 | 2 |
| Weakly positive samples | 2 | 2 | |
| Negative samples | 0 | 0 | |
| FTD/Salmonella spp. | Strongly positive samples | 0 | 0 |
| Weakly positive samples | 1 | 3 | |
| Negative samples | 0 | 0 | |
| FTD/Shigella spp./enteroinvasive E. coli (EIEC) | Strongly positive samples | 0 | 1 |
| Weakly positive samples | 1 | 2 | |
| Negative samples | 0 | 0 | |
| FTD/Yersinia enterocolitica | Strongly positive samples | 1 | 1 |
| Weakly positive samples | 1 | 0 | |
| Negative samples | 0 | 0 | |
| ampliCube/Campylobacter spp. | Strongly positive samples | 1 | 1 |
| Weakly positive samples | 0 | 2 | |
| Negative samples | 0 | 0 | |
| ampliCube/Salmonella spp. | Strongly positive samples | 2 | 2 |
| Weakly positive samples | 1 | 2 | |
| Negative samples | 0 | 0 | |
| ampliCube/Yersinia enterocolitica | Strongly positive samples | 1 | 0 |
| Weakly positive samples | 0 | 3 | |
| Negative samples | 0 | 0 | |
| ampliCube/enterohemorrhagic E. coli (EHEC) | Strongly positive samples | 0 | 1 |
| Weakly positive samples | 2 | 0 | |
| Negative samples | 0 | 0 | |
| ampliCube/enterotoxigenic E. coli (ETEC) | Strongly positive samples | 1 | 1 |
| Weakly positive samples | 4 | 2 | |
| Negative samples | 0 | 0 | |
| ampliCube/Shigella spp./enteroinvasive E. coli (EIEC) | Strongly positive samples | 0 | 1 |
| Weakly positive samples | 1 | 2 | |
| Negative samples | 0 | 0 |
Results of the latent class analysis-based test comparison
As negative results in the gold standard PCRs, however, do not necessarily exclude the abundance of target DNA, the test results were subjected to a LCA-based test comparison approach without a gold standard in a next step for the parameters salmonellae, shigellae/EIEC, campylobacter, yersiniae, ETEC, and EHEC, respectively. Concerning C. difficile, such an approach was unfeasible, because this parameter was included in the FTD assay only. As shown in Table 3, calculated sensitivities were slightly higher, ranging from 81.2 to 100% with calculated specificities ranging from 58.5 to 100%.
Sensitivity and specificity as calculated by latent class analysis, mean as well as median cycle threshold values and agreement kappa. N = number. Ct = cycle threshold. SD = standard deviation. CI = confidence interval
| PCR with target organism | N | Positives n (%) | Sensitivity (0.95 CI) | Specificity (0.95 CI) | Positives Ct mean (SD), median of the positives | Cohen's kappa (0.95 CI) |
| Salmonella in-house | 341 | 79 | 0.945 | 0.988 | 22.36 | 0.839 |
| (23) | (0.861, 0.979) | (0.963, 0.996) | (5.09) | (0.771, 0.883) | ||
| 22 | ||||||
| Salmonella FTD | 341 | 93 | 0.985 | 0.947 | 22.24 | |
| (27) | (0.903, 0.998) | (0.911, 0.968) | (7.20) | |||
| 22 | ||||||
| Salmonella ampliCube | 341 | 74 | 0.908 | 0.996 | 21.00 | |
| (22) | (0.817, 0.956) | (0.966, 0.999) | (6.52) | |||
| 20 | ||||||
| Yersinia in-house | 341 | 12 | 1 | 0.997 | 21.83 | 0.888 |
| (4) | (0, 1) | (0.979, 0.999) | (3.43) | (0.744, 0.959) | ||
| 22 | ||||||
| Yersinia FTD | 341 | 14 | 1 | 0.990 | 23.78 (5.91) | |
| (4) | (0, 1) | (0.972, 0.997) | 22.50 | |||
| Yersinia ampliCube | 341 | 11 | 1 | 1 | 20.27 | |
| (3) | (0, 1) | (n.e.) | (3.28) | |||
| 20 | ||||||
| Campylobacter in-house | 341 | 99 | 0.832 | 1 | 21.75 | 0.542 |
| (29) | (0.753, 0.889) | (0, 1) | (4.76) | (0.468, 0.623) | ||
| 21 | ||||||
| Campylobacter FTD | 341 | 119 | 1 | 1 | 20.78 | |
| (35) | (0, 1) | (0, 1) | (5.82) | |||
| 19 | ||||||
| Campylobacter ampliCube | 341 | 209 | 0.983 | 0.585 | 23.38 | |
| (61) | (0.935, 0.995) | (0.519, 0.648) | (6.62) | |||
| 23 | ||||||
| Shigella/EIEC in-house | 341 | 20 | 0.902 | 1 | 21.40 | 0.803 |
| (6) | (0.673, 0.976) | (n.e.) | (7.23) | (0.669, 0.890) | ||
| 19 | ||||||
| Shigella/EIEC FTD | 341 | 27 | 1 | 0.985 | 22.25 | |
| (8) | (0, 1) | (0.962, 0.993) | (8.15) | |||
| 21 | ||||||
| Shigella/EIEC ampliCube | 341 | 24 | 0.899 | 0.987 | 21.83 | |
| (7) | (0.676, 0.974) | (0.966, 0.995) | (8.56) | |||
| 19 | ||||||
| ETEC in-house | 341 | 29 | 0.949 | 0.987 | 25.96 | 0.828 |
| (9) | (n.e.) | (0, 1) | (6.14) | (0.718, 0.938) | ||
| 25 | ||||||
| ETEC ampliCube | 341 | 28 | 0.956 | 0.991 | 18.82 | |
| (8) | (0, 1) | (0.963, 0.997) | (4.87) | |||
| 17 | ||||||
| EHEC FTD | 341 | 36 | 1 | 0.995 | 28.63 | 0.862 |
| (11) | (0, 1) | (n.e.) | (3.44) | (0.769, 0.956) | ||
| 30 | ||||||
| EHEC ampliCube | 341 | 28 | 0.812 | 1 | 30.46 | |
| (8) | (0.630, 0.916) | (0, 1) | (3.78) | |||
| 31 |
Focussing on the FTD platform, calculated sensitivity of 100% was observed for all assays with the exception of the Salmonella spp.-specific assay (98.5%) and C. difficile-specific assay, for which this assessment was unfeasible. Calculated specificity was 100% for C. coli/jejuni/lari and >90% for all other parameters.
Focussing on the ampliCube platform, sensitivity of 100% was calculated for the Y. enterocolitica-specific assay, sensitivity >90% for the assays targeting Salmonella spp., Campylobacter spp., and ETEC, sensitivity >80% for the assays targeting EHEC and Shigella spp./EIEC. Calculated specificity was 100% for the assays targeting EHEC and Y. enterocolitica, >90% for the assays targeting ETEC, Shigella spp./EIEC and Salmonella spp., and >50% for the Campylobacter spp.-specific assay.
In line with this latter result, almost perfect agreement as defined by Landis and Koch [12] (Cohen’s kappa 0.81–1.00) could be shown for all assessed parameters with the exception of campylobacter, for which only moderate agreement (0.41–0.60 according to [12]) was recorded. Details including mean Ct-values are shown in Table 3.
Discussion
As shown in this study, similar performances were observed for the commercial assays and the in-house approaches, making the commercial platforms suitable as likely replacements in response to Regulation (EU) 2017/746. Thereby, the overall performance of the FTD platform was slightly more promising than the performance of the ampliCube kits, but the differences were marginal.
The reduced sensitivity of the C. difficile assay was the major weakness of the FTD platform, while the low calculated specificity and the thus difficult interpretability of positive results of the Campylobacter spp. assay was the major weakness of the ampliCube platform. However, the single outlier with only 58.5% calculated specificity for the ampliCube Campylobacter spp. PCR most likely resulted from the fact that the ampliCube assay was genus-specific while species-specific sequences were the PCR targets of the chosen competitor assays. For the other PCR targets of the ampliCube platform, quite acceptable specificities ranging from 94.7 to 100% were observed. For both assessed commercial platforms, the observed phenomena should be considered when their diagnostic results are interpreted in the clinical setting.
While PCR was formerly primarily used for the detection of difficult to grow or culturally poorly discriminable pathogens like C. difficile [13–25], Camplycobacter spp. [26–29], or diarrheagenic E. coli [4, 30, 31], diagnostic real-time PCR as an initial screening tool for pathogen diagnostics from stool samples is increasingly used with broad detection panels [1, 32–36]. The here presented study is just a piece in the puzzle of the ongoing assessments of performance characteristics of those molecular tools and is meant to help estimating their usefulness but also the limits of their usefulness for the diagnostic routine.
Nevertheless, the study has a number of limitations. First of all, the study included a limited number of compared test assays adapted to the PCR platforms as available in our laboratory, while other and also broader syndromic PCR panels for stool samples are available as described [1, 32–36]. Secondly, only limited numbers of sample materials were available, limiting the interpretability of the results. Thirdly, due to the preselection of samples for the assessments, the methodological prerequisites for the application of LCA testing [10, 11] cannot be considered as completely fulfilled, so the results of those calculations have to be regarded as approximations only. In spite of those limitations, the data provided by the assessment can help to assess the performance characteristics of the compared assays in a diagnostic real-life setting.
Conclusion
As demonstrated by the test comparison, both commercial diagnostic approaches are suitable for a replacement of the previously applied in-house assay.
Funding
No financial support was received for this study.
Authors’ contribution
KT, AH, and HF jointly planned the study. KT performed the experiments and assessed the data, AH performed the calculations, and HF prepared the manuscript. All authors have jointly optimized and reviewed the manuscript.
Conflict of interest
Nothing to declare.
Acknowledgements
Annett Michel is gratefully acknowledged for excellent technical assistance.
References
- 1.↑
Antikainen J, Kantele A, Pakkanen SH, Lääveri T, Riutta J, Vaara M, et al.. A quantitative polymerase chain reaction assay for rapid detection of 9 pathogens directly from stools of travelers with diarrhea. Clin Gastroenterol Hepatol 2013;11:1300–7.e3.
- 2.↑
Frickmann H, Warnke P, Frey C, Schmidt S, Janke C, Erkens K, et al.. Surveillance of food- and smear-transmitted pathogens in European soldiers with diarrhea on deployment in the tropics: experience from the European Union Training Mission (EUTM) Mali. Biomed Res Int 2015;2015:573904.
- 3.↑
Wiemer D, Loderstaedt U, von Wulffen H, Priesnitz S, Fischer M, Tannich E, et al.. Real-time multiplex PCR for simultaneous detection of Campylobacter jejuni, Salmonella, Shigella and Yersinia species in fecal samples. Int J Med Microbiol 2011;301:577–84.
- 4.↑
Niesters HGM. Quantitation of viral load using real-time amplification techniques. Methods 2001;25:419–29.
- 5.
Schawaller M, Wiemer D, Hagen RM, Frickmann H. Infectious diseases in German military personnel after predominantly tropical deployments: a retrospective assessment over 13 years. BMJ Mil Health 2020; [Epub ahead of print], https://doi.org/10.1136/bmjmilitary-2020-001575.
- 6.
Halfter M, Müseler U, Hagen RM, Frickmann H. Enteric pathogens in German police officers after predominantly tropical deployments – a retrospective assessment over 5 years. Eur J Microbiol Immunol (Bp) 2020;10:172–7.
- 7.↑
Maaßen W, Wiemer D, Frey C, Kreuzberg C, Tannich E, Hinz R, et al.. Microbiological screenings for infection control in unaccompanied minor refugees: the German Armed Forces Medical Service’s experience. Mil Med Res 2017;4:13.
- 8.↑
Hahn A, Luetgehetmann M, Landt O, Schwarz NG, Frickmann H. Comparison of one commercial and two in-house TaqMan multiplex real-time PCR assays for detection of enteropathogenic, enterotoxigenic and enteroaggregative Escherichia coli. Trop Med Int Health 2017;22:1371–6.
- 9.↑
Rabenau HF, Kessler HH, Kortenbusch M, Steinhorst A, Raggam RB, Berger A. Verification and validation of diagnostic laboratory tests in clinical virology. J Clin Virol 2007;40:93–8.
- 10.
Qu Y, Tan M, Kutner M. Random effects models in latent class analysis for evaluating accuracy of diagnostic test. Biometrics 1996;52:797–810.
- 11.
Hahn A, Podbielski A, Meyer T, Zautner AE, Loderstädt U, Schwarz NG, et al.. On detection thresholds-a review on diagnostic approaches in the infectious disease laboratory and the interpretation of their results. Acta Trop 2020;205:105377.
- 12.↑
Landis JR, Koch GG. The measurement of observer agreement of categorical data. Biometrics 1977;33:159–74.
- 13.
Collier MC, Stock F, DeGirolami PC, Samore MH, Cartwright CP. Comparison of PCR-based approaches to molecular epidemiologic analysis of Clostridium difficile. J Clin Microbiol 1996;34:1153–7.
- 14.
Stamper PD, Alcabasa R, Aird D, Babiker W, Wehrlin J, Ikpeama I, et al.. Comparison of a commercial real-time PCR assay for tcdB detection to a cell culture cytotoxicity assay and toxigenic culture for direct detection of toxin-producing Clostridium difficile in clinical samples. J Clin Microbiol 2009;47:373–8.
- 15.
Kuijper EJ, van den Berg RJ, Brazier JS. Comparison of molecular typing methods applied to Clostridium difficile. Methods Mol Biol 2009;551:159–71.
- 16.
Eastwood K, Else P, Charlett A, Wilcox M. Comparison of nine commercially available Clostridium difficile toxin detection assays, a real-time PCR assay for C. difficile tcdB, and a glutamate dehydrogenase detection assay to cytotoxin testing and cytotoxigenic culture methods. J Clin Microbiol 2009;47:3211–7.
- 17.
Huang H, Weintraub A, Fang H, Nord CE. Comparison of a commercial multiplex real-time PCR to the cell cytotoxicity neutralization assay for diagnosis of Clostridium difficile infections. J Clin Microbiol 2009;47:3729–31.
- 18.
Kvach EJ, Ferguson D, Riska PF, Landry ML. Comparison of BD GeneOhm Cdiff real-time PCR assay with a two-step algorithm and a toxin A/B enzyme-linked immunosorbent assay for diagnosis of toxigenic Clostridium difficile infection. J Clin Microbiol 2010;48:109–14.
- 19.
Larson AM, Fung AM, Fang FC. Evaluation of tcdB real-time PCR in a three-step diagnostic algorithm for detection of toxigenic Clostridium difficile. J Clin Microbiol 2010;48:124–30.
- 20.
Crobach MJ, Dekkers OM, Wilcox MH, Kuijper EJ. European Society of Clinical Microbiology and Infectious Diseases (ESCMID): data review and recommendations for diagnosing Clostridium difficile-infection (CDI). Clin Microbiol Infect 2009;15:1053–66.
- 21.
Chapin KC, Dickenson RA, Wu F, Andrea SB. Comparison of five assays for detection of Clostridium difficile toxin. J Mol Diagn 2011;13:395–400.
- 22.
Davies KA, Berry CE, Morris KA, Smith R, Young S, Davis TE, et al.. Comparison of the Vidas C. difficile GDH Automated Enzyme-Linked Fluorescence Immunoassay (ELFA) with Another Commercial Enzyme Immunoassay (EIA) (Quik Chek-60), Two Selective Media, and a PCR Assay for gluD for Detection of Clostridium difficile in Fecal Samples. J Clin Microbiol 2015;53:1931–4.
- 23.
Sandlund J, Mills R, Griego-Fullbright C, Wagner A, Estis J, Bartolome A, et al.. Laboratory comparison between cell cytotoxicity neutralization assay and ultrasensitive single molecule counting technology for detection of Clostridioides difficile toxins A and B, PCR, enzyme immunoassays, and multistep algorithms. Diagn Microbiol Infect Dis 2019;95:20–4.
- 24.
Kouhsari E, Douraghi M, Barati M, Yaseri HF, Talebi M, Abbasian S, et al.. Rapid simultaneous molecular stool-based detection of toxigenic Clostridioides difficile by quantitative TaqMan Real-Time PCR assay. Clin Lab 2019;65:4.
- 25.
Sloan LM, Duresko BJ, Gustafson DR, Rosenblatt JE. Comparison of real-time PCR for detection of the tcdC gene with four toxin immunoassays and culture in diagnosis of Clostridium difficile infection. J Clin Microbiol 2008;46:1996–2001.
- 26.
Lawson AJ, Shafi MS, Pathak K, Stanley J. Detection of campylobacter in gastroenteritis: comparison of direct PCR assay of faecal samples with selective culture. Epidemiol Infect 1998;121:547–53.
- 27.
Al Amri A, Senok AC, Ismaeel AY, Al-Mahmeed AE, Botta GA. Multiplex PCR for direct identification of Campylobacter spp. in human and chicken stools. J Med Microbiol 2007;56(Pt 10):1350–5.
- 28.
Shiramaru S, Asakura M, Inoue H, Nagita A, Matsuhisa A, Yamasaki S. A cytolethal distending toxin gene-based multiplex PCR assay for detection of Campylobacter spp. in stool specimens and comparison with culture method. J Vet Med Sci 2012;74:857–62.
- 29.
Kabir SML, Chowdhury N, Asakura M, Shiramaru S, Kikuchi K, Hinenoya A, et al.. Comparison of Established PCR Assays for Accurate Identification of Campylobacter jejuni and Campylobacter coli. Jpn J Infect Dis 2019;72:81–7.
- 30.
Dutta S, Chatterjee A, Dutta P, Rajendran K, Roy S, Pramanik KC, et al.. Sensitivity and performance characteristics of a direct PCR with stool samples in comparison to conventional techniques for diagnosis of Shigella and enteroinvasive Escherichia coli infection in children with acute diarrhoea in Calcutta, India. J Med Microbiol 2001;50:667–74.
- 31.
Grys TE, Sloan LM, Rosenblatt JE, Patel R. Rapid and sensitive detection of Shiga toxin-producing Escherichia coli from nonenriched stool specimens by real-time PCR in comparison to enzyme immunoassay and culture. J Clin Microbiol 2009;47:2008–12.
- 32.
Kabayiza JC, Andersson ME, Welinder-Olsson C, Bergström T, Muhirwa G, Lindh M. Comparison of rectal swabs and faeces for real-time PCR detection of enteric agents in Rwandan children with gastroenteritis. BMC Infect Dis 2013;13:447.
- 33.
Bruijnesteijn van Coppenraet LE, Dullaert-de Boer M, Ruijs GJ, van der Reijden WA, van der Zanden AG, Weel JF, et al.. Case-control comparison of bacterial and protozoan microorganisms associated with gastroenteritis: application of molecular detection. Clin Microbiol Infect 2015;21:592.e9–19.
- 34.
Kellner T, Parsons B, Chui L, Berenger BM, Xie J, Burnham CA, et al.. Comparative evaluation of enteric bacterial culture and a molecular multiplex syndromic panel in children with acute gastroenteritis. J Clin Microbiol 2019;57:e00205–19.
- 35.
Amrud K, Slinger R, Sant N, Desjardins M, Toye B. A comparison of the Allplex™ bacterial and viral assays to conventional methods for detection of gastroenteritis agents. BMC Res Notes 2018;11:514.
- 36.
Deng J, Luo X, Wang R, Jiang L, Ding X, Hao W, et al.. A comparison of Luminex xTAG® Gastrointestinal Pathogen Panel (xTAG GPP) and routine tests for the detection of enteropathogens circulating in Southern China. Diagn Microbiol Infect Dis 2015;83:325–30.
