Abstract
During the coronavirus disease 2019 (COVID-19) pandemic, artificial intelligence (AI) based software on chest computed tomography (CT) imaging has proven to have a valuable role in accelerating diagnosis and screening. The proposed AI-based tools proved to be rapid and reproducible techniques to guide patient management and treatment protocols. Although no specific guidelines exist, CT-imaging and clinical features are used for patient staging. To shed light on the role of AI techniques that have been developed in fighting COVID-19, in this review, studies investigating the usage of commonly used AI models on chest CT imaging for disease quantification and prognostication are collected.
Introduction
The World Health Organization (WHO) announced the novel severe acute respiratory syndrome coronavirus 2 (SARs-CoV-2) causing COVID-19, as a global outbreak on 30th January 2020, after its first reported case on 31st December 2019 in Wuhan, China. Having impacted the global healthcare system dramatically, COVID-19 was recognized as a pandemic on 11th March 2020 [1]. A total of 657,977,736 confirmed cases of COVID-19 have been registered up to 6th January 2023, resulting in a fatality rate of 10.15% with approximately 6,681,433 deaths [2].
The correct usage of clinical biomarkers and image interpretation to diagnose and stage the disease is essential for managing the pandemic and optimizing patient management and treatment. Being able to diagnose and quantify the disease in its early stages can help reduce the number of fatalities [3, 4]. The gold standard for COVID-19 detection is Reverse Transcription-Polymerase Chain Reaction (RT-PCR). However, the exponential increase in the number of infections impacted the global healthcare system making traditional diagnostic methods ineffective [5]. Even though RT-PCR results in high specificity, factors such as sample preparation and quality control [6] cause the assay to have a sensitivity as low as 60–70% [7].
On the other hand, the identification of COVID-19 pneumonia infection using chest computed tomography (CT) has been reported to have sensitivities as high as 98% [7]. Moreover, clinical parameters and CT features are correlated with the severity of COVID-19 infection requiring intensive care unit (ICU) management [8, 9]. In an attempt to solve this issue, researchers and medical experts turned to artificial intelligence (AI) to provide a helping hand in managing the numerous infections by improving the performance of traditional techniques. Of patients hospitalized with COVID-19, 14–30% required ICU admission and 20–33% died [10]. The ability to quantify and stage patients during hospital admissions is detrimental to patient management and prognosis, and it is important for optimizing limited resources in the ICU [10].
Based on Chest CT images, the COVID-19 infection has been reported to have 4 stages [11]. The early stage occurs after the onset of symptoms on days 0–4, where ground glass opacity (GGO) can be seen unilaterally or bilaterally in the subpleural regions of the lower lobes. When GGO becomes diffuse, with bilateral crazy-paving patterns and multi-lobular consolidation, the infection is in the progression stage on days 5–8. The peak stage on days 9–13 is reached with the dense consolidation prevailing until around day 14 when the absorption stage begins with only some GGO remaining. Other abnormal findings are septal thickening, halo signs, bronchiectasis, broncho-vascular thickening, lymphadenopathy, and pleural/pericardial effusions. Chest CT scans proved to have high sensitivity in the detection, grading, and determination of the radiological patterns of disease progression [12].
In order to predict the level of suspicion for COVID-19 infection based on CT findings, the Dutch Radiological Society developed the COVID-19 Reporting and Data System (CO-RADS) which has been adopted to classify lung involvement. The CO-RADS scoring system, has a scale from 1 to 6, with 1 indicating a normal, healthy image, 5 denoting a typical COVID-19 presentation, and 6 referring to patients who tested positive for COVID-19 with RT-PCR [13].
Radiologists have also implemented various semi-quantitative methods to determine disease extent from CT images. For example, the Percentage Opacity (PO) is a method used to assess lung volume involvement [14], while the CT severity score (CTSS) has been proposed to evaluate the severity of each individual lobe (0: no involvement, 1: <5% involvement, 2: 5–25% involvement, 3: 26–49% involvement, 4: 50–75% involvement, 5: >75% involvement), for a total severity sum of all lobes ranging from 0 (no involvement) to 25 (maximum involvement) [15].
Experienced pulmonologists who simply used a CT scoring method were able to correlate higher scores with increased mortality [16] and COVID-19 severity with the visual quantitative assessment of CT images [4]. Lieveld et al. [17] validated the CO-RADS and also found a significant association between CTSS and hospital admission, ICU admission, and 30-day mortality in COVID-19 patients, illustrating its value in patient management. However, this semi-quantitative visual assessment is subjective and time-consuming, resulting in a lack of inter-observer reproducibility [18]. This exemplifies the need and usefulness of tools such as automatic disease quantification.
The Clinical Progression Scale of the World Health Organization (WHO) is also used to grade clinical severity. As summarized in Table 1, disease severity measurement is provided by a scale ranging from 0 (not infected) to 10 (death) [19]. The risk of COVID-19 pneumonia progressing to acute respiratory distress syndrome (ARDS) is high. These patients have an increased mortality risk, as the ratio of well-aerated lung volume and total lung capacity of these patients is reported to be less than 40% [20].
The WHO Clinical Progression Scale classifies patients into four categories based on their outcome, with scores ranging from 0 (not infected) to 10 (death) [19]. Abbreviations: WHO = World Health Organization. ECMO = extracorporeal membrane oxygenation. FiO2 = fraction of inspired oxygen. NIV = non-invasive ventilation. pO2 = partial pressure of oxygen. SpO2 = oxygen saturation
| Patient State | Descriptor | Score |
| Uninfected | Uninfected; no viral RNA detected | 0 |
Ambulatory  mild disease | Asymptomatic; viral RNA detected | 1 |
| Symptomatic; independent | 2 | |
| Symptomatic; assistance needed | 3 | |
Hospitalized:  moderate disease | Hospitalized; no oxygen therapy | 4 |
| Hospitalized; oxygen by mask or nasal prongs | 5 | |
Hospitalized:  severe diseases | Hospitalized; oxygen by NIV or high flow | 6 |
| Intubation and mechanical ventilation, pO2/FiO2 ≥ 150 or SpO2/FiO2 ≥200 | 7 | |
| Mechanical ventilation pO2/FiO2 <150 (SpO2/FiO2 <200) or vasopressors | 8 | |
| Mechanical ventilation pO2/FiO2<150 and vasopressors, dialysis, or ECMO | 9 | |
| Dead | Dead | 10 |
Quantifying the degree of lung involvement could provide a more objective assessment of the extent of disease burden [21], by objectively identifying important features present in COVID-19 pneumonia (number, size, density, or extent of the lesions) that are not always identifiable by radiologists [22]. With the commencement of the pandemic, the use of AI techniques such as machine learning (ML) and deep learning (DL) has exponentially increased and evolved in an attempt to find an alternative to RT-PCR to help reduce the impact of COVID-19 [23]. Generally speaking, the ability to distinguish healthy lung parenchyma from various types of pathological lung tissues is the initial goal when developing AI models. Clinically speaking, the use of imaging techniques such as chest X-rays and thoracic CTs, has proven to be of great assistance to medical experts. More specifically, the contribution of AI techniques emerging with the use of medical imaging has been significant for COVID-19 [24]. Lessmann et al. [13, 25] developed an AI-based automatic score to define the likelihood and extent of COVID-19 on chest CTs (CORADS-AI) that was successfully validated using CO-RADS and CTSS scores.
To assess and quantify COVID-19 pneumonia, X-rays or CT images are processed in a step called segmentation. This refers to the delineation of the regions of interest (ROIs), including lung, lobes, bronchopulmonary segments, and any regions that are infected. Since the manual slice-by-slice segmentation of the structures in 3D scans can be extremely time-consuming, DL-based approaches are being used more and more widely to accurately segment ROIs on CT images. Chest X-rays, on the other hand, produce an obfuscated 2D image contrast due to the ribs projecting onto the lung parenchyma, making lung segmentation more challenging. Although X-rays are an easily accessible image modality in hospitals, they range from 30 to 70% sensitivity in detecting COVID-19 pneumonia. In addition in the early or mid-stage of the disease, images appear normal, with 69% showing abnormal findings when first admitted, and 80% at some point during their hospitalization [26].
ML algorithms after training, allow computers to independently execute actions. The detection and quantification of COVID-19 is achievable by extracting unique features from medical images. Although different scoring systems exist for the automated diagnosis and quantification of COVID-19 pneumonia, each accomplishes the same results. It is beneficial to aid physicians in the prediction of clinical outcomes of COVID-19 patients by triaging patients as soon as possible [27, 28]. This review focuses on the usage of common AI quantification methods with the direct selection and extraction of image characteristics from CT scans to determine the prognosis of COVID-19 patients.
Artificial intelligence: machine learning and deep learning explained
The goal of AI is to allow computers to imitate human behavior. ML, which is a subset of AI, is a collective term for algorithms that are trained to process and learn from a given set of data. These algorithms are then able to make decisions and draw conclusions based on patterns hidden in the data that are frequently challenging for humans to see with the unaided eye. Supervised ML algorithms are trained on input data from the training dataset to learn pattern recognition and build a prediction model. They are then applied to a test dataset to evaluate the prediction accuracy. DL is a subset of ML, which refers to artificial neural networks that mimic the human brain structure. These DL algorithms have a complex structure – comprising an input layer, several hidden layers, and an output layer – that require more computational power to train and run, and larger training datasets to achieve adequate prediction accuracy [29]. A brief categorization of AI can be observed in Fig. 1.
The skeleton of artificial intelligence (AI) can be subcategorized to include ML and DL. Abbreviations: AI = artificial intelligence. ML = machine learning. DL = deep learning
Citation: Imaging 16, 1; 10.1556/1647.2024.00167
ML algorithms can be divided into supervised, unsupervised, and reinforcement learning algorithms. Supervised learning allows the algorithms to learn from labeled training datasets. Based on the style of data processing and the nature of outcome variables, these models can be further categorized as regression and classification models [29].
During ML model construction for a regression problem, in order to calculate a continuous output variable, regression models use continuous numerical values to define the relationship between the features (or independent predictors) and the outcome (or dependent variable). Thus, a probability numerical value is predicted when evaluating the cases in the test set. This category includes simple, clear-cut models like linear, logistic, or polynomial regression models as well as more complex, less transparent models, such as decision tree regression (DTR), random forest regression (RFR), support vector regression (SVR) models, and neural networks [30].
In contrast, the output of classification models consists of discrete values, thus the test cases are assigned into specific categories. This includes a logistic regression classifier, support vector machine classifier (SVC) [31], decision tree classifier (DTC), random forest classifier (RFC), and neural networks. SVC and RFC have shown the most success in the diagnosis of COVID-19 [32]. The SVC model defines a hyperplane in an n-dimensional space constructed by the observed features, that can distinctly classify the data points into given output categories [32]. RFC models use decision trees to generate tree-structured classifiers from random vectors, to perform classifications in COVID-19. The correlation and the strength of the tree classifiers, along with the addition of trees during training, limit the generalization error value and allow for the minimization of the risk of overfitting [33].
The unsupervised learning approaches of ML are used to draw inferences and find patterns from input data without references to the labeled outcomes. These algorithms gather unlabeled data and group them together without guidance in order to discover hidden patterns in the data [34]. These algorithms can be used not just for clustering the data by revealing associations between the cases based on their features, but also for anomaly detection and dimensionality reduction. Unsupervised models are therefore often used prior to, and in combination with, supervised approaches.
Neural networks are some of the most used DL approaches in investigating COVID-19. They are multilayered pathways, which imitate the human mind as a human brain-like structure. Based on networks of mathematical equations, they are trained to recognize patterns from input data and predict the output in new similar datasets [35]. They consist of 3 main components: an input layer, one or more hidden layers, and an output layer. The input serves as an activation function acting as a characteristic match feature. Each layer has neurons (nodes), containing a function where each input passes through to ultimately lead to the output [29]. DL is one of the latest advances in artificial intelligence, referring to complex “deep” artificial neural network algorithms (DNN) with multiple hidden layers that aim to automatically extract meaningful features from images without human intervention. Each neural network architecture varies depending on the number of nodes it has per layer and the number of hidden layers. Each connection between the nodes is associated with a so-called weight, determining the importance of the input value. The heavier-weighted nodes have more significance when they reach the output layer.
The use of DNN has been the most common technique to identify and segment lung lesions in patients with COVID-19. These DNN models can be further categorized into classification models and segmentation models based on the task for which they were trained. The classification models compare healthy lungs with features extracted from affected regions of COVID-19-infected lungs to make a decision if they are diseased or healthy [36–39]. Segmentation models are usually based on convolutional neural networks (CNN), which detect abnormal patterns and directly select the affected regions of the lung [22, 40–42]. It is able to manage immense amounts of data by layering information and convoluting it in a linear fashion between matrices. CNN has high activity for image recognition patterns, automatically detecting patterns without human supervision, however, they are not able to encode the position of an object and require a huge amount of training data to work efficiently [43]. Regarding network architectures, U-Net has dominated image segmentation in the literature. Multi-task DL networks, functioning by extracting image features for classification and lesion segmentation separately and then combining them, have been shown to yield accurate results in a study done by Wu et al. [44]. A summary of the findings of the studies presented in this review can be seen in Table 2.
Study findings summary. Abbreviations: AI = artificial intelligence, AUC = area under the curve, CNN = Convolutional Neural Network, CT = computed tomography, DL = deep learning, DNN = Deep Neural Network, DSC = dice similarity coefficient, GGO = ground glass opacity, HITL = human-in-the-loop, OR: odds ratio, YACTA = Yet Another CT-Analyzer
| Study | AI approach | Code if publicly available | Endpoint | Limitation | Patient number (with CT) | Accuracy and AUC | Sensitivity | Specificity | DSC |
| Commercially available (software or module) | |||||||||
| Huang et al. [21] | InferReadTM CT Lung (COVID-19) (Infervision, Europe GmbH, Wiesbaden, Germany) | N/A | Identifying clinical parameter and quantitative CT parameter that are different between mild, moderate, severe, and critical type cases. | Differences were evaluated but no model building was completed, the classification performance was not assessed. | 126 | N/A | N/A | N/A | N/A |
| Chabi et al. [45] | Syngo.via software CT Pneumonia Analysis (Siemens Healthineers, Forchheim, Germany) | N/A | Identifying clinical parameter and quantitative CT parameter that can predict clinical deterioration or death. | The accuracy of the models is not reported. In case of the combined model, only the AUC value is included. | 323 | (1) lung volume occupied by opacities: AUC: 0.70 (2) consolidation: AUC: 0.68 | (1) lung volume occupied by opacities: 81% (2) consolidation: 46% | (1) lung volume occupied by opacities: 54% (2) consolidation: 87% | N/A |
| Stasiow et al. [46] | Syngo.via software CT Pneumonia Analysis (Siemens Healthineers, Forchheim, Germany) | N/A | Founding independent predictors of (1) hospitalization in the ICU, (2) need for artificial ventilation, and (3) death. | The AUC, accuracy, sensitivity, and specificity of the logistic regression model are not reported, only the OR values. The models were not tested on independent test cases. | 128 | N/A | N/A | N/A | N/A |
| Sezer et al. [47] | Syngo.via software CT Pneumonia Analysis (Siemens Healthineers, Forchheim, Germany) | N/A | Assessing the performance of lung opacity score in (1) differentiating patients with good vs. bad clinical course, and (2) predicting mortality. | The accuracies are not reported. The added diagnostic value of quantitative CT features to clinical variables is not investigated. The models were not tested on independent test cases. | 96 | (1) clinical course: AUC: 0.72 (2) mortality: AUC: 0.76 | (1) clinical course: 63.6% (2) mortality: 67.65% | (1) clinical course: 75.44% (2) mortality: 74.19% | N/A |
| Gashi et al. [48] | Syngo.via software CT Pneumonia Analysis (Siemens Healthineers, Forchheim, Germany) | N/A | Building a random forest prediction model from quantitative CT parameters to differentiate symptomatic COVID-19 patients from asymptomatic group. | The added diagnostic value of quantitative CT features to clinical variables is not investigated. The models were not tested on independent test cases. | 196 | AUC: 0.95 accuracy: 94% | 97% | 90% | N/A |
| Okuma et al. [49] | Syngo.via software CT Pneumonia Analysis (Siemens Healthineers, Forchheim, Germany) | N/A | Testing the prediction performance of quantitative CT parameters and laboratory test results in differentiating between (1) mild vs. moderate, and (2) moderate vs. severe cases | Only the individual CT parameters', and blood test values' prediction performance is reported. No model was built from the combination of these parameters. Only the AUC, sensitivity, and specificity values are reported, but accuracy is not included. The models were not tested on independent test cases. | 100 | (1) mild vs. moderate: mean density AUC: 0.75 (2) moderate vs. severe: mean density AUC: 0.71 | (1) mild vs. moderate: 82% (2) moderate vs. severe: 59% | (1) mild vs. moderate: 65% AUC: 0.75 (2) moderate vs. severe: 86% | N/A |
| Mergen et al. [50] | Syngo.via software CT Pneumonia Analysis (Siemens Healthineers, Forchheim, Germany) | N/A | Investigating the difference in quantitative CT parameters between lung lobes and contrast-enhanced vs. non-contrast enhanced scans. Assessing the correlation between quantitative CT features and clinical parameters. | The diagnostic/predictive performance of parameters was not investigated. | 60 | N/A | N/A | N/A | N/A |
| Szabó et al. [52] | CAD4COVID (Thirona, Nijmegen, Netherlands) | N/A | Predicting later clinical deterioration based on clinical parameters and baseline CT scans performed at the time of hospital admission. | Only the AUC values are reported. The accuracy, sensitivity and specificity values are not included. The statistical significance of adding the deep learning-based score to the clinical parameters is not investigated. The models were not tested on independent test cases. | 326 | deep learning-based severity score: AUC: 0.71 clinical model: AUC: 0.77 combined model: AUC: 0.81 | N/A | N/A | N/A |
| Grodecki et al. [65] | LungQuant v.1.0, (Cedars-Sinai Medical Center, Los Angeles, CA, USA) | N/A | Building a multiple logistic regression model from clinical and quantitative CT parameters to predict deterioration/death. | Only AUC and OR values are reported, the accuracy, sensitivity, specificity, PPV and NPV values of the proposed logistic regression model is not included. The models were not tested on independent test cases. | 93 | - AUC all opacities: 0.93 | N/A | N/A | N/A |
| Pan et al. [84] | YITU CT software COVID-Lesion Net module (YITU healthcare Technology Co. Ltd.) | N/A | Assessing the correlation between conventional CT scoring and AI-based severity quantification, comparing these parameters between patient groups at different time points, and assessing the dynamic changes of these parameters. | 465 CT scans from 95 patients were included as independent samples. The predictive performance of the deep learning-based algorithm and its superiority in risk assessment compared to the conventional CT scoring were not investigated. | 95 | N/A | N/A | N/A | N/A |
| Chrzan et al. [91] | YITU CT software (YITU healthcare Technology Co. Ltd.) | N/A | Differentiating between COVID-19 pneumonia, bronchopneumonia and atypical pneumonia. | Only the difference in quantitative lung involvement parameters between the three patient groups was evaluated. The diagnostic performance of these parameters was not investigated. | 150 | N/A | N/A | N/A | N/A |
| Chrzan et al. [92] | YITU CT software (YITU healthcare Technology Co. Ltd.) | N/A | Identifying clinical and radiological predictors of in-hospital death. | The predictive performance of absolute and percentage consolidation volume, ground glass volume, and inflammation volume were assessed, however their additional value to clinical parameters/clinical model were not investigated. AUC, sensitivity and specificity values are reported, but accuracy is not included. The models were not tested on independent test cases. | 804 | AUC: 0.69 | 0.51 | 0.78 | 82.08% |
| Pang et al. [93] | YITU CT software (YITU healthcare Technology Co. Ltd.) | N/A | Identifying radiological parameters on admission that can be used to predict patient outcomes critical vs. non-critical. | The predictive performance of pneumonia volume, ground-glass opacity volume, and consolidation volume was investigated, however their additional value to clinical parameters/clinical model were not assessed. AUC, sensitivity and specificity values are reported, but the accuracy is not included. The models were not tested on independent test cases. | 140 | AUC: 0.87 | 81.3% | 80.6% | N/A |
| Ding et al. [94] | YITU CT software (YITU healthcare Technology Co. Ltd.) | N/A | Identifying radiological parameters on admission that can be used to predict patient outcomes critical vs. non-critical. | The predictive performance of AI-based quantitative chest CT parameters in terms of lung volume percentages in density intervals was evaluated, but its added value to the clinical variable was not investigated. AUC, sensitivity and specificity values are reported, but the accuracy is not included. The models were not tested on independent test cases. | 156 | AUC: 0.81 | 71.79% | 84.62% | N/A |
| Caruso et al. [101] | Thoracic VCAR application (GE Healthcare, Milwaukee, WI) | N/A | Building a multiple logistic regression model from clinical and quantitative CT parameter to predict fibrotic-like changes at six-month follow-up. | The accuracy of the clinical, radiological or combined model is not reported; only the AUC, sensitivity and specificity values are included. The models were not tested on independent test cases. | 118 | AUC: 0.92 | 100% | 73% | N/A |
| Han et al. [102] | Thoracic VCAR application (GE Healthcare, Milwaukee, WI) | N/A | Identifying clinical and radiological parameters that show association with developing fibrotic lung changes on the follow-up CT scans. | Only OR is reported; the AUC, accuracy, sensitivity, and specificity of the proposed multivariable logistic regression model is not included. | 114 | N/A | N/A | N/A | N/A |
| Yousef et al. [103] | Fujifilm Synapse 3D version 3.5: Thresholding histogram-based CT quantification model | N/A | Assessing the correlation between semi-quantitative CT severity scoring and automated quantification. | Only Spearman's correlation coefficients are reported to compare the two methods; the accuracy of automated CT severity scoring in patient risk stratification is not investigated. | 153 | N/A | N/A | N/A | N/A |
| Salvatore et al. [105] | Thoracic VCAR application (GE Healthcare, Milwaukee, WI) | N/A | Identify clinical and radiological parameters that can used to build a multivariate logistic regression model to identify discharged versus hospitalized/dead patients or to identify discharged/stable patients versus critical/dead patients. | The AI tool was not able to automatically segment 24/98 (24.5%) cases. The models were not tested on independent test cases. | 74 | (1) discharged vs. hospitalized/dead patients AUC 0.75; accuracy 70% (2) discharged/stable vs. critical/dead patients AUC 0.83, accuracy 81% | (1) discharged vs. hospitalized/dead patients 61% (2) discharged/stable vs. critical/dead patients 88% | (1) discharged vs. hospitalized/dead patients 85% (2) discharged/stable vs. critical/dead patients 78% | N/A |
| Fervers et al. [106] | CT Pulmo Auto Results (Philips Healthcare, Best, The Netherlands) | N/A | Comparing the results of threshold-based and deep-learning-based COVID-19 pneumonia lung involvement assessment and severity quantification. | The correlation between the deep learning-based lung involvement quantification and the manual chest CT score was investigated, but its impact on patient risk stratification and outcome prediction was not. | 250 | N/A | N/A | N/A | N/A |
| Freely available (software, module or code) | |||||||||
| Fan et al. [40] | Res2Net: Inf-Net Res2Net: Semi-Inf-Net, a 2D U-Net (Infection regions) | https://github.com/DengPingFan/Inf-Net | Creating a fully automated and a semi-supervised deep learning algorithm for segmenting lung areas affected by COVID-19 infection. | Despite the proposed neural networks overperformed the state-of-the-art deep learning models, their accuracy in terms of Dice coefficients were low. | N/A | N/A | - Inf-Net: 69.2% - Semi-Inf-Net: 72.5% | - Inf-Net: 94.3% - Semi-Inf-Net: 96.0% | - Inf-Net: 68.2% - Semi-Inf-Net 73.9% |
| Do et al. [51] | Yet Another CT-Analyzer (YACTA) | Identifying quantitative CT features and clinical parameters that can predict the necessity of invasive ventilation. | Accuracy is not reported. The models were not tested on independent test cases. | 53 | (1) 75th lung percentile parameter: AUC: 0.87 (2) Combined model: AUC: 1.0 | (1) 75th lung percentile parameter: 100% (2) Combined model: 100% | (1) 75th lung percentile parameter: 69% (2) Combined model: 100% | N/A | |
| Yue et al. [60] | FeAture Explorer on Python | https://github.com/salan668/FAE | Building radiomics-based machine-learning models to predict short vs. long-term in-hospital stay. | Accuracy is not reported. The test dataset only consisted of 5 patients. | 31 | (1) Logistic regression: AUC: 0.97 (2) Random forest: AUC: 0.92 | (1) Logistic regression: 100% (2) Random forest: 75% | (1) Logistic regression: 89% (2) Random forest: 100% | N/A |
| Yu et al. [66] | MVPNet: Multi-view PointNet for 3D Scene Understanding | https://github.com/maxjaritz/mvpnet | Identifying imaging features and clinical parameters as independent predictors for adverse clinical outcome (admission to ICU, occurrence of acute respiratory failure, or progressing into shock). | Only the OR values are reported, the accuracy, AUC, sensitivity and specificity values of the logistic regression model is not investigated. | 421 | N/A | N/A | N/A | N/A |
| Müller et al. [70] | MIScnn: Medical Image Segmentation with Convolutional Neural Networks and Deep Learning | https://github.com/frankkramer-lab/MIScnn | Creating a 3D U-Net- based image segmentation pipeline for lung and infection segmentation. | The performance of the model was evaluated during 5-fold cross-validation, no independent test set was used for validation. | 20 | N/A | COVID-19 infection segmentation: 73.0% | COVID-19 infection segmentation: 99.9% | COVID-19 infection segmentation: 76.1% |
| Qui et al. [73] | MiniSeg | https://github.com/yun-liu/MiniSeg | Creating a convolutional neural network for COVID-19 infection segmentation. | Although the proposed algorithm overperformed the state-of-the-art neural networks, its accuracy in terms of Dice score was low. The performance of the proposed algorithm was not tested on independent test cases. | 50 | N/A | 84.95% | 97.72% | 75.91% |
| Enshaei et al. [75] | DNN - COVID-CT-Rate | https://github.com/ct-segmentation/COVID-Rate | Creating a deep learning model for the segmentation of COVID-19 related lung lesions. | The external independent test set consisted of only 8 patients. | 82 | N/A | 83.5% | 99.7% | 80.7% |
| Yoo et al. [77] | MEDIP COVID19 | https://medicalip.com/ | To train and test a 2D U-Net-based model in COVID-19 related lung lesion segmentation. | 131 | N/A | (1) first external test set: 70.6% (2) second external test set: 79.9% (3) third external test set: 70.7% | N/A | (1) first external test set: 73.4% (2) second external test set: 71.9% (3) third external test set: 77.0% | |
| Czempiel et al. [86] | CNN, DenseNet | Developing a self-supervised learning scheme for COVID-19 related lung lesion segmentation, and to predict the presence of GGO and consolidation. | The sensitivity and specificity values of the proposed classification model is not reported. | 25 | (1) Overall opacities: accuracy: 0.76 AUC: 0.855 (2) GGO: accuracy: 0.75 (3) Consolidation: accuracy: 0.76 | N/A | N/A | (1) Overall opacities: 54.1% (2) GGO: 41.3% (3) Consolidation: 65.9% | |
| Published (code is not publicly available) | |||||||||
| Shan et al. [61] | VB-Net | To train and test a VB-Net-based model in COVID-19 related lung lesion segmentation. | 300 | N/A | N/A | N/A | 91.6% | ||
| Mahmoudi et al. [64] | U-Net | Training and testing a U-Net-based model for lung and lung infection segmentation, and classification. | The performance of the proposed model was not evaluated on independent test cases. The accuracy, sensitivity, and specificity values of the classification model are not reported. | 20 | (1) Lung segmentation: accuracy: 95% (2) Infection segmentation: accuracy: 94% (3) COVID-19 Classification: accuracy: 98% | N/A | N/A | (1) Lung segmentation model: 98% (2) Infection segmentation model: 91% | |
| Wang et al. [69] | COVID-19 Pneumonia Lesion segmentation network (COPLE-Net) | Creating a U-Net-based deep learning framework for the automatic segmentation of COVID-19 pneumonia lesions. | 558 | N/A | N/A | N/A | 80.72% | ||
| Yan et al. [71] | COVID-SegNet 3D U-Net model with Encoder and Decoder | Creating a deep learning framework for the automatic segmentation of lung and COVID-19 pneumonia lesions. | 130 | N/A | (1) Lung segmentation: 98.6% (2) COVID-19 lesion segmentation: 75.1% | N/A | (1) Lung segmentation: 98.7% (2) COVID-19 lesion segmentation: 72.6% | ||
| Gerard et al. [72] | LungNet with two CNN models: low-resolution model LungNet-LR and high-resolution model LungNet-HR | Creating a deep learning framework for the automatic segmentation of lung and lung lobes. | The performance of the proposed framework was not compared to other state-of-the-art methods. | 87 | N/A | N/A | N/A | 98.5% | |
| Pei et al. [74] | MPS-Net: Multi-Point Supervised Network | Creating a U-Net-based deep learning framework for the automatic segmentation of COVID-19 pneumonia lesions. | The exact number of cases is not reported. | ∼50 | N/A | 84.1% | 99.9% | 83.2% | |
| Bartoli et al. [76] | 2D CNN | Creating deep learning framework for the automatic segmentation and quantification of COVID-19 pneumonia lesions on low-dose CTs. | 174 | N/A | N/A | N/A | All opacities: 75% | ||
| Zhang et al. [82] | uAI Intelligent Assistant Analysis System: 3D CNN joined with VB-Net architecture | Determining the lobar distribution of COVID-19 pneumonia lesions by the uAI Intelligent Assistant Analysis System. | Only descriptive results are reported, and the primary presence of GGO and sub-solid lesions. The segmentation accuracy of the framework or its predictive performance are not investigated. | 2,460 | N/A | N/A | N/A | N/A | |
| Li et al. [83] | CNN VB-Net segmentation | Evaluating the difference in clinical characteristics and quantitative CT imaging features between baseline and follow-up CT scans. | Only descriptive results are reported, and the difference between baseline and follow-up scans. The segmentation accuracy of the framework or its predictive performance are not investigated. Low number of patients. | 14 | N/A | N/A | N/A | N/A | |
| Pu et al. [85] | U-Net model | Training and testing a U-Net-based model for lung, lung infection, and vessel segmentation. | The exact number of cases is not reported. It is not clear whether the CT scans were randomly split into training, validation and test datasets respectively to the patients. | 24 | Detection of affected regions: accuracy: 95% | Detection of affected regions: 95% | Detection of affected regions: 84% | (1) Lung segmentation: 95% (2) Pneumonia segmentation: 81% | |
| Grodecki et al. [90] | Convolutional network–Long Short-Term Memory (ConvLSTM) | Creating deep learning framework for the automatic multiclass segmentation of GGOs and high opacities including consolidation and pleural effusion. | (1) Internal test set: 197 (2) External test set: 68 | N/A | N/A | N/A | (1) Internal test set: 89.2% (1) External test set: 89.4% | ||
| Lu et al. [107] | uAI Discover-2019nCoV, Shanghai United Imaging Intelligence Co., Ltd, China: VB-Net-based approach incorporating thresholding and HITL model | Distinguishing patients with vs. without COVID-19 pneumonia based on quantitative CT parameters. | Only AUC, sensitivity and specificity are included, the accuracy is not included. The added diagnostic value of the quantitative CT features to the clinical parameters is not investigated. The performance of the parameters was not evaluated on independent test cases. | 47 | (1) GGO volume: AUC: 0.77 (2) GGO percentage in whole lung: AUC: 0.77 | (1) GGO volume: 66.67% (2) GGO percentage in whole lung: 66.67% | (1) GGO volume: 94.74% (2) GGO percentage in whole lung: 86.84 | N/A | |
Efficacy of AI software
Various studies assessed the value of AI approaches, listed in Table 3, to predict clinical deterioration or death in patients hospitalized with COVID-19 by quantifying CT scans for the extent of lung lesions.
AI Software approaches assessed in COVID-19 studies. Abbreviations: GGO = ground glass opacity. COVID-19 = coronavirus disease 2019. CNN = convolutional neural networks
| AI softwares | Approach: |
| CT Pneumonia Analysis | Segments and quantifies the extent of overall abnormalities by delineating GGO and consolidation opacities on a global and lobe-wise disease burden relative to the total lung volume. |
| Yet Another CT-Analyzer (YACTA) | Lung segmentation analyzing lung density, fibrosis index, GGO and well-aerated lung regions. |
| CAD4COVID | Lobar quantification of COVID-19 severity extent by calculating a severity score for each pulmonary lobe as the percentage of the total affected lung parenchyma volume. |
| RAD-LogicsTM | Detects lung nodules and focal opacities by combining an AI software with a 2D U-Net CNN for detecting large-size diffuse opacities. |
An AI-based application called “CT Pneumonia Analysis” made available on the syngo.via workstation by Siemens Healthineers, automatically segments and quantifies the lungs, lobes, and affected regions by delineating the abnormal regions and measuring the global and lobe-wise burden of the disease relative to the total lung volume. By detecting abnormal CT patterns, the algorithm quantifies the extent of overall abnormalities and the incidence of opacities. As an output, the software application calculates a pneumonia severity score by measuring the percentage of GGO, consolidation, and total opacity score for each lobe and the sum for all the lung parenchyma as an early outcome predictor for COVID-19 pneumonia [45–50].
In a study conducted on 323 COVID-19 patients, Chabi et al. [45] used the AI-Rad Companion Research CT Pneumonia Analysis application developed by Siemens Healthineers for COVID-19 burden of the lung and found that clinical deterioration and mortality correlated with a higher extent of lung volumes affected by all opacities (37.48 ± 21.47%) and consolidation (8.87 ± 8.22%). On the other hand, those patients who did not require critical care or were discharged had a lower extent of lung volumes affected (22.63 ± 17.01% and 5.00 ± 5.64%, respectively). They also concluded that CT-derived measurements combined with clinical and biological markers led to significant improvements in the prediction of clinical deterioration and in-hospital mortality (P = 0.049). The study proved that these two quantitative CT parameters, namely volume percentage of all opacities (OR, 2.70, 95% CI: 1.49–4.88, P < 0.001) and consolidation (OR, 4.08; 95% CI: 1.90–8.78, P < 0.001) are independent predictors for clinical deterioration or mortality [45]. Stasiow et al. [46] automatically calculated the percentage of affected lung tissue using the syngo.via software. They reported the percentage of lung involvement, chest CT score, and total opacity score to be independent predictors of ICU admission, need for artificial ventilation, and death [46]. Sezer et al. [47] also correlated the disease severity depicted by higher total opacity scores in 96 COVID-19 patients, with ICU entries, mortality rates, and patients having a worse clinical course. Gashi et al. [48] conducted a retrospective study on 108 COVID-19 patients (group A) and 88 asymptomatic patients performing pre-operative CT scans (group B), to assess the performance of syngo.via software, in detecting COVID-19 abnormalities compared to CO-RADS scores which were calculated by 3 experienced radiologists (inter-reader reliability kappa coefficient 0.87). All AI-assessed variables showed significant differences between group A and B patients (P < 0.01). Although there was a statistically significant difference between the AI assessment and human scoring for all variables, the AI-based application showed a good clinical correlation in perspective with the CO-RADS score predicting COVID-19 with an accuracy of 0.94 [48]. Okuma et al. [49] studied the ability of syngo.via software to differentiate the severity of COVID-19 pneumonia in the initial stages of 100 COVID-19 patients. The study concluded that the mean density of the lung parenchyma in Hounsfield units (HU), and the eGFR, are independent factors (P < 0.0001) in predicting the severity of COVID-19 pneumonia, with cut-off values of −801 HU and 77 mL/min/1.73 m2 for mild and moderate pneumonia and −704 HU and 53 mL/min/1.73 m2 for moderate and severe pneumonia [49]. Mergen et al. [50] also used the syngo.via software on 60 COVID-19 patients and found that patients requiring mechanical ventilation had a significantly higher percentage of opacity (indicating GGO and consolidation) and a higher percentage of high opacity (indicating consolidation). In fact, the need for mechanical ventilation increased significantly in patients with a higher percentage of opacity (median 44%, IQR: 23–58% vs. median 13%, IQR: 10–24%; P = 0.001) and those with a higher percentage of high opacity (median: 11%, IQR: 6–21% vs. median 3%, IQR: 2–7 %; P = 0.002) when compared to those without [50].
Do et al. [51] conducted a study on 53 hospitalized COVID-19 patients using another AI-based lung segmentation software, the so-called Yet Another CT-Analyzer (YACTA). The authors analyzed quantitative CT parameters including mean and percentiles of lung density, fibrosis index (defined as the percentage of segmented lung voxels ≥ −700HU), GGO, and well-aerated lung regions. The 75th percentile of lung density (−621.5 ± 158.5 HU) and the fibrosis index (30.6 ± 14.47%) were found to be independent predictors of the patient's outcome in regards to the number of days with invasive ventilation. In addition, when combining the AI-based parameters with laboratory markers such as LDH and procalcitonin, which were also found to be independent predictors of patient outcome, an AUC of 0.99 and 1.0 were achieved, respectively. This proved the feasible significance of using automatic quantification in facilitating the prediction of the mode of ventilation in COVID-19 patients [51].
Szabó et al. [52] automatically quantified the CT scans of 326 COVID-19-confirmed, hospitalized patients using an AI-based algorithm called CAD4COVID by calculating a severity score for each pulmonary lobe as the percentage of the total affected lung parenchyma volume. A representative example of the AI-based CAD4COVID–CT software can be seen in Fig. 2 (central illustration). The AI-based COVID-19 severity score (OR = 1.08; 95% CI = 1.02–1.15) was found to be an independent predictor for clinical deterioration, which was defined as the need for ICU, invasive ventilation, use of vasopressors or in-hospital mortality [52].
AI-based CAD4COVID–CT software of a patient with a total CT severity score of 8. A 67-year-old patient undergoing chemotherapy for prostate cancer was hospitalized with an SpO2 of 95% at the time of admission. CT severity scores, affected areas, lobe volumes, and emphysema areas are reported on the right side. Severity scores were calculated using the percentage of the affected area of the parenchyma [52]. Abbreviations: CT = computed tomography. SpO2 = oxygen saturation
Citation: Imaging 16, 1; 10.1556/1647.2024.00167
In another study, a commercially available AI-based software developed by the RAD-LogicsTM for detecting lung nodules and focal opacities was combined with a 2D U-Net CNN for detecting large-size diffuse opacities. The first AI-based submodel utilizes 3D volumes of the whole CT scan to detect nodules and focal opacities. However, detecting small-sized opacities is not sufficient in COVID-19 patients. A U-Net-based second submodel was therefore developed to segment slice-by-slice larger and diffused opacities. The two submodels were combined to complement each other and create a volumetric measurement of opacity burden called Corona score. This score tracks the progression and regression of COVID-19 over time as described by the extent of lung injury. The disease evolution in a COVID-19-confirmed patient was shown to have a decreasing Corona score with recovery. The first CT obtained 1–4 days after initial symptoms had a Corona score of 191.5 cm3. The second CT 4 days later, had a 97.1 cm3 score, showing a 49% reduction in opacity-burden. Finally, 15 days later the patient had no opacity-burden with a Corona score of 0. In addition, 18 COVID-19 patients were quantitatively assessed with a Corona score for 30 days, showing a decreasing trend in lesion volume over time [53].
Efficacy of neural networks
By automatically extracting the hidden contextual features of images, CNNs have achieved promising results in various studies. The architecture of CNNs shows wide variations, however, the pipeline of the CNN can be divided into four simple steps. First, the input layer receives the input image and transfers it to the hidden layers. Second, the convolutional layers extract feature maps using kernels (also known as convolution filters). Third, pooling layers are often used to reduce dimensionality, which not only shortens training time but also reduces the risk of overfitting. And fourth, the architecture is finally completed with fully-connected layers (also known as dense layers), where the classification decision is made [54]. In order to optimize the performance, cost functions such as cross-entropy are used to monitor the training and validation loss, and optimizers such as stochastic gradient descent or Adam optimizer are used to iteratively update the weights during training the CNN [55].
CNN can automatically generate features from image input and construct a consequential representation. The usage of a CNN with a patch-based architecture was studied in the quantification of five interstitial lung disease (ILD) patterns [56]. Since COVID-19 shares patterns that are similar to ILDs, techniques for the quantification of the progression of ILD translate as important tools for quantifying COVID-19. In fact, CNNs have been revised further by implementing semantic segmentation instead of only image classification. Thus, instead of deducing a single class for an image, with semantic segmentation each of the pixels of an image is assigned a class [57, 58]. Segmentation has proven to be important since locating the precise region of lesions and nodules is a challenging task due to their small sizes, different shapes, and patterns [58]. The most prevalent segmentation networks, which have been used for COVID-19 are the U-Net [21, 59, 60], the U-Net++, and the VB-Net [61] architectures. A list of the neural networks approaches assessed can be seen in Table 4.
Neural networks approaches assessed in COVID-19 studies. Abbreviations: COVID-19 = coronavirus disease 2019
| Neural networks | Approach: |
| U-Net architecture | Encoding extracts high-resolution features from input images (encoding), and integrates finer image features by fusing and allocating their precise location that segments the regions of interest by generating masks (decoding). |
| U-Net++ architecture | inserts a nested convolutional structure between the encoding and decoding paths of the U-Net architecture |
| V-Net and VB-Net architecture | V-Net: adds residual modules to enable learning a residual function, and uses convolution layers instead of max pooling layers for up-sampling and down-sampling. VB-Net: V-Net equipped with bottleneck residual blocks, employing the use of two paths responsible for extracting global image features to integrate finer image features. |
| Deep learning models for longitudinal studies | DL models applied to investigate their performance in quantifying affected regions during follow-up periods |
| YITU Software | combines U-net and Fully convolutional networks, consisting of convolution segments with layers for downsampling and upsampling to detect lung lesions and grade the severity of COVID-19 pneumonia. |
U-net architecture
Ronneberger et al. [14] proposed U-Net in 2015, a deep CNN architecture for medical image segmentation that is currently widely used. The U-Net has an encoding path (also known as the contracting path) responsible for the extraction of high-resolution features from input images, and a decoding path (also known as the expansive path) responsible for integrating finer image features by fusing and allocating their precise location that segments the regions of interest by generating masks [14]. This architecture offers promising results for segmenting medical images, especially when limited data is available [62]. The novelty of this network architecture is that it is able to learn better visual semantics and more detailed contextures by employing concatenation, direct links between the layers of the encoder and decoder paths of the same given level [63]. Later in 2016, the 3D U-Net developed at the University of Freiburg was introduced by Cicek et al. [58] for volumetric medical image segmentation.
The U-Net was successfully used for lung region and lung lesion segmentation in patients with COVID-19 [21, 59, 60]. However, adequate training of a robust segmentation network requires large training datasets, and the manual slice-by-slice delineation of lesions is very laborious and tedious. The U-Net-based automatic segmentation allows for accurate quantification of the affected lung regions, which can be used for follow-up of disease progression and evaluating the severity. Yue et al. [60] predicted the hospital stay of patients by extracting radiomics features from the usage of a U-Net-based algorithm. The logistic regression discriminating short-term (≤10 days) and long-term (>10 days) hospital stay in COVID-19 patients with an AUC of 0.97 (95%CI 0.83–1.0), a sensitivity of 1.0, and a specificity of 0.89 [60]. Huang et al. [21] monitored COVID-19 progression by segmenting lung regions and calculating lung involvement through the quantification of GGO percentage in 126 hospitalized COVID-19 patients' CT images. The whole-lung opacity percentage increased significantly in patient CT follow-up when compared to the baseline CT (median [IQ range]: 8.7% [2.7%, 21.2%] vs. 3.6% [0.5%, 12.1%]; P < 0.01) illustrating a progression of the disease [21]. In another study, a U-Net architecture was used to segment lungs and generate a 3D reconstruction, reporting 98% accuracy in quantifying the ratio of infected lungs to healthy lungs when classified. The study consisted of four stages, lung segmentation, infection segmentation, COVID-19 classification, and its 3D reconstruction by using CNN techniques for the layering and classification [64]. Grodecki et al. [65] conducted a study on 120 patients with COVID-19 to examine the value of CT-derived quantitative pulmonary burden measured with AI-assisted FusionQuant Lung software in predicting clinical deterioration (ICU admission, invasive mechanical ventilation, or vasopressor therapy) or in-hospital death. The U-Net-based DL model was trained for segmenting opacity lesions and labeling them according to the presence of GGO, consolidation, or pleural effusion. In this study, emphysema and fibrosis were excluded from the segmentation. Bilateral consolidation seen on CT was reported to predict clinical deterioration (OR 4.84). Increasing GGO attenuation (OR 3.2; 95% CI: 1.3, 8.3 per standard deviation; P = 0.02) and consolidation burden (OR 3.4; 95% CI: 1.7, 6.9 per doubling; P = 0.001) were independent predictors for deterioration or death. It was found that patients with a GGO burden ≥13.5% and consolidation burden ≥1.8% resulted in a greater risk of adverse outcomes by approximately fivefold. Moreover, an incremental predictive performance was seen with the addition of quantitative CT parameters compared to the value of clinical variables alone (AUC 0.93 vs. 0.83; P = 0.006) [65]. Yu et al. [66] performed a similar study by automatically quantifying the lesion volume percentage, reporting that COVID-19 patients who had large consolidation of the upper lung at admission, correlated to higher risk. In this study, however, either emphysema or fibrosis was not excluded, and the models were not adjusted for clinical variables [66].
Different variations to the basic architecture of U-Net have been proposed to improve its performance. Some include integrating DenseNet blocks in its encoding path or incorporating attention mechanisms [14]. Transfer learning is another useful technique used in DL to overcome data shortage by taking advantage of an already pre-trained dataset (i.e. natural image ImageNet dataset) as the backbone of the encoding path in a CNN model, to improve segmentation performance [67].
Dice similarity coefficient (DSC) is an often used metric to evaluate the overlap ratio and reproducibility found between an automatically segmented infected region and the corresponding manually segmented regions by an expert. Both 2D and 3D U-Net models have been investigated. The 2D U-Net models process 2D images (CT slices) differently than 3D U-Net models, which require large-scale datasets, and preserve temporal information by producing 3D volumes of the frames [68]. In practice, 3D networks, which are able to segment whole lung volumes for disease extent, are more requested. Apart from an economic standpoint, this would also require large amounts of 3D lung volumes for training. A U-Net model evaluated 2D and 3D variants on 558 chest CT images to assess COVID-19 infection segmentation. The models resulted in a DSC of 79.97% for the 2D U-Net model and 70.35% for the 3D U-Net model, highlighting better results on smaller datasets for 2D [69]. Müller et al. [70] used data augmentation on their 3D U-Net model, to generate unique and random image patches to segment COVID-19 lesions resulting in a DSC of 76.1%. COVID-SegNet 3D U-Net model proposed by Yan et al. [71] achieved a DSC of 72.6%, a sensitivity of 75.1%, and a precision of 72.6% for lesion segmentation. Gerard et al. [72] added polymorphism to the SegNet architecture, which increased the performance further when it segmented poorly or non-aerated lung regions (having up to 25% volume of the non-aerated lung compartment). Fan et al. [40] used Semi-Inf-Net, a 2D U-Net architecture employing an attention technique called ground-truth, which segments infected regions on chest CT, enhances and devotes more importance to some features while excluding others. This technique obtained a DSC of 73.9% and a specificity rate of 96.0% [40]. MiniSeg, a lightweight DL model to segment ROIs, functioning by compressing DNN into compact ones to use less memory, was proposed by Qui et al. [73] achieving a DSC of 75.91% and a specificity rate of 97.72% for COVID-19 infection segmentation. Pei et al. [74] used a 2D multi-point supervision network (MPS-Net) to segment COVID-19 lesions obtaining a DSC of 83.2%, a sensitivity of 84%, and a specificity of 99.8%. Enshaei et al. [75] evaluated a DNN called COVID-Rate by segmenting COVID-19 lesions efficiently using both 2D CT slices and whole-volume scans. A DSC of 80.7%, a specificity of 99.7%, and a sensitivity of 83.5% were found to support its efficiency. Bartoli et al. [76] investigated the quantification accuracy of a 2D CNN model by comparing the manual segmentation of pulmonary lesions in 30 COVID-19 patients. The authors reported acceptable accuracy with a DSC of 75% for lesion segmentation [76]. The quantification of lung extent (%) and weight (g) of pneumonia calculated from a 2D U-Net architecture model from South Korea - MEDICALIF, compared to manually derived segmentations from radiologists with a Pearson correlation coefficient (r) of 0.908 and 0.899 respectively. Both extent and weight were independently associated with symptoms with an OR = 5.523; r = 0.041 and 10.561, r = 0.016 respectively [77].
U-Net++ architecture
U-Net++, a more complicated architecture of U-Net, works by inserting a nested convolutional structure between the encoding and decoding paths. Although training this type of network is more difficult, it aids in the localization of COVID-19 lesions and thus advances the performance of segmentation [78]. Zhou et al. [79] used a 2D U-Net++, which with various skip layers allows a more accurate segmentation in diseased areas. By incorporating residual neural networks, Zhang et al. [80] improved U-Net further to create the Res-Unet, which is able to demonstrate additional performance improvement. Residual networks work by trying to increase the depth of the network to reduce the number of parameters.
V-Net and VB-Net architectures
The volumetric V-Net architecture has been developed inspired by the U-Net architecture in 2016. The U-Net was originally designed for 2D image segmentation, and its 3D variation with 3D kernels was applied for segmenting volumetric medical images. The V-Net was also designed for processing 3D volumetric medical images by using volumetric kernels, thus can complete the segmentation of 3D volumes. The V-Net is comparable to a 3D U-Net architecture with the difference that it adds residual modules to enable learning a residual function, and uses convolution layers instead of max pooling layers for up-sampling and down-sampling [81].
Further improving the efficiency of the segmentation is VB-Net, V-Net equipped with bottleneck residual blocks, employing the use of two paths responsible for extracting global image features to integrate finer image features. The incorporation of bottleneck structure allows VB-Net to be faster than V-Net by reducing and combining features [81].
The V-net architecture and its variants each with different modifications to try to improve their performance, were successfully applied to COVID-19 patients. Shan et al. [61] with the help of a human-in-the-loop (HITL) strategy to accelerate the manual delineation of lesions on CT scans during training, validated the VB-Net neural network on 300 COVID-19 patients. HITL techniques are widely used by radiologists to refine ML performance and speed up the tuning of a particular algorithm. The automatic DL segmentation model had higher accuracy than manual segmentation when delineating lung regions, resulting in an average DSC of 91.6% [61]. Zhang et al. [82] also propose a 3D CNN joined with VB-Net architecture, to localize and quantify COVID-19 pneumonia. Their study tested 2,460 CT images from a single-center hospital in Wuhan, China, which was able to assess and evaluate COVID-19 patients.
Deep learning models for longitudinal studies
The uses of DL models have also been applied to investigate their performance in longitudinal studies. In a small study of 14 patients, CNN VB-Net segmentation was used to delineate infected regions on follow-up CT scans in order to quantify the decrease of affected regions during the 4-month follow-up period. The percentage of infection significantly decreased in the follow-up period in all 14 patients (P < 0.001) with a significant decrease in GGO and consolidation (P < 0.001) [83].
Serial CT images were quantified in 95 COVID-19 patients by comparing the efficacy of a U-Net model, YITU, and a standard semi-quantitative CT scoring system. The potential of the DL's ability to assess severity was confirmed with a Spearman's correlation coefficient of 0.92 between the groups [84]. In another study, 72 serial CT scans were tested of 24 COVID-19 patients using a U-Net model to identify the affected regions and quantify the progression of the disease. The model had a DSC of 81%, a sensitivity of 95%, and a specificity of 84% in quantifying the COVID-19 lung infiltrates on CT images [85].
Czempiel et al. [86] investigated a CNN, DenseNet, which incorporated two data augmentation methods, context disordering and black patches, to quantify GGO and consolidation. By creating modified copies of already existent data, these techniques serve to artificially augment the training set. A total of 50 serial CTs of 25 COVID-19-confirmed patients were tested resulting in a DSC of 62.3% vs. 64.4%, 63.5% vs. 65.2%, and 63.6% vs. 65.9% when comparing with or without pre-training, black patches, and context disordering respectively. Context disordering proved to perform best with an AUC of 0.844 and an accuracy of 0.755 [86].
Recurrent neural networks (RNN) are a type of DL that works on sequence data. Through connections between nodes, directional graphs are formed along a temporal sequence via their internal memory to process input. This is a more complex model, which feeds the results back into the model to predict the outcome layer. This backpropagation has a memory cell that computes and implements operations to work toward the correct prediction. Long-Short-Term-Memory (LSTM), an RNN, is used with CNNs to extend their neighboring pixels. However, they cannot process long sequences and have less feature compatibility. They have achieved promising predictions with sequential datasets by using current information along with previous timestamps. This model is used when predictions require information from past and current knowledge, such as whether a patient is infected with COVID-19 or not [87]. The architecture of these networks is divided into two branches: the main segmentation branch to extract larger and easy-to-classify lesions [88], and an attention branch using five-fold cross-validation to segment the lesions and correct the borderline of the main branch. This allows to preserve relevant features while dismissing irrelevant ones simultaneously, resulting in a feedback loop for a memory-sparing strategy [89].
Grodecki et al. [90] proposed a convolutional network–Long Short-Term Memory (ConvLSTM), to quantify and differentiate COVID-19 pneumonia lung lesions. They reported lung lesions of 0.87 ± 0.005, illustrating a strong correlation between expert manual measurements and automatic segmentation, with DSC of 0.978 for GGO and 0.981 for high opacity volumes. When validated on an external test dataset of 67 patients, a DSC of 0.767 ± 0.009, and correlations of 0.989 and 0.996 for GGO and consolidation were achieved, respectively. This proves the rapid ability to assess COVID-19 pneumonia severity on chest CT [90]. Longitudinal studies have helped improve classification and segmentation models to better understand the effects of COVID-19 by allowing the analysis of data from serial images at different times to track the severity of the disease.
YITU software
Yitu Healthcare based in Shanghai, China, released the DL-based YITU software functioning by combining U-net and Fully convolutional networks, which consist of convolution segments with layers for downsampling and upsampling to grade the severity of COVID-19 pneumonia.
Chrzan et al. [91] used the software to illustrate the significant difference between GGO and consolidation in patients with COVID-19 pneumonia, bronchopneumonia, and atypical pneumonia. Both COVID-19 and atypical pneumonia showed an elevated volume percentage of GGO of 8.61% and 10.41% respectively when compared to bronchopneumonia of 3.42%. Even though COVID-19 pneumonia had significantly elevated consolidation volume (2.95%) when compared to atypical pneumonia (1.26%), the usefulness of differentiating the two is limited due to the partial overlap [91]. Chrzan et al. [92] in their other study predicted the severity of COVID-19 pneumonia using the YITU software. The radiological parameters were correlated to the patient's clinical parameters, reflecting the severity of the disease (P < 0.05). Absolute consolidation volume and percentage consolidation volume were reported to be independent factors associated with in-hospital mortality (OR = 4.08, 95% CI = 2.62–6.35 and OR = 4.05, 95% CI = 2.60–6.30, respectively) [92].
In a similar study, Pang et al. [93] also used the YITU software to automatically quantify COVID-19 patients by assessing the percentage volume of inflammation, GGO, and consolidation. A cutoff value of 22.6% with an AUC of 0.868, a sensitivity of 81.3%, and a specificity of 80.6% for percentage inflammation volume was found to be the most effective predictor of critical illness (defined by ICU admission, respiratory failure requiring mechanical ventilation, shock or death) in patients [93].
Pan et al. [84] reported a Spearman's correlation coefficient of 0.929 (P < 0.001) between the performances of the YITU software and the conventional CT scoring in assessing the percentage of pulmonary lesions. Clinically severe COVID-19 patients presented GGO and consolidation lesions (23.03% and 4.99%, respectively) higher in comparison to moderate patients. In addition, consolidation was found to resorb at an earlier time than GGO (19 days vs. 23 days) [84].
The YITU software was used to calculate quantitative parameters of lung volume percentages at different density intervals in a study conducted by Ding et al. [94], to predict the critical (critical or death) or non-critical (mild, moderate, severe) outcomes of COVID-19 patients. They reported the density interval of −200 to 60 HU as the best predictor of critical illness with a cut-off value of 5.93% (AUC = 0.808; sensitivity = 71.79%; specificity = 84.62%). In the early stages (days 0–4), no significant difference between the two groups was seen in lung volume percentage at different density intervals. However, a significant difference was seen within days 5–9, reflecting the progression and potential ability to distinguish the severity of COVID-19 [94].
Thresholding networks
Since affected opacity regions have markedly different densities compared to healthy lung parenchyma, thresholding-based DL approaches have also been investigated for the quantification of COVID-19 pneumonia. This technique calculates lung involvement via volumes, percentages, and the densities of GGO and consolidation in both lungs for the quantitative assessment of the affected regions. It is done by segmenting images based on the intensity histogram of the grayscale values, which rely on the analysis of multiple attenuation thresholds determined from the frequency distribution of the histogram. Depending on the multiple grayscale thresholds, the voxels within the scans are placed into various classes [95–97].
GE HealthCare designed the Thoracic VCAR application providing the automatic segmentation of the lungs by using an adaptive thresholding HU to identify different density-based morphology. By constructing a calorimetric map, the software is able to recognize and differentiate the GGOs from consolidation and quantify them as percentages with respect to healthy lung parenchyma by evaluating the progression or regression of the disease. Grassi et al. [98] tested the Thoracic VCAR application, on CT images to calculate the healthy residual lung parenchyma, emphysema, GGO, and consolidation volumes on 116 COVID-19 patients. GGO lesions were found to be the most prevalent in patients (93.6%) revealing a 19.50% for volume percentage value and 0.64 L as a median value. GGO volume also showed statistical significance between patients with suspected and without suspected COVID-19 infection (P<<0.01) [98]. Grassi et al. [99] correlated the evolution of the disease with the quantified assessment of the affected regions determined by using the Thoracic VCAR application. Patients who were discharged demonstrated 12.5% of lung involvement, while patients that had deceased showed 57.1% involvement. A complete resolution was seen on CT images in 48.1% of patients who reported ≤5% lung disease involvement. Patients who were hospitalized with a more severe extent of disease involvement reported CT abnormalities until day 16, which began resorbing only after 21 days [99]. This can be explained by Zhou et al. [100] who described COVID-19 as a course having three stages: the early rapid progressive stage from day 1–7, the advanced stage from day 8–14, with the lung abnormalities starting to decrease after day 14. The abnormalities were predominantly seen to be in the peripheral, middle, and lower areas of the lung [100].
Caruso et al. [101] evaluated the performance of the Thoracic VCAR application in predicting fibrotic-like changes (characterized by reticular pattern and/or honeycombing) on 6-month CT scans, seen in 72% of the 118 COVID-19 patients investigated. The Thoracic VCAR determined aerated lung volumes (cutoff ≤3.75 L and ≤80%) predicted the 6-month changes with an AUC >0.88 with respect to the healthy lung parenchyma. The presence of fibrosis-like changes on CTs showed a significant increase (P < 0.05), whereas all non-fibrosis-like abnormalities had decreased [101]. Han et al. [102] also saw these findings with approximately one-third of survivor patients (35%) who were infected with COVID-19 manifested fibrosis-like changes evolving from GGOs and/or consolidations, when evaluated at a 6-month chest CT follow-up [102].
Yousef et al. [103] performed a study on 153 positive COVID-19 patients, which were divided into severity groups based on the need to intubate. They reported a good quantitative correlation of COVID-19 pneumonia with an automated density-based multi-level image thresholding histogram-based CT quantification model (Synapse 3D version 3.5) and two expert radiologists using conventional severity scoring with a Spearman's correlation coefficient of 0.934 with a P < 0.0001) [103].
Lanza et al. [104] found that the percent of compromised lung volume defined as the sum of poorly and non-aerated volumes (−500, 100 HU) determined from a density-based quantitative lung analysis model, is the best predictor of the need for clinical respiratory support and hospital death in COVID-19 pneumonia. Patients with values 6–23% had an increased risk for the need for oxygen support, with values >23% having an increased risk for intubation and it was also identified as an increased risk factor for in-hospital mortality (P < 0.001) [104].
Similarly, Salvatore et al. [105] also found the volumes of consolidation, and residual healthy lung quantified by a density-based CT quantitative technique to be statistically significant for clinical outcomes of COVID-19 patients such as discharge, hospitalization in stable conditions, hospitalization in critical conditions, and mortality. GGO volumes on the other hand were reported not to affect patient outcomes [105].
The DL-based application “CT Pulmo Auto Results” of the IntelliSpace Portal (Philips Healthcare, Best, Netherlands) and the implementation of threshold-based segmentation, were evaluated by Fervers et al. [106] in a multi-center study. The threshold method showed a robust cutoff at −522 HU for the reproducibility of lung involvement quantification (r2 = 0.80). However, when compared to semi-quantitative CT scoring, it had a weaker correlation than the DL (r2 = 0.62 vs. r2 = 0.80). This could be explained by the overestimation of lung involvement in the threshold-based model due to larger pulmonary vessels and lung areas being above the threshold density [106].
Lu et al. [107] investigated an improved VB-Net-based approach incorporating thresholding and HITL model strategy by two experts, to automatically segment and quantify the extent of lung GGOs and consolidation in 47 suspected COVID-19 patients. The volume of GGOs and the percentage of GGOs in the whole lung at a threshold of −300 HU were found to be the most effective in diagnosing COVID-19 with an AUC of 0.769 and 0.769, a sensitivity of 66.67 and 66.67%, a specificity of 94.74% and 86.84%, respectively. When compared to the consolidation in the whole lung the thresholds at −400HU, −350HU, and −250HU were found to be statistically significant [107].
The percentage of lung abnormality (PLA) from COVID-19 infection was calculated by a U-Net-based DL approach, which evaluated seven widely used thresholding methods on CT scans. From these, Khan et al. [108] proposed a model, which used the best-fit methods to generate the most appropriate threshold values for segmentation. The proposed model reported PLA to have an improved precision of 47.49% and a specificity of 98.40% with a difference of ±3.89% from ground-truth lung ROI segmentation. This can play an important role in assessing disease progression and severity of pneumonia in the early stages [108]. Image segmentation is a tool for lung delineation and measuring the area or volume of the given lesions. However, the integration of clinical manifestations along with laboratory results and imaging is still important to create a full clinical image of a patient.
Limitations of AI
Various AI approaches have been established in an attempt to provide clinical support. An issue that must be overcome in many research studies using AI, is the lack of evaluating the models on reliable external CT image datasets, resulting in the risk of overfitting to a particular dataset when trained. Although ML and DL require large datasets to produce acceptable results, good results were also obtained with smaller datasets [23]. When the model tries to learn a voluminous amount of details in the relatively small-sized training set, it confounds it with noise as well. This can be mitigated with techniques that have been mentioned such as attention mechanisms, segmentation-before-classification, or data augmentation techniques [109].
DL architectures have evolved over the years. Lung imaging has played an invaluable role during the pandemic in being able to identify and manage individuals in the early stage of the disease [110]. Chest CT imaging has been demonstrated to be the most accurate in detecting the severity of acute manifestations of the lung [111]. Many studies have made the base models more effective by tackling different problems that AI techniques may come across. A proper combination of some of the techniques could potentially lead to an applicable model.
Problems with these models are that they are only trained on COVID-19-related images and they cannot differentiate COVID-19 lesions from other pneumonia lesions, which have similar features on CT. Studies have further been conducted to distinguish COVID-19 from non-COVID-19 pneumonia. An example of such a study was by Chen et al. [112], who were able to successfully differentiate COVID-19 pneumonia from other non-COVID-19 pneumonia using AI. This aids clinicians and radiologists in identifying COVID-19 patients and preventing the spread of the disease by early diagnosis [112]. CT indications and acquisition protocols are also not standardized across different facilities. A limitation such methods face, however, is the question of reproducibility and generalizability, which is jeopardized when trained on data from single centers and from the same geographical regions [113]. The majority of studies investigate small datasets and are single-centric studies conducted in one center with one geographic region. Future studies would benefit from databases that have datasets available to be able to perform multi-centered and larger-scale patient investigations.
Conclusion
AI techniques have been shown to have tremendous contributions to the diagnostics and follow-up of COVID-19 patients. The pandemic has enabled the use of different ML models to investigate their relevance for integration into the clinical management infrastructure. The risings of AI technologies play an essential role in strengthening the use of imaging tools in fighting against COVID-19 pneumonia. Facilitating the accurate delineation and quantification of COVID-19 infection-affected lung regions on chest CT images allows for improved patient management in a clinical setting. This review focused on the integration of AI techniques with CT images for lung quantification, in an attempt to help against the COVID-19 pandemic. Automated DL-based disease quantification on CT scans combined with clinical biomarkers in an integrated approach has shown to facilitate the staging of COVID-19 patients.
Authors' contributions
CN: Literature review, Writing - Original Draft Preparation; JS: Writing - Review and Editing; BKB: Literature review, Writing - Review and Editing
Conflict of interest
No conflict of interest.
Data availability statement
Not applicable.
Ethical statement
Due to the article type (review article), ethical approval is not required. This article does not contain any studies with human or animal participants, therefore, informed consent is not applicable.
References
- [1]↑
Sohrabi C, Alsafi Z, O’Neill N, Khan M, Kerwan A, Al-Jabir A, Iosifidis C, et al.: World Health Organization declares global emergency: a review of the 2019 novel coronavirus (COVID-19). Int Surg J 2020; 76: 71–76.
- [2]↑
World Health Organization: Coronavirus disease 2019 (COVID-19) situation report, WHO, Geneva, Switzerland. Accessed: Jan. 06, 2023. [Online]. Available: https://covid19.who.int.
- [3]↑
Zhou F, Yu T, Du R, Fan G, Liu Y, Liu Z, Xiang J, et al.: Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: A retrospective cohort study. Lancet 2020; 395: 1054–1062.
- [4]↑
Li K, Fang Y, Li W, Pan C, Qin P, Zhong Y, Liu X, et al.: CT image visual quantitative evaluation and clinical classification of coronavirus disease (COVID-19). Eur Radiol 2020; 30: 4407–4416.
- [5]↑
Feng W, Newbigging AM, Le C, Pang B, Peng H, Cao Y, Wu J, et al.: Molecular diagnosis of COVID-19: challenges and research needs. Anal Chem 2020; 92: 10196–10209.
- [6]↑
Liang T: Handbook of COVID-19 prevention and treatment. Zhejiang University School of Medicine 2020. Accessed: Jan. 06, 2023. [Online]. Available: https://globalce.org/downloads/Handbook_of_COVID_19_Prevention_en_Mobile.pdf.
- [7]↑
Fang Y, Zhang H, Xie J, Lin M, Ying L, Pang P, Ji W: Sensitivity of chest CT for COVID-19: comparison to RT-PCR. Radiol 2020; 296: E115–E117.
- [8]↑
Leonardi A, Scipione R, Alfieri G, Petrillo R, Dolciami M, Ciccarelli F, Perotti S, et al.: Role of computed tomography in predicting critical disease in patients with covid-19 pneumonia: a retrospective study using a semiautomatic quantitative method. Eur J Radiol 2020; 130: 109202.
- [9]↑
Xiong Y, Sun D, Liu Y, Fan Y, Zhao L, Li X, Zhu W: Clinical and high-resolution CT features of the COVID-19 infection: comparison of the initial and follow-up changes. Invest Radiol 2020; 55: 332–339.
- [10]↑
Phua J, Weng L, Ling L, Egi M, Lim CM, Divatia JV, Shrestha B R, et al.: Intensive care management of coronavirus disease 2019 (COVID-19): Challenges and recommendations. Lancet Respir Med 2020; 8: 506–517.
- [11]↑
Pan F, Ye T, Sun P, Gui S, Liang B, Li L: Time course of lung changes at chest CT during recovery. Radiol 2020; 295: 715–721.
- [12]↑
Pan F, Ye T, Sun P, Gui S, Liang B, Li L, Zheng D, et al.: Time course of lung changes on chest CT during recovery from 2019 novel coronavirus (COVID-19) pneumonia. Radiol 2020; 296: 715–721.
- [13]↑
Prokop M, Van Everdingen W, Van Rees Vellinga T, Van Ufford H Q, Stöger L, Beenen L, Geurts B, et al.: CO-RADS: a categorical CT assessment scheme for patients suspected of having COVID-19-definition and evaluation. Radiol 2020; 296: E97–E104.
- [14]↑
Chaganti S, Grenier P, Balachandran A, Chabin G, Cohen S, Flohr T, Georgescu B, et al.: Automated quantification of CT patterns associated with COVID-19 from chest CT. Radiol Artif Intell 2020; 2: e200048.
- [15]↑
Francone M, Iafrate F, Masci GM, Coco S, Cilia F, Manganaro L, Panebianco V, et al.: Chest CT score in COVID-19 patients: Correlation with disease severity and short-term prognosis. Eur Radiol 2020; 30: 6808–6817.
- [16]↑
Yuan M, Yin W, Tao Z, Tan W, Hu Y: Association of radiologic findings with mortality of patients infected with 2019 novel coronavirus in Wuhan, China. PLoS One 2020; 15: e0230548.
- [17]↑
Lieveld AWE, Azijli K, Teunissen BP, van Haaften RM, Kootte RS, van den Berk IAH, van der Horst SFB, et al.: Chest CT in COVID-19 at the ED: validation of the COVID-19 reporting and data system (CO-RADS) and CT severity score: a prospective, multicenter, observational study. Chest 2021; 159: 1126–1135.
- [18]↑
Chang Y C, Yu C J, Chang S C, Galvin J R, Liu H M, Hsiao C H, Kuo P H, et al.: Pulmonary sequelae in convalescent patients after severe acute respiratory syndrome: evaluation with thin-section CT. Radiol 2005; 236: 1067–1075.
- [19]↑
Marshall JC, Murthy S, Diaz J, Adhikari N, Angus DC, Arabi YM, Baillie K, et al.: A minimal common outcome measure set for COVID-19 clinical research. Lancet Infect Dis 2020; 20: 192–197.
- [20]↑
Nishiyama A, Kawata N, Yokota H, Sugiura T, Matsumura Y, Higashide T, Horikoshi T, et al.: A predictive factor for patients with acute respiratory distress syndrome: CT lung volumetry of the well-aerated region as an automated method. Eur J Radiol 2020; 122: 108748.
- [21]↑
Huang L, Han R, Ai T, Yu P, Kang H, Tao Q, Xia L: Serial quantitative chest ct assessment of covid-19: a deep learning approach. Radiol Cardiothorac Imaging 2020; 2: e200075.
- [22]↑
Zhang K, Liu X, Shen J, Li Z, Sang Y, Wu X, Zha Y, et al.: Clinically applicable AI system for accurate diagnosis, quantitative measurements, and prognosis of COVID-19 pneumonia using computed tomography. Cell 2020; 181: 1423–1433.E11.
- [24]↑
Bullock J, Luccioni A, Pham KH, Lam CSN, Luengo-Oroz M: Mapping the landscape of artificial intelligence applications against COVID-19. JAIR 2020; 69: 807–845.
- [25]↑
Lessmann N, Sánchez CI, Beenen L, Boulogne LH, Brink M, Calli E, Charbonnier JP, et al.: Automated assessment of COVID-19 reporting and data system and chest CT severity scores in patients suspected of having COVID-19 using artificial intelligence. Radiol 2021; 298: E18–E28.
- [26]↑
Wong HYF, Lam HYS, Fong AHT, Leung ST, Chin TWY, Lo CSY, Lui MMS, et al.: Frequency and distribution of chest radiographic findings in patients positive for COVID-19. Radiol 2020; 296: E72–E78.
- [27]↑
Malpani Dhoot N, Goenka U, Ghosh S, Jajodia S, Chand R, Majumdar S, Ramasubban S: Assigning computed tomography involvement score in COVID-19 patients: Prognosis prediction and impact on management. BJR|Open 2020; 2: 20200024.
- [28]↑
Yang R, Li X, Liu H, Zhen Y, Zhang X, Xiong Q, Luo Y, et al.: Chest ct severity score: an imaging tool for assessing severe covid-19. Radiol Cardiothorac Imaging 2020; 2: e200047.
- [29]↑
Jovel J, Greiner R: An introduction to machine learning approaches for biomedical research. Front Med 2021; 8: 771607.
- [30]↑
Alodat M: Using deep learning model for adapting and managing Covid-19 pandemic crisis. Procedia Comput Sci 2021; 184: 558–564.
- [31]↑
Yadav M, Perumal M, Srinivas M: Analysis on novel coronavirus (COVID-19) using machine learning methods. Chaos Solit Fractals 2020; 139: 110050.
- [34]↑
Abasi AK, Khader AT, Al-Betar MA, Naim S, Makhadmeh SN, Alyasseri ZAA: Link-based multi-verse optimizer for text documents clustering. Appl Soft Comput. J 2020; 87: 106002.
- [35]↑
Liu W, Wang Z, Liu X, Zeng N, Liu Y, Alsaadi FE: A survey of deep neural network architectures and their applications. Neurocomputing. 2017; 234: 11–26.
- [36]↑
Wang J, Bao Y, Wen Y, Lu H, Luo H, Xiang Y, Li X, et al.: Prior-attention residual learning for more discriminative COVID-19 screening in CT images. IEEE Trans Med Imaging 2020; 39: 2572–2583.
- [37]
Bai HX, Wang R, Xiong Z, Hsieh B, Chang K, Halsey K, My Linh Tran T, et al.: AI augmentation of radiologist performance in distinguishing COVID-19 from pneumonia of other etiology on chest CT. Radiol 2020; 296: E156–E165.
- [38]
Li L, Qin L, Xu Z, Yin Y, Wang X, Kong B, Bai J, et al.: Using artificial intelligence to detect COVID-19 and community-acquired pneumonia based on pulmonary CT: evaluation of the diagnostic accuracy. Radiol 2020; 296: E65–E71.
- [39]
Mei X, Lee HC, Diao Kyue, Huang M, Lin B, Liu C, Xie Z, et al.: Artificial intelligence–enabled rapid diagnosis of patients with COVID-19. Nat Med 2020; 26: 1224–1228.
- [40]↑
Fan DP, Zhou T, Ji GP, Zhou Y, Chen G, Fu H, Shen J, et al.: Inf-net: automatic COVID-19 lung infection segmentation from CT images. IEEE Trans Med Imaging 2020; 39: 2626–2637.
- [41]
Shan F, Gao Y, Wang J, Shi W, Shi N, Han M, Xue Z, et al.: Abnormal lung quantification in chest CT images of COVID-19 patients with deep learning and its application to severity prediction. Med Phys 2021; 48: 1633–1645.
- [42]
Zhou T, Canu S, Ruan S. Automatic COVID-19 CT segmentation using U-Net integrated spatial and channel attention mechanism. Int J Imaging Syst Technol 2021; 31: 16–27.
- [43]↑
Albawi S, Mohammed TA, Al-Zawi S: Understanding of a convolutional neural network. ICET 2017; 1–6.
- [44]↑
Wu YH, Gao SH, Mei J, Xu J, Fan DP, Zhang RG, Cheng MM: Jcs: an explainable COVID-19 diagnosis system by joint classification and segmentation. IEEE Trans Image Process 2021; 30: 3113–3126.
- [45]↑
Chabi ML, Dana O, Kennel T, Gence-Breney A, Salvator H, Ballester MC, Vasse M, et al.: Automated ai-driven ct quantification of lung disease predicts adverse outcomes in patients hospitalized for covid-19 pneumonia. Diagnostics 2021; 11: 878.
- [46]↑
Stasiow B, Sosna M, Młynarska A, Młynarski R: Severity of inflammatory lung lesions in COVID-19 based on computed tomography examination as a prognostic factor for respiratory failure and death. Polish Arch Intern Med 2021; 131: 882–884.
- [47]↑
Sezer R, Esendagli D, Erol C, Hekimoglu K: New challenges for management of COVID-19 patients: analysis of MDCT based “Automated pneumonia analysis program.” Eur J Radiol Open 2021; 8: 100370.
- [48]↑
Gashi A, Kubik-Huch RA, Chatzaraki V, Potempa A, Rauch F, Grbic S, Wiggli B, et al.: Detection and characterization of COVID-19 findings in chest CT: feasibility and applicability of an AI-based software tool. Medicine (Baltimore) 2021; 100: e27478.
- [49]↑
Okuma T, Hamamoto S, Maebayashi T, Taniguchi A, Hirakawa K: Quantitative evaluation of COVID - 19 pneumonia severity by CT pneumonia analysis algorithm using deep learning technology and blood test results. Jpn J Radiol 2021; 39: 956–965.
- [50]↑
Mergen V, Kobe A, Blüthgen C, Euler A, Flohr T, Frauenfelder T, Alkadhi H, et al.: Deep learning for automatic quantification of lung abnormalities in COVID-19 patients: first experience and correlation with clinical parameters. Eur J Radiol Open 2020; 7: 100272.
- [51]↑
Do TD, Skornitzke S, Merle U, Kittel M, Hofbaur S, Melzig C, Kauczor HU, et al.: COVID-19 pneumonia: prediction of patient outcome by CT-based quantitative lung parenchyma analysis combined with laboratory parameters. PLoS One 2022; 17: e0271787.
- [52]↑
Szabó IV, Simon J, Nardocci C, Kardos AS, Nagy N, Abdelrahman RH, Zsarnóczay E, et al.: The predictive role of artificial intelligence-based chest CT quantification in patients with COVID-19 pneumonia. Tomography 2021; 7: 697–710.
- [53]↑
Gozes O, Ayan Frid-Adar M, Greenspan H, Browning PD, Zhang H, Ji W, Bernheim A, et al.: Rapid AI development cycle for the coronavirus (COVID-19) pandemic: initial results for automated detection & patient monitoring using deep learning CT image analysis authors. Arxiv 2020; abs/2003.05037.
- [54]↑
Guo Y, Liu Y, Oerlemans A, Lao S, Wu S, Lew MS: Deep learning for visual understanding: a review. Neurocomputing 2016; 187: 27–48.
- [55]↑
Roy PK, Singh JP: Predicting closed questions on community question answering sites using convolutional neural network. Neural Comput Appl 2020; 32(14): 10555–10572.
- [56]↑
Anthimopoulos M, Christodoulidis S, Ebner L, Christe A, Mougiakakou S: Lung pattern classification for interstitial lung diseases using a deep convolutional neural network. IEEE Trans Med Imaging 2016; 35: 1207–1216.
- [57]↑
Badrinarayanan V, Kendall A, Cipolla R: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 2017; 39: 2481–2495.
- [58]↑
Çiçek Ö, Abdulkadir A, Lienkamp S S, Brox T, Ronneberger O: 3D U-net: learning dense volumetric segmentation from sparse annotation. In: Medical image computing and computer-assisted intervention – MICCAI 2016. MICCAI 2016 (eds: Ourselin S, Joskowicz L, Sabuncu M, Unal G, Wells W). Lecture notes in computer science. Springer, Cham, 2016; 9901, pp. 424–432.
- [59]↑
Cao Y, Xu Z, Feng J, Jin C, Han X, Wu H, Shi H: Longitudinal assessment of covid-19 using a deep learning–based quantitative ct pipeline: illustration of two cases. Radiology: Cardiothoracic Imaging 2020; 2: e200082.
- [60]↑
Yue H, Yu Q, Liu C, Huang Y, Jiang Z, Shao C, Zhang H, et al.: Machine learning-based CT radiomics method for predicting hospital stay in patients with pneumonia associated with SARS-CoV-2 infection: A multicenter study. Ann Transl Med 2020; 8: 859.
- [61]↑
Shan F, Gao Y, Wang J, Shi W, Shi N, Han M, Xue Z, et al.: Lung infection quantification of COVID-19 in CT images with deep learning. Arxiv 2020; abs/2003.04655.
- [62]↑
Ronneberger O, Fischer P, Brox T: U-Net: Convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention – MICCAI 2015. MICCAI 2015 (eds: Navab N, Hornegger J, Wells W, Frangi A). Lecture notes in computer science. Springer, Cham, 2015; 9351, pp. 234–241.
- [63]↑
Reyes-Figueroa A, Flores VH, Rivera M: Deep neural network for fringe pattern filtering and normalization. Appl Opt 2021; 60: 2022–2036.
- [64]↑
Mahmoudi R, Benameur N, Mabrouk R, Mohammed MA, Garcia-Zapirain B, Bedoui MH: A deep learning-based diagnosis system for COVID-19 detection and pneumonia screening using CT imaging. Appl Sci 2022; 12: 4825.
- [65]↑
Grodecki K, Lin A, Cadet S, McElhinney P A, Razipour A, Chan C, Pressman B, et al.: Quantitative burden of covid-19 pneumonia at chest ct predicts adverse outcomes: a post hoc analysis of a prospective international registry. Radiol Cardiothorac Imaging 2020; 2: e200389.
- [66]↑
Yu Q, Wang Y, Huang S, Liu S, Zhou Z, Zhang S, Zhao Z, et al.: Multicenter cohort study demonstrates more consolidation in upper lungs on initial CT increases the risk of adverse clinical outcome in COVID-19 patients. Theranostics 2020; 10: 5641–5648.
- [67]↑
Enshaei N, Afshar P, Heidarian S, Mohammadi A, Rafiee MJ, Oikonomou A, Fard FB, et al.: An ensemble learning framework for multi-class Covid-19 lesion segmentation from chest CT images. IEEE ICAS 2021.
- [68]↑
Garcia-Garcia A, Orts-Escolano S, Oprea S, Villena-Martinez V, Garcia-Rodriguez J: A review on deep learning techniques applied to semantic segmentation. Arxiv 2017; abs/1704.06857.
- [69]↑
Wang G, Liu X, Li C, Xu Z, Ruan J, Zhu H, Meng T, et al.: A noise-robust framework for automatic segmentation of COVID-19 pneumonia lesions from CT images. IEEE Trans Med Imaging 2020; 39: 2653–2663.
- [70]↑
Müller D, Soto Rey I, Kramer F: Automated chest CT image segmentation of COVID-19 lung infection based on 3D U-net. Arxiv 2020; abs/2007.04774.
- [71]↑
Yan Q, Wang B, Gong D, Luo C, Zhao W, Shen J, Shi Q, et al.: COVID-19 chest CT image segmentation - a deep convolutional neural network solution. Arxiv 2020; abs/2004.10987.
- [72]↑
Gerard S E, Herrmann J, Xin Y, Martin K T, Rezoagli E, Ippolito D, Bellani G, et al.: CT image segmentation for inflamed and fibrotic lungs using a multi-resolution convolutional neural network. Sci Rep 2021; 11: 1455.
- [73]↑
Qiu Y, Liu Y, Li S, Xu J: MiniSeg: an extremely minimum network for efficient COVID-19 segmentation. IEEE Trans Neural Netw Learn Syst 2022; 1–15.
- [74]↑
Pei HY, Yang D, Liu GR, Lu T: MPS-net: multi-point supervised network for ct image segmentation of covid-19. IEEE 2021; 9: 47144–47153.
- [75]↑
Enshaei N, Oikonomou A, Rafiee MJ, Afshar P, Heidarian S, Mohammadi A, Plataniotis KN: Covid - rate : An automated framework for segmentation of COVID - 19 lesions from chest CT images. Sci Rep 2022; 12: 3212.
- [76]↑
Bartoli A, Fournel J, Maurin A, Marchi B, Habert P, Castelli M, Gaubert J-Y, et al.: Value and prognostic impact of a deep learning segmentation model of COVID-19 lung lesions on low-dose chest CT. Res Diagnostic Interv Imaging 2022; 1: 100003.
- [77]↑
Yoo SJ, Qi X, Inui S, Kim H, Jeong YJ, Lee KH, Lee YK, et al.: Deep learning-based automatic CT quantification of coronavirus disease 2019 pneumonia: an international collaborative study. J Comput Assist Tomogr 2022; 46: 413–422.
- [78]↑
Chen J, Wu L, Zhang J, Zhang L, Gong D, Zhao Y, Chen Q, et al.: Deep learning-based model for detecting 2019 novel coronavirus pneumonia on high-resolution computed tomography. Sci Rep 2020; 10: 19196.
- [79]↑
Zhou Z, Rahman Siddiquee MM, Tajbakhsh N, Liang J: UNet++: a nested U-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support (eds: Stoyanov D, et al.) DLMIA ML-CDS 2018. LNCS, 2018; 11045, pp. 3–11.
- [80]↑
Zhang Z, Liu Q, Wang Y: Road extraction by deep residual U-net. IEEE Geosci Remote Sens Lett 2018; 15: 749–753.
- [81]↑
Milletari F, Navab N, Ahmadi SA: V-net: fully convolutional neural networks for volumetric medical image segmentation. Arxiv 2016.
- [82]↑
Zhang H, Zhang J, Zhang H, Nan Y, Zhao Y, Fu E: Automated detection and quantification of COVID-19 pneumonia : CT imaging analysis by a deep learning-based software. Eur J Nucl Med Mol Imaging 2020; 47: 2525–2532.
- [83]↑
Li J, Zuo W, Lei Q, Jiang G, Jianhao Y, Long L: CT-based quantitative assessment of coronavirus disease 2019 using a deep learning-based segmentation system: a longitudinal study. Radiol Cardiothorac Imaging 2021.
- [84]↑
Pan F, Li L, Liu B, Ye T, Li L, Liu D, Ding Z, et al.: A novel deep learning-based quantification of serial chest computed tomography in Coronavirus Disease 2019 (COVID-19). Sci Rep 2021; 11: 417.
- [85]↑
Pu J, Leader JK, Bandos A, Ke S, Wang J, Shi J, Du P, et al.: Automated quantification of COVID-19 severity and progression using chest CT images. Eur Radiol 2021; 31: 436–446.
- [86]↑
Czempiel T, Rogers C, Keicher M, Paschali M, Braren R, Burian E, Makowski M, et al.: Longitudinal self-supervision for COVID-19 pathology quantification. Arxiv 2022; abs/2203.10804.
- [88]↑
Huang G, Liu Z, van der Maaten L, Weinberger KQ: Densely connected convolutional networks. IEEE CVPR 2017; 2261–2269.
- [89]↑
Shi X, Chen Z, Wang H, Yeung DY, Wong WK, Woo WC: Convolutional LSTM network: a machine learning approach for precipitation nowcasting. NIPS 2015; 802–810.
- [90]↑
Grodecki K, Killekar A, Lin A, Cadet S, McElhinney P, Razipour A, Chan C, et al.: Rapid quantification of COVID-19 pneumonia burden from computed tomography with convolutional LSTM networks. J Med Imaging (Bellingham) 2022; 9: 054001.
- [91]↑
Chrzan R, Bociaga-Jasik˛ M, Bryll A, Grochowska A, Popiela T: Differences among COVID-19, bronchopneumonia and atypical pneumonia in chest high resolution computed tomography assessed by artificial intelligence technology. J Pers Med 2020; 11: 391.
- [92]↑
Chrzan R, Wojciechowska W, Terlecki M, Klocek M, Rajzer M, Popiela T. The role of artificial intelligence technology analysis of HRCT images in predicting the severity of COVID-19 pneumonia. Polish Arch. Intern. Med. 2022; 132: 16332.
- [93]↑
Pang B, Li H, Liu Q, Wu P, Xia T, Zhang X, Le W, et al.: CT quantification of COVID-19 pneumonia at admission can predict progression to critical illness: a retrospective multicenter cohort study. Front Med 2021; 8: 689568.
- [94]↑
Ding X, Xu J, Xu H, Zhou Y, Long Q: Risk assessment using early quantitative chest CT parameters for the severity of COVID-19. Iran J Radiol 2020; 18(3): e109439.
- [95]↑
Bankier AA, Madani A, Gevenois PA: CT quantification of pulmonary emphysema: assessment of lung structure and function. Crit Rev Comput Tomogr 2002; 43: 399–417.
- [96]
Alihodzic A, Tuba M: Improved bat algorithm applied to multilevel image thresholding. Sci World J 2014; 2014:176718.
- [97]
Ohkubo H, Nakagawa H, Niimi A: Computer-based quantitative computed tomography image analysis in idiopathic pulmonary fibrosis: a mini review. Respir Investig 2018. 56: 5–13.
- [98]↑
Grassi R, Paola M, Alessandro B, Gianluigi M, Fabrizio P, Giacobbe G, Fusco R, et al.: Covid - 19 pneumonia: Computer - aided quantification of healthy lung parenchyma, emphysema, ground glass and consolidation on chest computed tomography (CT). Radiol Med 2021; 126: 553–560.
- [99]↑
Grassi R, Cappabianca S, Urraro F, Granata V, Giacobbe G, Magliocchetti S, Cozzi D, et al.: Evolution of CT findings and lung residue in patients with COVID-19 pneumonia : quantitative analysis of the disease with a computer automatic tool. J Pers Med 2021; 11: 641.
- [100]↑
Zhou S, Zhu T, Wang Y, Xia LM: Imaging features and evolution on CT in 100 COVID-19 pneumonia patients in Wuhan, China. Eur Radiol 2020; 30: 5446–5454.
- [101]↑
Caruso D, Guido G, Zerunian M, Polidori T, Lucertini E, Pucciarelli F, Polici M, et al.: Post-acute sequelae of COVID-19 pneumonia: six-month chest CT follow-up. Radiol 2021; 301: E396–E405.
- [102]↑
Han X, Fan Y, Alwalid O, Li N, Jia X, Yuan M, Li Y, et al.: Six-month follow-up chest CT findings after severe COVID-19 pneumonia. Radiology 2021; 299(1): E177–E186.
- [103]↑
Yousef H A, Mansour E, Moussa M, Zidan M, Abdel M, Mohamed M, Ahmed S, et al.: Automated quantification of COVID - 19 pneumonia severity in chest CT using histogram - based multi - level thresholding segmentation. EJRNM 2021; 52: 293.
- [104]↑
Lanza E, Muglia R, Bolengo I, Santonocito O G, Lisi C, Angelotti G, Morandini P, et al.: Quantitative chest CT analysis in COVID-19 to predict the need for oxygenation support and intubation. Eur Radiol 2020; 30: 6770–6778.
- [105]↑
Salvatore C, Roberta F, Angela deL, Cesare P, Alfredo C, Giuliano G, Giulio L, et al.: Clinical and laboratory data, radiological structured report findings and quantitative evaluation of lung involvement on baseline chest CT in COVID-19 patients to predict prognosis. Radiol Medica 2021; 126: 29–39.
- [106]↑
Fervers P, Fervers F, Jaiswal A, Rinneburger M, Weisthoff M, Pollmann-schweckhorst P, Kottlors J, et al.: Assessment of COVID-19 lung involvement on computed tomography by deep-learning- , threshold- , and human reader- based approaches — an international , multi-center comparative study. Quant Imaging Med Surg 2022; 97: 5156–5170.
- [107]↑
Lu W, Wei J, Xu T, Ding M, Li X, He M, Chen K, et al.: Quantitative CT for detecting COVID - 19 pneumonia in suspected cases. BMC Infect Dis 2021; 21: 836.
- [108]↑
Khan A, Garner R, Rocca MLa, Salehi S, Duncan D: A novel threshold-based segmentation method for quantification of COVID-19 lung abnormalities. Signal Image Video Process 2022; 17: 907–914.
- [109]↑
Zheng C, Deng X, Fu Q, Zhou Q, Feng J, Ma H, Liu W, et al.: Deep learning-based detection for COVID-19 from chest CT using weak label. IEEE Trans Med Imaging 2020: 20027185.
- [110]↑
Rubin GD, Ryerson CJ, Haramati LB, Sverzellati N, Kanne JP, Raoof S, Schluger NW, et al.: The role of chest imaging in patient management during the covid-19 pandemic: a multinational consensus statement from the fleischner society. Radiol 2020; 296: 172–180.
- [111]↑
Zhu J, Zhong Z, Li H, Ji P, Pang J, Li B, Zhang J: CT imaging features of 4121 patients with COVID-19: a meta-analysis. J Med Virol 2020; 92(7): 891–902.
- [112]↑
Chen HJ, Mao L, Chen Y, Yuan L, Wang F, Li X, Cai Q, et al.: Machine learning-based CT radiomics model distinguishes COVID-19 from non-COVID-19 pneumonia. BMC Infect Dis 2021; 21: 931.
- [113]↑
Guan W, Ni Z, Hu Y, Liang W, Ou C, He J, Liu L, et al.: Clinical characteristics of coronavirus disease 2019 in China. N Engl J Med 2020; 382: 1708–1720.