Introduction

Proper patient positioning is crucial for diagnostic quality of radiographs [1] and relevant quality criteria and recommendations have been formulated in official standards [2]. The European guidelines on quality criteria for diagnostic radiographic images list three image criteria concerning patient positioning [3]: (1) Performed at full inspiration (as assessed by the position of the ribs above the diaphragm — either 6 anteriorly or 10 posteriorly) (2) Symmetrical reproduction of the thorax as shown by central position of the spinous process between the medial ends of the clavicles (3) Reproduction of the whole rib cage above the diaphragm. Furthermore, the ALARA (“as low as reasonably achievable”) principle states that unnecessary X-ray exposure should be avoided, which requires collimation always to be balanced between completeness and dose considerations.

In daily routine, there are many factors causing deviations from these standards, such as high workload, lack of training, missing education, missing feed-back, and lack of rewards. A study found approx. 15%, approx. 5%, and approx. 50% of 116 images were violating the above three criteria when applied strictly [4]. Such images may cause extra workload for the reading radiologist, mean potential harm to the patient through delayed or missed diagnosis and avoidable exposure by too wide collimation or even retakes. Typically, a radiology department board occasionally discusses the quality of only a limited sample of images in order to maintain or improve certain quality levels [5, 6]. Quality monitoring based on well reproducible measurements [7] applied to every acquired image would be preferable, but it requires an automated robust method to evaluate adherence to the most relevant quality criteria [8]. Any automation on quality control that tries to reproduce expert performance is challenged by the fact that different experts may apply different criteria and use different thresholds on them to judge a given image. Some studies report a moderate agreement (Cohen's Kappa $0.4<\hspace{0.17em}\kappa \le 0.6$) between experts [9], while others report a fair agreement ($0.2<\hspace{0.17em}\kappa \le 0.4$) only [4, 10]. Also, the differences between radiologists from different countries have been addressed [11].

There are two approaches reported recently on automatic evaluation of chest radiography positioning quality. In the first approach, a number of training images were classified by an expert to be either insufficient (too narrow), appropriate, or excessive (too wide) in each single edge of the image [12, 13]. Further on, inspiration and rotation were both rated appropriate or not among some further criteria. A classification neural network was trained end-to-end to reproduce these ratings directly from the image. In contrast, the second approach used machine learning to derive quality metrics based on comprehensible measurements like estimating the number of ribs above the diaphragm or measuring specific distances in the image like those between a clavicle and the spinous processes or the abdominal space below the lung base being exposed to radiation [14, 15]. Some thresholds may be applied on these quality metrics to finally come to a classification (e.g. ok or unacceptable). That has been done in a prospective study giving immediate feedback to the radiographers whether one of the three quality aspects leads to imprecise positioning [16]. The thresholds used for the AI-method in that study were listed but their choice was not motivated nor have the consequences of this choice been discussed. This second approach is followed here because it appears more comprehensible and configurable than end-to-end learning. For quality classification consistent with experts however it is based on the assumption that for each of the three quality aspects thresholds can be established that separate images considered ok from images considered unacceptable based on these metrics only. Here, objective thresholds are to be established that can be applied to the three quality metrics in an automated context. To that end, a multi-site multi-reader study was performed that aims at testing the hypothesis that (i) the automatically computable quality metrics correspond to expert's quality impressions of the image, (ii) empirical quality classes are separable using these quality metrics, and (iii) variability among experts' assessments does not counteract using such an automatic method.

Materials and methods

All adult patients' CXRs imaged in an upright and posterioanterior orientation were selected from the publicly available chest X-ray8 data base [17] from the National Institute of Health giving 64,315 such images. These images were processed by an AI-method capable of automated quantification of six quality metrics aimed at assessing the quality of collimation, rotation and inhalation through anatomical feature detection [14]. The collimation metrics ($\Delta N,\Delta S,\Delta E,\Delta W$) measure the distance for each image border to the bounding box surrounding landmarks on the lung borders (Fig. 1a). A wide collimation is indicated by a positive value and an intersection between the bounding box and the collimation is indicated by a negative value. For the rotation metric based on the distances between both clavicle heads to the spinous process line (Fig. 1b) a rotation asymmetry metric $\alpha$ is calculated from $\alpha =\frac{d_R-d_L}{d_R+d_L}$. This value is bounded to [−1, 1] and quantifies deviations from the desired balanced position at $\alpha =0$. For the metrics on inhalation status the AI-method uses an anatomical atlas matching the rib cage and approximating the amount of posterior ribs visible above both diaphragms as $c_L$ and $c_R$ (Fig. 1c). These numbers may also contain fractions of a rib. In the original approach $c$ is used as the sum of both, here $c_{\mathit{max}}$ the highest number of both sides is used, as pathology or anatomical variance has turned out to complicate the assessment of inhalation status on one side.

View Full Size

Given the localization of the anatomical landmarks, the quality metrics $\Delta N$, $\Delta S$, $\Delta W$ and $\Delta E$, $\alpha$ are geometrically derived and rib counts $c_L$ and $c_R$ are estimated

Citation: Imaging 16, 1; 10.1556/1647.2024.00187

• Download Figure
• Download figure as PowerPoint slide

From the processed dataset a sampling of 920 images was done. 600 images were specifically sampled for the collimation test in total, 150 for each of the four image borders. 200 additional images were sampled for the rotation test and 120 for the inhalation status test. The sampling was random apart from the one parameter relevant for each test, the metric with the aim to oversample images with interesting such values. This means the share of cases with values close to an anticipated threshold should be higher than in the entire dataset but the whole parameter range should still be covered. Figure 2 shows the result of this sampling strategy, the distribution of all quality metrics calculated atomatically on the entire data set, and the distribution of the trusted reference quality metrics on each of the corresponding samples.

View Full Size

Distribution of quality metrics ($\Delta N$, $\Delta S$, $\Delta W,\Delta E$, α, c, c_max) in the entire data set (64,315 cases, brown line) and in the six different samples of 150, 150, 150, 150, 200, and 120 cases to be used in the corresponding study tasks (blue). For the inhalation sample the two different metrics c and c_max are both analyzed separately

Citation: Imaging 16, 1; 10.1556/1647.2024.00187

• Download Figure
• Download figure as PowerPoint slide

To exclude potential errors in the anatomical feature detection by the AI-method, landmark placement for the inhalation metric (lung borders), rotation metric (medial clavicle heads and spinous processes) or rib counts above the diaphragms were done manually for all 920 cases by some of the authors. In this case unlike in the AI-method rib counts were set to be integer values for practical reasons. To gain further insights on the quality of the automated assessment on this dataset this manually annotated reference standard was compared to the AI results. The absolute differences between trusted and automatic metrics are on average (median) deviations of $\Delta N$: 3.2 mm (2.7 mm), $\Delta S$: 7.3 mm (6.1 mm), $\Delta W$/$\Delta E$: 3.9 mm (3.0 mm), $\alpha$: 0.11 (0.07), rounded $c_{\mathit{max}}$: 0.51 ribs (0 ribs). Figure 3 shows a comparison of these reference quality metrics with those provided by the AI-method automatically.

View Full Size

Comparison of reference quality metrics calculated from manually localized anatomical landmarks (x-axis) and those estimated by the AI-method (y-axis) on the corresponding data set

Citation: Imaging 16, 1; 10.1556/1647.2024.00187

• Download Figure
• Download figure as PowerPoint slide

The seven readers were three radiologists from a university medical center in the Netherlands, two residents in radiology from a university hospital in Germany and two experienced radiographers certified to read CXR images (reading radiographers) from a regional supply hospital in Denmark. The raters were instructed on the acquisition quality criteria following the European guidelines [3] and had access to guidance documents at any time. The images were presented to the readers in randomized batches, in which the collimation, rotation and inhalation quality were separately scored as being ‘OK’, ‘inadequate’ or ‘unacceptable’. For the collimation per image border additional options ‘too narrow’ and ‘too wide’ were given. A ROC analysis was done for each of the metrics on how it separates the images for these different scores. Optimal thresholds were determined to maximize the accuracy of a classifier separating ‘OK’ from ‘unacceptable’ cases on votes of subsets of the raters and finally on the votes of all seven raters to represent a kind of ‘super reader’.

As an extension, these thresholds were applied to classify both the six samples separately as well as the entire data set into ‘OK’ and ‘unacceptable’ fully automatically based on the metrics of the AI-method for all the quality criteria.

Results

Ratings related to reference quality metrics

Figure 4 shows the distribution of expert quality classes on the corresponding reference quality metrics. The classes appear generally ordered by the reference quality metrics, some of them overlap a little. The responses of the different raters appear generally similar to each other, sometimes there is a little shift between their responses that means some tend to apply stricter criteria than others. Most strikingly, two raters (R4 and R6) did not choose ‘too wide, unacceptable’ at any lateral Field-of-View case at all.

View Full Size

Distribution of raters' quality classes plotted on the axis of the corresponding reference quality metric that was geometrically derived from the manual landmark localizations. Collimation ($\Delta N$, $\Delta S$, $\Delta W$ and $\Delta E$) – from left to right: too wide – unacceptable (red), too wide – inadequate (grey), OK (green), too narrow – inadequate (grey), too narrow – unacceptable (red). Rotation (α) and Inhalation (c): unacceptable (red), inadequate (grey), OK (green). Thresholds are indicated that minimize the number of misclassified rates (numbers also shown close to the threshold line). All seven raters are analyzed separately and in the bottom line their pooled rates are used to derive common thresholds

Citation: Imaging 16, 1; 10.1556/1647.2024.00187

• Download Figure
• Download figure as PowerPoint slide

Discussion

In order to establish an automated quality assessment on patient positioning for posteroanterior CXR images a method is pursued that applies metrics for the three most relevant quality issues concerning collimation, patient rotation, and inhalation. International guidelines already introduce some formal criteria to do such quality assessment, but typically not providing exact numbers for decision making. An AI-method was adapted that establishes and implements some formal metrics for all these quality criteria. It calculates these metrics geometrically from anatomical landmarks that it localizes in the image. This way the method estimates the distances of the rib cage to the image borders, balances the distance of the clavicles to the spinous processes, it estimates and interpolates the number of ribs above the diaphragm. An adherence to guidelines is given to this method by design. Unlike other known approaches the method is comprehensible because it does not only provide a binary decision but also all decision criteria. It may also be configured and parameterized to specific requirements by adapting the thresholds. What was missing so far is to prove whether such an automated method also corresponds to experts' assessment of positioning quality, especially for interesting borderline cases. Also, thresholds on these metrics were missing that optimally reproduce the experts' ability to classify images into okay and unacceptable.

A careful approach was followed that tries to separate the metrics themselves from the implemented AI-method by introducing additional trusted reference quality metrics. They are also calculated geometrically from the landmarks, but based on landmark positions that are manually determined by some of the authors in the images, so basically in a pen and paper act without any machine learning involved. The inhalation reference quality metrics was determined by counting ribs. The study cases were carefully sampled with interesting borderline cases being over-represented, almost a thousand cases in total.

The main result of the study is that when sorting the study cases based upon their reference quality metrics this separates the quality classes as given by the experts, even when all their ratings are pooled. Especially for the strong binary decision on ‘Ok’ versus ‘Unacceptable’ there was actually not much overlap of these two classes. Thresholds could be established to do such an optimal separation by each of the quality metrics based on the empirical data of all raters. Two of the seven raters did not choose ‘too wide, unacceptable’ at any lateral Field-of-View case at all and explained that later by the fact that they do not see a strong negative effect in extending the radiated area into the empty space to the patient side. Considering that, both quality metrics with their corresponding thresholds ${\Delta }_{W,w}$ and ${\Delta }_{E,w}$ appear not as relevant as the other ones.

A limitation of the study design lies in the annotation of rib counts by integer values. It restricts possible reasonable thresholds on $c_{\mathit{max}}$ for Inhalation practically to be set either between eight and nine or between nine and ten ribs. The latter resulted from the raters' votes, very much in line with the guideline. A threshold of 9.5 ribs resulted in a rejection rate of 36% on the entire data set. The study results would justify any threshold between nine and ten ribs and using 9.0 ribs reduces the rejection rate to 24.8%.

When using the AI-method as a reader it showed high AUC values for classifying all study cases rated ok or unacceptable. Its performance is similar to what could be achieved when applying the determined common thresholds on the trusted reference metrics to reproduce all pooled expert rates ok or unacceptable. When comparing these values it should be considered that the particular impacting factor for the expert group is disagreement among each other while for the AI-method it is the inaccuracies in landmark localizations with respect to those of the trusted reference landmarks.

When applying the empiric thresholds based on guidelines and expert opinions to the entire data set of 64,315 cases, this gives percentages of cases that fail the corresponding quality criteria. For the too narrow Field-of-View this is 6.6%, for the too wide Field-of-View it is 61.1%. For Rotation it is 4.1%, and 24.8% for Inhalation. These numbers fit well into the variety found in earlier studies. Also, in our large data set the majority of diagnosed cases fail at least one quality criterion. This shows a high discrepancy between expectations and real practice in positioning quality with implications on dose, potentially missed diagnosis, and additional workload. That turned into a positive statement, we see a big potential for improvements in radiology departments that may be addressed by automated quality feedback and control on an every acquisition basis. Further studies are planned to prove practical appropriateness of the AI-method with the given thresholds in a real radiology setting on the everyday image data at different institutions.

Conclusion

Thresholds on quality metrics were determined in a reader study that allow reproducing expert classifications between ok and unacceptable for three relevant patient positioning criteria in chest radiography: collimation, patient rotation, and inhalation. An AI-method was introduced and validated that is able to estimate these metrics from the image according to the guidelines. Applying the threshold to these metrics achieves automatic but comprehensible quality classification for quality management that is consistent with expert judgements.

Authors' contribution

J.v.B., K.H., N.W., T.H., S.Y., H.L. contributed to the conception and design of the work, J.v.B., K.H., M.E., D.P., N.W., S. K.-H., T.H., S.Y., H.L. contributed to data collection, J.v.B., K.H., N.W., S.K.-H., T.H., A.G., D.B., M.B., S.Y., H.L. contributed to data analysis, while all authors contributed to manuscript writing. All authors reviewed the final version of the manuscript and agreed to submit it to IMAGING for publication.

Conflict of interest

J.v.B, N.W, S.K.-H, T.H, A.G, D.B, M.B, and S.Y are all employees of Philips.

Funding sources

D.P. was funded by the Federal Ministry of Education and Research (BMBF) and the Free State of Bavaria under the Excellence Strategy of the Federal Government and the Länder, the German Research Foundation (GRK2274), as well as by the Technical University of Munich–Institute for Advanced Study.

Ethical statement

The study has been conducted in accordance with the Declaration of Helsinki and according to requirements of all applicable local and international standards.