Uncertainty-aware 轻量级 CNN 肺部 CT 切片分类研究
Uncertainty-aware lung CT slice classification using a lightweight CNN with selective referral
该研究提出了一种结合轻量级 CNN 和多种不确定性估计策略的肺部 CT 切片分类框架。使用 IQ-OTH/NCCD 数据集,通过 Monte Carlo Dropout、深度集成和集成 Monte Carlo Dropout 评估不确定性估计方法,计算置信度、归一化预测熵等指标,定义了选择性接受或转介规则。研究结果显示,该方法在分类性能、校准分析和风险覆盖方面表现良好。
Introduction
Lung cancer remains a major cause of cancer-related mortality worldwide, and CT imaging provides detailed cross-sectional information for assessing pulmonary abnormalities. Deep learning, particularly convolutional neural networks (CNNs), has shown strong performance in medical image classification; however, classification accuracy alone does not indicate whether an individual prediction is reliable. Conventional deterministic classifiers typically return a label for every input, even when the image is ambiguous, degraded, or different from the training distribution. In medical-image decision support, this limitation motivates explicit uncertainty estimation and selective referral rather than forced prediction. This study uses the public IQ-OTH/NCCD dataset of benign, malignant, and normal CT slices1. Monte Carlo (MC) Dropout provides an efficient approximate Bayesian approach by retaining dropout during inference and sampling multiple stochastic subnetworks2, while predictive entropy, mutual information, and variation ratio summarize complementary aspects of predictive uncertainty3,4. Calibration is also important because neural-network confidence can be misaligned with observed correctness5, particularly under dataset shift6. Selective classification allows uncertain predictions to be referred rather than automatically accepted7,8, and Grad-CAM provides qualitative visualization of image regions contributing to a CNN prediction9. Accordingly, the present work evaluates an uncertainty-aware CT-slice classification framework that combines a lightweight CNN with multiple uncertainty estimators, validation-selected selective referral, calibration analysis, controlled stress testing, and qualitative Grad-CAM visualization. The study is framed as image-level decision-support research rather than patient-level diagnostic validation.
The contributions of the work can be summarized in the following manner:
-
Exact duplicate removal was performed before training/validation/test splitting to reduce the risk of exact-image data leakage.
-
A compact task-specific CNN with 6,664,387 trainable parameters was designed for three-class classification of lung CT slices.
-
Monte Carlo Dropout with 50 stochastic forward passes, Deep Ensemble with three independently trained members (seeds 42, 142, and 242), and Ensemble MC Dropout with 3 × 50 stochastic predictions were evaluated as uncertainty-estimation strategies.
-
Confidence, normalized predictive entropy, normalized mutual information, and variation ratio were calculated to characterize predictive reliability.
-
A reproducible selective accept-or-refer rule was defined using the scalar score U = max(1 − Confidence, Normalized predictive entropy, Normalized mutual information, Variation ratio), with the operating threshold selected from validation predictions only.
-
Evaluation included class-wise sensitivity/specificity with exact 95% confidence intervals, ROC/precision-recall analysis, image-level five-fold cross-validation, 10-bin equal-width calibration with Brier score and negative log-likelihood, risk-coverage analysis, expanded same-split baselines, simulated domain-shift stress tests, and a separately retrained second-dataset benchmark.
-
Grad-CAM was used for qualitative analysis of model attention; the generated heatmaps are not interpreted as validated tumor-localization maps.
Related work
CNN-based lung CT classification has been investigated using custom architectures, transfer learning, hybrid feature extraction, recurrent integration, attention, and transformers. Recent uncertainty-quantification research in medical imaging includes probabilistic and non-probabilistic methods10,11,12,13, while calibration is essential for interpreting predictive confidence14. Reporting guidance such as the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) emphasizes transparent evaluation of AI systems in medical imaging15, and large-scale CT screening work has demonstrated the broader potential of deep learning in lung-cancer assessment16. On IQ-OTH/NCCD, recent studies include Lung-EffNet17, a VGG16/ResNet50/InceptionV3 ensemble with Grad-CAM18, a compact CNN with patient-level splitting19, VGG16 with self-attention20, modified EfficientNet-B0 with Grad-CAM21, CNN-GRU22, a CNN-transformer approach23, and ResNet + with attention-based modifications24. These studies report strong classification performance but differ in dataset versions, splitting units, augmentation, model-selection procedures, and patient-level controls, limiting direct numerical comparison. Moreover, many approaches provide a forced class prediction without an explicit selective-referral mechanism. The present work therefore focuses on the complementary question of reliability at the CT-slice level by comparing MC Dropout, Deep Ensemble, and Ensemble MC Dropout and by reporting calibration, coverage, referral rate, and selective risk under a validation-selected decision rule (Table 1).
Dataset and preprocessing
IQ-OTH/NCCD dataset
The IQ-OTH/NCCD dataset was obtained from the Iraq-Oncology Teaching Hospital and the National Center for Cancer Diseases. According to the public Mendeley citation record, the dataset contains 1,190 CT slices from 110 cases: 15 benign, 40 malignant, and 55 normal cases1. The images were examined by oncologists and radiologists and were acquired using a Siemens SOMATOM scanner. The 1,097-image working copy used in this study was obtained from the IQ-OTH/NCCD Kaggle distribution (https://www.kaggle.com/datasets/hamdallak/the-iqothnccd-lung-cancer-dataset; and contained 120 benign, 561 malignant, and 416 normal image files. This discrepancy from the public record is reported explicitly to distinguish the nominal dataset size from the computational files available for the present experiment. Before any data splitting, exact content-based duplicate filtering removed 43 files (18 benign, 14 malignant, and 11 normal), leaving 1,054 unique CT slices: 102 benign, 547 malignant, and 405 normal. Thus, the benign class represented 9.68% of the curated dataset and was the minority class. Figure 1 shows representative CT slices from the three classes, Fig. 2 shows the post-deduplication class distribution, and Table 2 summarizes the dataset reconciliation and duplicate-removal process.
Representative CT images from the benign, malignant, and normal classes. Images are real samples from the working dataset.
Class distribution after removal of 43 exact duplicate images.
Dataset division
After removal of 43 exact duplicates, the duplicate-controlled dataset contained 1,054 CT slices. The principal experiment used a fixed random seed of 42 and an image-level stratified split of 842 training images (82 benign, 437 malignant, 323 normal), 106 validation images (10 benign, 55 malignant, 41 normal), and 106 test images (10 benign, 55 malignant, 41 normal). The test split was not used for model fitting or for selection of the selective-referral operating threshold. A patient-identifier audit of the public working copy found only class folders and sequential image filenames: 1,054/1,054 images lacked a recoverable reliable patient identifier. Consequently, patient-wise grouping could not be performed. Exact duplicate removal prevents exact-file overlap but does not prevent correlated slices from the same case from entering different image-level partitions. Accordingly, the test set is described throughout as a held-out image-level test split, not as an independent patient-level test set, and all performance estimates are interpreted as internal proof-of-concept results (Table 3).
Image preprocessing and data augmentation
All CT images were decoded in three-channel format, resized to 224 × 224 pixels, and cast to float32 while retaining the [0,255] intensity scale. Input normalization to [0,1] was performed internally by the lightweight CNN through its Rescaling(1/255) layer. Class labels were encoded as benign, malignant, and normal. Data augmentation was applied only to the training subset and included small rotations, translations, zoom, horizontal flips, and brightness changes. Validation and test images were not randomly augmented, and no test-time augmentation was used. Preprocessing was performed after global exact-duplicate removal and creation of the fixed training, validation, and test indices, which were preserved for reproducibility.
Dataset curation and preprocessing.
Proposed methodology
The proposed framework is an uncertainty-aware decision-support pipeline for three-class CT-slice classification. Compared with a conventional CNN that returns only a class probability vector, the framework adds a reliability pathway that estimates predictive uncertainty and determines whether a prediction should be automatically accepted or referred for expert review. As illustrated in Fig. 3, each preprocessed CT slice first passes through the lightweight CNN to obtain Softmax probabilities for the benign, malignant, and normal classes. During uncertainty-aware inference, dropout remains active and the same slice is evaluated in 50 stochastic forward passes using MC Dropout2. The resulting probability vectors are averaged to obtain the mean predictive distribution, while confidence, predictive entropy, mutual information, and variation ratio are computed from the stochastic predictions3,4. These quantities are combined by the selective decision module to determine whether the prediction satisfies the validation-selected reliability criterion7,8. Grad-CAM is then used to visualize image regions that contributed to the predicted class9. The Grad-CAM maps are qualitative model-attention visualizations and are not validated tumor-localization maps.
Overall workflow of the proposed explainable uncertainty-aware CNN framework.
There are two paths in the workflow, namely the classification path and the reliability path. In the classification path, the CT image that has undergone preprocessing passes through the lightweight CNN, thus producing class probabilities of being benign, malignant, and normal. On the other hand, in the reliability path, Monte Carlo Dropout is turned on during the inference phase, and the same image is run 50 times. This results in stochastic outputs that are averaged to produce the final mean output. Additionally, confidence, entropy, mutual information, and variation ratio metrics are computed as measures of uncertainty. The selective decision component takes reliable predictions and sends uncertain predictions to experts for consideration. Finally, the Grad-CAM technique is applied to visualize the regions responsible for class prediction.
Lightweight CNN architecture
The proposed neural network architecture is a light-weight task-oriented CNN for three-class classification of lung CT. The network takes a 224 × 224 × 3 image as an input and outputs Softmax probabilities of the benign, malignant, and normal classes. The neural network has a feature extraction part composed of four convolutional blocks and a classification head. The convolutional feature extractor has four Conv2D layers with 32, 64, 128, and 128 filters, respectively. The convolutional layers have a 3 × 3 kernel, use 'same' padding, and have ReLU activation functions. A max-pooling with the kernel size 2 × 2 is performed after each convolutional layer to progressively downsample the image and retain discriminatory features. At the output of the fourth pooling layer, the feature map has a size 14 × 14 × 128. The feature maps are then flattened to a 25,088-dimensional vector and fed to the fully-connected layer with 256 neurons and ReLU activation function. A dropout layer with a rate of 0.30 follows it to prevent overfitting during training. The same dropout layer is also used for Monte Carlo Dropout predictions of uncertainty during inference. Finally, the classification is done via a three-neuron Softmax output layer. The total number of parameters in the proposed CNN is 6,664,387. The stages of the proposed lightweight CNN model are described in Table 4.
For input image x and model parameters θ, the Softmax probability of class c is:
$$ p\left( {y = c\left| {x,\theta } \right.} \right) = \frac{{\exp \left( {z_{c} } \right)}}{{\sum\nolimits_{j = 1}^{C} {\exp \left( {z_{j} } \right)} }} $$
(1)
where \(z_{c}\) is the logit value for class c, c = 3 is the number of classes, and θ represents the model parameters of the CNN.
Such a design was chosen as a compromise between good classification results and simplicity of computations. While compared to computationally intensive transfer learning architectures, the suggested CNN architecture is simpler to train and understand, but at the same time it possesses enough computational power to recognize distinctive lung CT images. Also, the dropout layer serves two purposes – it avoids overfitting and allows uncertainty calculations via MC Dropout.
Monte Carlo dropout inference
In normal inference, dropout is turned off, and the trained model generates only one probability vector2. In the first reliability-based strategy, dropout is kept on and T = 50 forward passes are done using dropout. Every forward pass generates a different sub-network that outputs a probability vector \({{\varvec{p}}}^{\left(t\right)}\), and the mean distribution, predicted class, and confidence are given by:
The predicted probability vector at stochastic pass (t) is represented as:
$$ {\text{p}}^{{\left( {\text{t}} \right)}} = {\text{p}}\left( {{\text{y}}\left| {{\text{x}},{\uptheta }^{{\left( {\text{t}} \right)}} } \right.} \right) $$
(2)
The final mean predictive probability is computed as:
$$\overline{\text{p}}=\frac{1}{\text{T}}{\sum}_{\text{t}=1}^{\text{T}}{\text{p}}^{\left(\text{t}\right)}$$
(3)
The final predicted class is obtained as:
$$\widehat{\text{y}}=\text{a}\text{r}\text{g}{\text{m}\text{a}\text{x}}_{\text{c}}{\overline{\text{p}}}_{\text{c}}$$
(4)
where \(\left({\overline{\text{p}}}_{\text{c}}\right)\) is the mean probability of class (c).
The confidence score is defined as:
$$\text{C}\text{o}\text{n}\text{f}(x)={\text{m}\text{a}\text{x}}_{c}{\overline{p}}_{c}$$
(5)
The deterministic single-pass result is reported as a secondary point estimate, whereas MC Dropout is the principal uncertainty-aware result because it provides both class probabilities and the uncertainty quantities used for selective referral.
Uncertainty measures
The predictive entropy is a measure of the uncertainty present in the average categorical distribution.
$$H(\overline{p})=-{\sum}_{c=1}^{C}{\overline{p}}_{c}\text{l}\text{o}\text{g}({\overline{p}}_{c})$$
(6)
As the entropy is a function of the number of categories, it was normalized by dividing by log(C):
$${H}_{\text{n}\text{o}\text{r}\text{m}}(\overline{p})=\frac{H(\overline{p})}{\text{l}\text{o}\text{g}(C)}$$
(7)
Mutual information MI estimates the disagreement between the predictive distribution averaged across stochastic forward passes and the average entropy of the individual stochastic predictions:
$$ MI\left( x \right) = H\left[ {\overline{p}\left( {y\left| x \right.} \right)} \right] - \frac{1}{T}\sum\limits_{t = 1}^{T} {H\left[ {p_{t} \left( {y\left| x \right.} \right)} \right]} $$
(8)
For comparability with the other normalized uncertainty measures, MI was normalized by the maximum entropy \(\text{l}\text{o}\text{g}(C)\), where \(C\) denotes the number of classes:
$$M{I}_{\text{n}\text{o}\text{r}\text{m}}(x)=\frac{MI(x)}{\text{l}\text{o}\text{g}(C)}$$
(9)
The variation ratio summarizes disagreement among the 50 hard class predictions:
$$VR(x)=1-\frac{{f}_{m}}{T}$$
(10)
where \({f}_{m}\) is the frequency of the most common predicted class. A variation ratio of zero indicates that all stochastic passes select the same class, whereas larger values indicate stronger disagreement.
Selective accept-or-refer decision
For reproducible selective prediction, the final decision was based on a single scalar uncertainty score combining the four reported reliability quantities:
$$U(x)=\text{m}\text{a}\text{x}\{1-\text{C}\text{o}\text{n}\text{f}\left(x\right),{H}_{\text{n}\text{o}\text{r}\text{m}}\left(x\right),M{I}_{\text{n}\text{o}\text{r}\text{m}}\left(x\right),VR\left(x\right)\}.$$
Larger values of \(U(x)\) indicate lower predictive reliability. The operating threshold, \({\tau}_{U}\), was selected once from the validation split as the 80th percentile of the validation \(U\) scores, targeting approximately 80% validation coverage. For the principal MC Dropout model, \({\tau}_{U}\)=0.174165949, corresponding to a realized validation coverage of 80.19%.
A test prediction was accepted only when \(U\left(x\right)\)≤ \({\tau}_{U}\) ; otherwise, it was referred for expert review:
$$\text{D}\text{e}\text{c}\text{i}\text{s}\text{i}\text{o}\text{n}(x)=\left\{\begin{array}{ccc}\text{A}\text{c}\text{c}\text{e}\text{p}\text{t},& U(x)\le {\tau}_{U}, \text{R}\text{e}\text{f}\text{e}\text{r},& U(x)>{\tau}_{U}.\end{array}\right.$$
(11)
Because \(U\left(x\right)\) is defined as the maximum of the four components, acceptance is equivalently obtained when
$$\text{C}\text{o}\text{n}\text{f}(x)\ge 0.825834051,{H}_{\text{n}\text{o}\text{r}\text{m}}(x)\le 0.174165949,M{I}_{\text{n}\text{o}\text{r}\text{m}}(x)\le 0.174165949,VR(x)\le 0.174165949.$$
No test labels, test errors, or test-set performance were used to select \({\tau}_{U}\). The referral output is not a diagnostic class; rather, it is a reliability-control action applied after predicting the benign, malignant, or normal class. The same validation-derived threshold was subsequently applied to the held-out image-level test split and to the simulated stress-test conditions. Risk-coverage curves were generated by ranking images according to the same scalar \(U\left(x\right)\), ensuring that the selectivity analysis and the reported accept-or-refer rule were mathematically consistent.
Grad-CAM explanation
To explain the CNN predictions, Gradient-weighted Class Activation Mapping (Grad-CAM) was applied. Grad-CAM was computed from the activation maps of the last convolutional layer, which retain high-level spatial features. For class c, the gradient of the class probability with respect to the activation map \({A}^{k}\) was spatially averaged to generate channel weights \({\alpha}_{k}^{c}\). The rectified weighted sum resulted in the class-specific heatmap9:
$${\alpha}_{k}^{c}=\frac{1}{Z}{\sum}_{i}{\sum}_{j}\frac{\partial {y}^{c}}{\partial {A}_{ij}^{k}}$$
(12)
where \({A}_{ij}^{k}\) represents the activation value at spatial location (i, j) in channel k, and Z denotes the total number of spatial locations in the feature map.
The class-specific heatmap was then obtained from the rectified weighted sum of the activation maps:
$${L}_{Grad-CAM}^{c}=ReLU\left({\sum}_{k}{\alpha}_{k}^{c}{A}^{k}\right)$$
(13)
where \({L}_{Grad-CAM}^{c}\) is the Grad-CAM heatmap corresponding to class c, \({\alpha}_{k}^{c}\) is the importance weight assigned to the feature map channel \(k\) and \({A}^{k}\) is the activation map corresponding to channel \(k\). The use of ReLU ensures that only positive contributions to the target class are retained.
The Grad-CAM heatmaps and overlays presented in this study for both the primary IQ-OTH/NCCD dataset and the second Chest CT-Scan Images Dataset were generated by co-author Shimaa M. Abdel-Moniem using the analysis code developed for this study. The original CT slices from the respective datasets were used as input images, while the Grad-CAM heatmaps and overlays were generated programmatically. No third-party drawings, stock illustrations, or externally copied figure panels were included. These visualizations provide qualitative information about the spatial regions contributing to the model predictions and should not be interpreted as radiologist-verified tumor or lesion localization, or as segmentation maps.
Algorithm 2 integrates CT-slice classification, uncertainty estimation, validation-selected selective referral, and qualitative Grad-CAM visualization. Referral is triggered by the scalar uncertainty score rather than by the predicted class and should be interpreted as a request for expert review, not as a fourth diagnostic category.
Uncertainty-aware classification with selective referral.
Experimental setup and evaluation metrics
Training configuration
The lightweight CNN used Adam with an initial learning rate of 1 × 10⁻4, categorical cross-entropy, batch size 4, dropout rate 0.30, and a maximum of 20 epochs. The principal experiment used 842 training and 106 validation CT slices. ModelCheckpoint monitored validation accuracy, EarlyStopping used patience 7 with restoration of the best validation weights, and ReduceLROnPlateau monitored validation loss. The seed-42 member completed 14 epochs and achieved its best validation accuracy of 98.11% at epoch 7 (one-based numbering). For ensemble comparison, otherwise identical CNN members were trained using seeds 42, 142, and 242. No test-time augmentation was used (Table 5).
This setup was selected to support stable training and uncertainty-aware inference. The seed-42 run completed 14 epochs under early stopping, and the best validation accuracy was 98.11% at epoch 7 when epochs are counted from 1. The best-validation checkpoint was used for the subsequent test inference, calibration, uncertainty quantification, selective referral, and Grad-CAM analyses. The batch size and learning rate were kept fixed as reported in the experimental configuration.
Classification metrics
The model’s classification performance was tested based on the following parameters: accuracy, precision, recall, specificity, and F1-score. All of these were determined based on the following values: true positive \(TP\), true negative \(TN\), false positive \(FP\), and false negative \(FN\).
Accuracy determines the percentage of correct image classification:
$$\text{A}\text{c}\text{c}\text{u}\text{r}\text{a}\text{c}\text{y}=\frac{TP+TN}{TP+TN+FP+FN}$$
(14)
where \(TP\), \(TN\), \(FP\), and \(FN\) represent true positives, true negatives, false positives, and false negatives, respectively.
Precision measures the proportion of correctly predicted positive samples among all samples predicted as positive:
$$\text{P}\text{r}\text{e}\text{c}\text{i}\text{s}\text{i}\text{o}\text{n}=\frac{TP}{TP+FP}$$
(15)
High precision indicates that the model produces fewer false-positive predictions.
Recall, also called sensitivity, measures the proportion of correctly detected positive samples among all actual positive samples:
$$\text{R}\text{e}\text{c}\text{a}\text{l}\text{l}=\frac{TP}{TP+FN}$$
(16)
In medical diagnosis, recall is particularly important because a low recall for malignant cases may indicate missed cancer cases.
Specificity measures the proportion of correctly identified negative samples among all actual negative samples:
$$\text{S}\text{p}\text{e}\text{c}\text{i}\text{f}\text{i}\text{c}\text{i}\text{t}\text{y}=\frac{TN}{TN+FP}$$
(17)
Specificity is important for evaluating how well the model avoids falsely assigning images to a particular class.
The F1-score represents the harmonic mean of precision and recall:
$$F1-score=2\times \frac{\text{P}\text{r}\text{e}\text{c}\text{i}\text{s}\text{i}\text{o}\text{n}\times \text{R}\text{e}\text{c}\text{a}\text{l}\text{l}}{\text{P}\text{r}\text{e}\text{c}\text{i}\text{s}\text{i}\text{o}\text{n}+\text{R}\text{e}\text{c}\text{a}\text{l}\text{l}}$$
(18)
The F1-score is useful when the dataset is imbalanced because it considers both false positives and false negatives.
In case of multiclass classification, the one-vs-rest method was used to calculate class-specific sensitivity and specificity. Here, one class is considered positive class, and other two classes are taken as negative class. It enables the analysis of performance for different classifications such as benign, malignant, and normal. Weighted average precision, recall, and F1-scores were also calculated due to unequal distribution of data among classes. In case of calculating weighted values, each class is weighted based on their support. The term support refers to the number of test samples belonging to that class. Consequently, the classes with large number of samples have an effect on the final weighted value. ROC and Precision-Recall curves were drawn by using probability prediction of classes. In order to calculate the performance ranking of individual classes in multiclass evaluation, one-vs-rest approach was followed. Both macro and micro average curves were also calculated. The macro average gives equal weight to all classes and averages their performance, while micro average sums up the performance of all classes.
Calibration and selective metrics
Calibration was evaluated using 10 equal-width confidence bins spanning [0,1]. Expected calibration error (ECE) was computed as the sample-count-weighted absolute difference between mean confidence and empirical accuracy within these bins. Because the test split contains only 106 images, bin-wise estimates especially sparse low-confidence bins were interpreted cautiously. In addition to ECE, multiclass Brier score and negative log-likelihood (NLL) were reported as proper scoring rules.
$$\text{E}\text{C}\text{E}={\sum}_{m=1}^{M}\frac{\left|{B}_{m}\right|}{n}|acc({B}_{m})-conf({B}_{m})|$$
(19)
where \({B}_{m}\) is the set of samples in bin (m), (n) is the total number of samples, \(acc({B}_{m})\) is the accuracy of bin (m), and \(conf({B}_{m})\) is the average confidence of bin (m). A lower ECE value indicates better calibration.
Selective performance was measured in terms of coverage, referral rate, accepted accuracy, and accepted error risk as follows:
-
Coverage is the ratio of test images that were accepted automatically by the model. It is calculated as:
$$\text{C}\text{o}\text{v}\text{e}\text{r}\text{a}\text{g}\text{e}=\frac{{N}_{\text{a}\text{c}\text{c}\text{e}\text{p}\text{t}\text{e}\text{d}}}{{N}_{\text{t}\text{o}\text{t}\text{a}\text{l}}}$$
(20)
-
Referral rate is the ratio of images referred for review by experts and can be measured using the following equation:
$$\text{R}\text{e}\text{f}\text{e}\text{r}\text{r}\text{a}\text{l} \text{R}\text{a}\text{t}\text{e}=\frac{{N}_{\text{r}\text{e}\text{f}\text{e}\text{r}\text{r}\text{e}\text{d}}}{{N}_{\text{t}\text{o}\text{t}\text{a}\text{l}}}=1-\text{C}\text{o}\text{v}\text{e}\text{r}\text{a}\text{g}\text{e}$$
(21)
-
Accepted accuracy is the accuracy ratio for the accepted predictions as follows:
$$\text{A}\text{c}\text{c}\text{e}\text{p}\text{t}\text{e}\text{d} \text{A}\text{c}\text{c}\text{u}\text{r}\text{a}\text{c}\text{y}=\frac{{N}_{\text{c}\text{o}\text{r}\text{r}\text{e}\text{c}\text{t},\text{a}\text{c}\text{c}\text{e}\text{p}\text{t}\text{e}\text{d}}}{{N}_{\text{a}\text{c}\text{c}\text{e}\text{p}\text{t}\text{e}\text{d}}}$$
(22)
-
Accepted error risk or the selective risk is the ratio of wrong predictions out of the accepted predictions as follows:
$$\text{A}\text{c}\text{c}\text{e}\text{p}\text{t}\text{e}\text{d} \text{E}\text{r}\text{r}\text{o}\text{r} \text{R}\text{i}\text{s}\text{k}=\frac{{N}_{\text{i}\text{n}\text{c}\text{o}\text{r}\text{r}\text{e}\text{c}\text{t},\text{a}\text{c}\text{c}\text{e}\text{p}\text{t}\text{e}\text{d}}}{{N}_{\text{a}\text{c}\text{c}\text{e}\text{p}\text{t}\text{e}\text{d}}}$$
(23)
where \({N}_{\text{a}\text{c}\text{c}\text{e}\text{p}\text{t}\text{e}\text{d}}\) is the number of automatically accepted predictions, \({N}_{\text{c}\text{o}\text{r}\text{r}\text{e}\text{c}\text{t},\text{a}\text{c}\text{c}\text{e}\text{p}\text{t}\text{e}\text{d}}\) is the number of correct accepted predictions, and \({N}_{\text{i}\text{n}\text{c}\text{o}\text{r}\text{r}\text{e}\text{c}\text{t},\text{a}\text{c}\text{c}\text{e}\text{p}\text{t}\text{e}\text{d}}\) is the number of incorrect accepted predictions.
The operating point represents a trade-off between automatic coverage and referral burden. A more stringent threshold can reduce the number of potentially unreliable accepted predictions but increases the proportion of cases referred for expert review, whereas a more permissive threshold increases coverage at the cost of potentially accepting more errors. The appropriate operating point therefore depends on the intended application and should be selected independently of test-set outcomes.
Cross-validation, baselines, and domain shifts
Five-fold stratified cross-validation was retained as an image-level consistency analysis on the 1,054 duplicate-controlled slices; because patient grouping could not be recovered, it does not establish patient-independent validation. The four transfer-learning baselines ResNet5025, InceptionV326, DenseNet12127, and MobileNetV228 used ImageNet initialization, model-specific preprocessing, light training augmentation (horizontal flip, rotation 0.015, zoom 0.03, contrast 0.05), a frozen-backbone head-training phase for 5 epochs with Adam at 1 × 10⁻3, followed by fine-tuning at 1 × 10⁻5 up to 20 total epochs. The last 40 layers were eligible for fine-tuning for ResNet50, MobileNetV2, and InceptionV3, and the last 80 for DenseNet121; BatchNormalization layers remained frozen. Early stopping with patience 7, best-validation checkpointing, and ReduceLROnPlateau were used during fine-tuning. No test-time augmentation or class-weighting was used in these transfer-learning runs. For simulated domain shift, transformations were applied to float images on the [0,255] scale and clipped back to [0,255]: low contrast used "tf.image.adjust_contrast" with factor 0.50; Gaussian noise was zero-mean with standard deviation 18 intensity units using stateless random generation and stress-test seed 20260813; brightness-down subtracted 51 intensity units; brightness-up added 51 intensity units. Accuracy, referral rate, mean uncertainty measures, and error-detection AUROC/AUPRC were reported for each condition.
Additional uncertainty-estimation methods
Deep Ensemble used three independently initialized copies of the same 6,664,387-parameter CNN with seeds 42, 142, and 242. Each member was trained on the same 842-image training split and selected using the 106-image validation split. Deep Ensemble predictions were obtained by averaging the three deterministic Softmax probability vectors. Ensemble MC Dropout retained dropout at inference for each member and aggregated 50 stochastic passes from each of the three members, giving 150 probability vectors per CT slice. For each method, its selective threshold was independently computed as the 80th percentile of that method's validation U scores and then fixed before test evaluation. The resulting thresholds were 0.174166 for MC Dropout, 0.263459 for Deep Ensemble, and 0.343320 for Ensemble MC Dropout.
Second-dataset benchmark
To address cross-dataset evaluation, a separate experiment used the Chest CT-Scan Images Dataset29. Its three cancer subtype folders were collapsed into one Cancer class and compared against Normal, producing a binary task different from the primary benign/malignant/normal taxonomy. A global audit removed 153 exact duplicates from 1,000 downloaded images and then removed two conservative cross-split near duplicates, leaving 845 images. The final split contained 497 training, 62 validation, and 286 test images. No reliable patient identifiers were available. A separately retrained lightweight CNN variant using four convolutional blocks, global average pooling, Dense(128), dropout, and a binary output layer was used; therefore, this experiment is not direct external validation of the original three-class model. The same composite uncertainty definition U and validation-only 80th-percentile selection criterion were used, yielding \({\tau}_{U}\)=0.161307.
Experimental results
Training behavior
In the seed-42 principal experiment, training accuracy increased from 60.21% at epoch 1 to 98.69% at epoch 7, while validation accuracy increased from 73.58% to its best value of 98.11% at epoch 7. Training loss decreased from 0.8745 to 0.0485 and validation loss from 0.6992 to 0.0521 over the same interval. Training continued for 14 epochs under early stopping, after which the best validation-accuracy weights were restored. The seed-142 and seed-242 ensemble members reached best validation accuracies of 97.17% and 99.06%, respectively, under the same architecture and split. Figure 4(a) shows the seed-42 training/validation curves, while Fig. 4(b) and (c) show the corresponding training histories for the seed-142 and seed-242 ensemble members. The best validation accuracy for seed 42 occurred at epoch 7 (one-based numbering).
(a) Training and validation accuracy and loss curves for the seed-42 ensemble member. (b) Training and validation accuracy and loss curves for the seed-142 ensemble member. (c) Training and validation accuracy and loss curves for the seed-242 ensemble member.
Primary uncertainty-aware test performance
The principal performance of the model during testing was calculated based on the Monte Carlo Dropout inference method with 50 stochastic forward passes of each test sample independently. In such a way, the final prediction was formed through averaging the 50 probability distributions and taking the class with the highest probability average value.
As shown in Table 6, with 50-pass MC Dropout, the model correctly classified 105/106 test CT slices (accuracy 99.06%; exact 95% CI 94.86–99.98%). Weighted precision, recall, and F1-score were 99.08%, 99.06%, and 99.04%, respectively. Macro ROC AUC was 1.0000, micro ROC AUC was 1.0000 after rounding (0.99996), and micro-average precision was 0.99991. The deterministic single-pass evaluation of the same seed-42 model also classified 105/106 slices correctly (99.06%) in the principal experiment. MC Dropout remains the principal single-model uncertainty result because it produces a stochastic uncertainty profile and supports selective referral.
The deterministic and MC Dropout point accuracies are identical in the principal experiment (105/106). Their roles nevertheless differ: deterministic inference supplies only a class probability vector, whereas MC Dropout supplies stochastic uncertainty quantities that are used by the referral rule. Accordingly, no superiority claim is made from the deterministic-versus-MC accuracy difference. The confusion matrix in Fig. 5 shows 105/106 correct MC Dropout classifications. The only error was a benign CT slice predicted as normal. All 55 malignant test slices were classified correctly in this image-level test split.
Confusion matrix of the primary uncertainty-aware CNN.
Class-wise uncertainty-aware performance is shown in Table 7. Benign sensitivity was 90.00% (9/10; exact 95% CI 55.50–99.75%). Malignant sensitivity was 100% (55/55; 95% CI 93.51–100%), but the small image-level test sample and lack of patient-wise independence preclude conclusions about a patient-level malignant false-negative rate.
Figure 6 summarizes the class-wise MC Dropout classification results. The normal class obtained precision, recall, and F1-score of 97.62%, 100%, and 98.80%, respectively; the malignant class obtained 100% precision, recall, and F1-score; and the benign class obtained 100% precision, 90.00% recall, and 94.74% F1-score. The reduced benign recall is explained by the single benign CT slice that was predicted as normal in the held-out 106-slice image-level test split.
Class-wise precision, recall, and F1-score.
The one-vs-rest ROC curves were generated from class-probability scores. The ROC AUC was 1.0000 for each class, and the micro-average ROC AUC was 0.99996 (1.0000 when rounded to four decimal places), as shown in Fig. 7.
One-vs-rest ROC curves of the uncertainty-aware CNN.
Precision-recall curves in Fig. 8 summarize ranking performance for each class. Class-wise average precision was 1.0000, and micro-average precision was 0.99991 (0.9999 when rounded to four decimal places). ROC AUC and average precision can remain near-perfect despite a single incorrect argmax classification because these curves evaluate ranking across continuous probability thresholds, whereas the confusion matrix uses the final highest-probability class.
Precision-recall curves of the uncertainty-aware CNN.
Five-fold cross-validation
Five-fold stratified cross-validation on the 1,054 duplicate-controlled CT slices produced a mean image-level accuracy of 97.82% ± 1.28%, with fold accuracies ranging from 95.73% to 99.05%. Because reliable patient identifiers were unavailable, these folds were created at image level and may contain correlated slices from the same patient across folds. The cross-validation result therefore provides an internal consistency estimate only and should not be interpreted as patient-independent validation (Table 8).
Accuracy values attained by the proposed method in each fold using the five-fold stratified validation technique are illustrated in Fig. 9. The highest accuracy was recorded at 99.05% in the 5th fold, while the lowest was recorded at 95.73% in the 4th fold, giving an average of 97.82% ± 1.28%.
Accuracy across five stratified cross-validation folds.
Uncertainty profile of the misclassified case
The only MC Dropout error in the held-out image-level test split was a benign CT slice predicted as normal. Its confidence was 0.4341, normalized predictive entropy 0.9733, normalized mutual information 0.1638, variation ratio 0.42, and composite U = 0.9733. Because U exceeded the fixed validation-derived \({\tau}_{U}\)=0.174166, the case was referred rather than accepted automatically. This observation demonstrates that the single observed test error was ranked as unreliable in this image-level sample; it does not prove that all future errors will be detected. The value of the case study is therefore selective rather than diagnostic: a conventional forced classifier would emit the predicted label, whereas the uncertainty-aware pipeline adds a reliability control that can request expert review when the stochastic prediction profile is insufficiently reliable (Table 9).
Accept-or-refer performance and risk-coverage analysis
At the validation-selected operating point \({\tau}_{U}\)=0.174166, 88/106 test slices were accepted and 18/106 were referred, corresponding to 83.02% coverage and a 16.98% referral rate. All 88 accepted slices were correctly classified in this sample, so the observed accepted accuracy was 100% (exact 95% CI 95.89–100%) and the observed selective risk was 0% (95% CI 0–4.11%) as shown in Table 10. The single classification error was among the referred slices. These are finite-sample point estimates and confidence intervals, not proof of zero underlying clinical error risk.
Risk-coverage analysis was generated by sorting the same composite score U used by the decision rule as illustrated in Table 11. At full coverage (106 accepted), observed selective risk was 0.94%. Once the single highest-risk error was excluded, observed selective risk became 0% for the reported lower-coverage operating points. The prespecified validation-selected operating point accepted 88 slices (83.02% coverage), rather than selecting a test operating point based on where the test error disappeared.
The risk-coverage curve illustrates an automation-versus-referral trade-off on this 106-slice test sample. The observed risk fell from 0.94% at full coverage to 0% after the single error was referred. Because only one test error occurred, the curve should not be interpreted as evidence that a zero-risk operating point will generalize to new patients or institutions. Figure 10 plots selective risk against coverage using the scalar U ranking. The validation-selected operating point (83.02% coverage) is reported separately from exploratory points on the test curve.
Risk-coverage curve derived from the reported operating points.
Figure 11 provides the corresponding referral-versus-observed-risk view. The finite-sample observation of zero accepted errors is accompanied by an exact 95% selective-risk confidence interval of 0–4.11%. In summary, selective referral detected the single observed classification error in this image-level test sample while retaining 83.02% automatic coverage at the prespecified validation-derived threshold. The result supports the feasibility of uncertainty-based triage but does not establish a zero-risk clinical operating point.
Referral rate versus accepted error risk.
Calibration analysis
Calibration was evaluated using 10 equal-width confidence bins. The MC Dropout test ECE was 0.0295; the multiclass Brier score was 0.0137, NLL was 0.0385, and mean confidence was 0.9693. Because the test set contains 106 slices and the low-confidence bins contain very few observations, these calibration estimates are descriptive internal estimates rather than precise population calibration measures.
Five of the ten equal-width bins were non-empty. Ninety-nine of 106 predictions were in the 0.9–1.0 confidence bin, whereas the single error was in the 0.4–0.5 bin with mean confidence 0.4341. Large gaps in sparse low-confidence bins should be interpreted cautiously because they contain only one to three images. Table 12 lists the five non-empty bins from the 10-bin equal-width calibration scheme; empty bins are omitted from the table. The concentration of predictions in the highest-confidence bin and the very small sample counts elsewhere limit fine-grained calibration conclusions.
The reliability diagram in Fig. 12 compares empirical accuracy with mean confidence across the non-empty confidence bins. Most predictions (99/106) were in the 0.9–1.0 bin and were correctly classified, whereas the single misclassified case was in the 0.4–0.5 bin with confidence 0.4341. Because the lower-confidence bins contain very few samples, these bin-level gaps should be interpreted cautiously.
Reliability diagram for all uncertainty-aware test predictions.
Simulated domain-shift stress testing
The simulated domain-shift results are shown in Table 13. Clean MC Dropout accuracy was 99.06% with a 16.98% referral rate. Low contrast reduced accuracy to 97.17% and increased referral to 51.89%; Gaussian noise reduced accuracy to 91.51% and increased referral to 38.68%. Brightness-down preserved 99.06% accuracy with 22.64% referral, whereas brightness-up produced 98.11% accuracy with 16.98% referral. The uncertainty score referred all observed errors under each perturbation in this stress test. Error-detection AUROC/AUPRC were 1.000/1.000 for clean, 1.000/1.000 for low contrast, 0.935/0.440 for Gaussian noise, 0.962/0.200 for brightness-down, and 1.000/1.000 for brightness-up. These perturbations are controlled sensitivity tests and do not substitute for real multi-center domain-shift evaluation.
The largest classification degradation occurred under Gaussian noise, while low contrast produced the highest referral rate. The error-detection metrics show that higher uncertainty generally ranked errors above correct predictions, although performance under Gaussian noise was less perfect, particularly in AUPRC, which is sensitive to the small number of errors. Figure 13 summarizes accuracy, mean predictive entropy, and referral rate for the clean and perturbed conditions. These results indicate rather than prove that the uncertainty-aware pipeline reacts to several controlled image-quality changes.
Accuracy, mean normalized predictive entropy, and referral rate under simulated low contrast, Gaussian noise, and brightness perturbations.
Representative uncertain and referred cases
Figure 14 presents representative accepted and referred test CT slices using the fixed composite U threshold. Correctly classified slices can still be referred when their uncertainty profile exceeds \({\tau}_{U}\); referral therefore indicates insufficient reliability for automatic acceptance rather than an additional diagnostic class. Labels in the figure denote ground truth (True), predicted class (Pred), confidence (Conf), normalized entropy (Ent), mutual information (MI), variation ratio (VR), and the final Accept/Refer action.
Representative accepted and referred CT images ranked among uncertain test cases.
The final decision in Fig. 14 is based on the complete composite score U, not on confidence or entropy alone. Mutual information and variation ratio contribute when stochastic subnetworks disagree, allowing two visually similar-confidence examples to receive different decisions if their MC Dropout disagreement differs.
Grad-CAM analysis
Grad-CAM activation maps were generated for representative benign, malignant, and normal CT slices using the trained CNN. The heatmaps provide qualitative information about spatial regions contributing to each class prediction. Some examples in Fig. 15 show activation within lung regions, whereas broader activation may extend toward the mediastinum, chest wall, or image background. These patterns should not be interpreted as verified lesion localization because no radiologist-annotated lesion masks were available for quantitative explanation validation. Grad-CAM is therefore presented only as a qualitative model-attention visualization.
Grad-CAM visualizations of representative benign, malignant, and normal lung CT slices.
Expanded same-split comparison with uncertainty methods and transfer-learning baselines
Figure 16 shows overall accuracy and benign sensitivity for the expanded same-split comparison. The two ensemble uncertainty methods had 100% point accuracy and 100% benign sensitivity on this 106-slice split, whereas the single-model MC Dropout result was 99.06% with 90% benign sensitivity. These finite-sample results should be interpreted in light of the reported confidence intervals and the image-level data limitation, rather than as evidence that one architecture will consistently outperform others on independent patients.
Same-split comparison of MC Dropout, Deep Ensemble, Ensemble MC Dropout, and four transfer-learning baselines: overall accuracy and benign sensitivity.
Comparison with recent IQ-OTH/NCCD studies
Recent IQ-OTH/NCCD and related CT-classification studies published between 2023 and 2026 are summarized in Table 15 for contextual comparison. Because these studies used different dataset versions, data splits, augmentation strategies, patient-level controls, and model-selection procedures, their reported results are not treated as controlled baselines. The controlled same-split experimental comparison is presented in Table 14, whereas Table 15 and Fig. 17 provide literature context only. Recent studies have reported high CT-slice classification performance using EfficientNet-, ensemble-, attention-, CNN-GRU-, CNN-transformer-, and ResNet-based approaches. The contribution of the present study lies in the explicit comparison of multiple uncertainty-estimation methods and the use of a reproducible validation-selected selective-referral rule, rather than in claiming superior classification performance. Moreover, because the IQ-OTH/NCCD experiments were performed using image-level splits, the reported results should not be interpreted as evidence of patient-level generalization.
Table 14 expands the controlled image-level comparison by adding the reviewer-requested Deep Ensemble and Ensemble MC Dropout results to the four transfer-learning baselines. All rows in Table 14 use the same 106-image test split. Deep Ensemble and Ensemble MC Dropout each achieved 106/106 correct classifications in this run; these point estimates do not establish superiority because the test set is small, image-level, and not patient-independent. Recent publisher-reported architectures are kept separately in Table 15 to avoid presenting results obtained under different datasets and protocols as controlled same-split comparisons.
Contextual comparison of publisher-reported IQ-OTH/NCCD accuracies and the primary proposed result.
Separately retrained second-dataset benchmark
The second-dataset experiment used the Chest CT-Scan Images Dataset29 as a separately retrained binary Cancer-versus-Normal CT-slice benchmark. Of 1,000 downloaded images, 153 exact duplicates were removed globally; two additional conservative cross-split near duplicates were removed, leaving 845 images. The final split contained 497 training, 62 validation, and 286 test images. The audit detected no exact raw-file or standardized-pixel overlap with IQ-OTH/NCCD. Reliable patient identifiers were unavailable, so this evaluation is also image-level. The separately retrained lightweight CNN variant correctly classified all 286 test slices (261 Cancer and 25 Normal), giving 100% observed accuracy and weighted F1-score (exact 95% accuracy CI 98.72–100%), ECE 0.0757, Brier score 0.0314, and NLL 0.0856. The binary selective threshold \({\tau}_{U}\)=0.161307 was chosen as the 80th percentile of validation U without using test labels or errors. On the test split, 134/286 images were accepted (46.85% coverage) and 152/286 referred (53.15%). Observed accepted accuracy was 100% (95% CI 97.28–100%) and observed selective risk was 0% (95% CI 0–2.72%). However, selective coverage was highly class-dependent: 134/261 Cancer slices were accepted, while 0/25 Normal slices were accepted and all Normal slices were referred. Therefore, the second-dataset result does not demonstrate balanced selective performance and is not direct external validation of the primary three-class model. The results are summarized in Table 16 and Figs. 18, 19, 20, 21. Figure 18 presents the training history, Fig. 19 shows the test confusion matrix, Fig. 20 summarizes classification and selective performance, and Fig. 21 provides qualitative Grad-CAM examples from the separately retrained model.
Training and validation loss and accuracy curves for the separately retrained second-dataset binary model after duplicate-controlled curation.
Test confusion matrix for the separately retrained second-dataset Cancer-versus-Normal model.
Second-dataset binary classification and selective-referral performance.
Qualitative Grad-CAM examples from the separately retrained second-dataset model. The heatmaps and overlays show model-attention patterns for representative Cancer and Normal CT slices and should not be interpreted as verified lesion-localization maps.
Discussion
The present study evaluated a lightweight uncertainty-aware CNN for image-level classification of benign, malignant, and normal lung CT slices, emphasizing uncertainty estimation and selective referral rather than accuracy alone. With 50-pass MC Dropout, the model correctly classified 105/106 test slices (99.06%; weighted F1-score 99.04%). The single misclassified benign slice was predicted as normal, exhibited the highest composite uncertainty score among the test samples, and was referred by the fixed validation-derived rule. This finding illustrates the intended role of selective prediction: the uncertainty layer does not create an additional diagnostic class but identifies cases that may be unsuitable for automatic acceptance. Because only one error occurred, however, the result should not be interpreted as evidence that all future errors will be detected. At the validation-selected operating point, MC Dropout accepted 88/106 slices (83.02% coverage) and referred 18/106, with 100% observed accepted accuracy and 0% observed selective risk; the corresponding 95% selective-risk interval of 0–4.11% emphasizes the finite-sample uncertainty around this point estimate. Deep Ensemble and Ensemble MC Dropout each classified 106/106 test slices correctly, but their calibration and selective behavior differed from MC Dropout. MC Dropout, Deep Ensemble, and Ensemble MC Dropout produced ECE/Brier/NLL values of 0.0295/0.0137/0.0385, 0.0375/0.0119/0.0418, and 0.0492/0.0155/0.0549, respectively, with validation-selected test coverages of 83.02%, 83.96%, and 84.91%. These differences show that identical or perfect point accuracy does not imply equivalent uncertainty or calibration behavior and support reporting reliability metrics alongside conventional classification performance. The simulated perturbation experiments further showed that classification accuracy and referral behavior changed when image quality was degraded, particularly under Gaussian noise. These controlled stress tests indicate sensitivity of the uncertainty-aware pipeline to image-quality changes but do not substitute for real multi-center domain-shift evaluation. Comparison with recent IQ-OTH/NCCD studies should likewise remain contextual because published results were obtained under different dataset versions, splits, augmentation strategies, patient-level controls, and model-selection procedures. The main contribution of the present study is therefore the explicit comparison of multiple uncertainty estimators and the reproducible validation-selected referral mechanism, rather than a claim of superior leaderboard performance. The separately retrained Chest CT-Scan binary benchmark provides additional dataset-specific evidence: although all 286 Cancer-versus-Normal test slices were classified correctly, only 46.85% were accepted and all 25 Normal slices were referred. This class-dependent coverage shows why selective performance should be reported by class and why perfect point accuracy alone is insufficient to establish balanced reliability. Overall, the findings support the feasibility of uncertainty-aware CT-slice triage while remaining consistent with the image-level proof-of-concept scope of the study.
Clinical interpretation
The proposed framework should be considered an experimental image-level decision-support approach rather than a diagnostic system. Its outputs include the predicted class, predictive confidence, normalized entropy, normalized mutual information, variation ratio, the composite uncertainty score U, the Accept/Refer action, and a qualitative Grad-CAM visualization. These quantities are associated with individual CT slices and do not represent patient-level diagnoses. An accepted prediction should therefore remain advisory, whereas a referred prediction indicates that the uncertainty level exceeded the prespecified operating threshold and that expert review is warranted. Importantly, the observed 0% selective risk at the selected operating point must be interpreted together with its 95% confidence interval of 0–4.11% and should not be presented as evidence of zero clinical risk.
Strengths of the proposed framework
The study has several methodological strengths. Exact duplicate images were removed before dataset partitioning, and the data-splitting procedure, patient-identifier audit, and random seeds were explicitly documented. Three uncertainty-estimation strategies were evaluated on the same image-level test split, and all selective decisions were linked to a mathematically defined composite uncertainty score with thresholds chosen exclusively from validation predictions. The study additionally reports exact confidence intervals, calibration metrics, Brier score, NLL, risk-coverage behavior, explicit stress-test parameters, error-detection measures when estimable, controlled transfer-learning baselines, and a separately retrained second-dataset benchmark. These additions improve reproducibility and provide a broader assessment of model behavior than classification accuracy alone.
Limitations
The principal limitation of this study is the absence of recoverable patient identifiers in the available IQ-OTH/NCCD working copy. Consequently, the training, validation, testing, and cross-validation procedures were performed at the image level, and correlated CT slices from the same patient may have been distributed across different subsets despite exact duplicate removal. This limitation may lead to optimistic performance estimates and prevents the reported results from being interpreted as patient-independent validation. In addition, the primary test set was relatively small and contained only 10 benign slices, which limits the precision of class-specific performance, calibration, and selective-risk estimates. Deep Ensemble and Ensemble MC Dropout produced no errors on the current test split, preventing reliable estimation of error-detection AUROC and AUPRC for these methods and not implying zero population error. The second-dataset experiment used a different binary Cancer-versus-Normal taxonomy and a separately retrained lightweight CNN variant; therefore, it does not constitute direct external validation of the original three-class model. Moreover, its selective-referral performance was strongly class-dependent, with all Normal test slices being referred. Finally, the simulated contrast, brightness, and Gaussian-noise perturbations represent simplified stress conditions and cannot fully reproduce real-world variations caused by differences in scanners, acquisition protocols, reconstruction settings, institutions, patient populations, or disease characteristics. Grad-CAM visualizations were also assessed qualitatively because radiologist-annotated lesion masks were unavailable for quantitative localization validation.
Conclusion and future work
This study presents a lightweight uncertainty-aware CNN framework for image-level classification of benign, malignant, and normal lung CT slices, combining Monte Carlo Dropout, Deep Ensemble, Ensemble MC Dropout, selective referral, calibration analysis, and Grad-CAM visualization. The 50-pass MC Dropout correctly classified 105 of 106 test slices, corresponding to an accuracy of 99.06% (exact 95% CI: 94.86–99.98%) and a weighted F1-score of 99.04%. The only misclassified slice was assigned the highest composite uncertainty score among the test samples and was referred by the validation-derived selective rule rather than being automatically accepted. At the selected operating point, 88 of 106 test slices were accepted and 18 were referred, resulting in 83.02% coverage, 100% observed accepted accuracy (95% CI: 95.89–100%), and 0% observed selective risk (95% CI: 0–4.11%). The comparison of uncertainty-estimation methods showed that Deep Ensemble and Ensemble MC Dropout each classified all 106 test slices correctly, although their calibration and referral characteristics differed from those of MC Dropout. This finding highlights that classification accuracy alone is insufficient for characterizing the reliability behavior of uncertainty-aware models. The simulated stress tests further showed that image degradation, particularly Gaussian noise, can reduce accuracy and increase referral, supporting the value of evaluating uncertainty under controlled perturbations rather than relying solely on clean test performance. The separately retrained Chest CT-Scan binary benchmark provided additional dataset-specific evidence, achieving 286/286 correct Cancer-versus-Normal classifications. However, its selective coverage was only 46.85%, and all 25 Normal test slices were referred. Therefore, the second-dataset experiment should be interpreted as a dataset-specific image-level benchmark rather than direct external validation of the original three-class model. The main limitation of the study is the lack of recoverable patient identifiers in the available IQ-OTH/NCCD working copy. Consequently, the training, validation, test, and cross-validation partitions were created at the image level, and correlated slices from the same patient may occur across subsets. The reported results should therefore be interpreted as internal proof-of-concept evidence rather than as patient-independent clinical validation. Overall, the proposed framework demonstrates the potential value of combining CT-slice classification with explicit uncertainty estimation and validation-selected selective referral. Future work should focus on patient-wise and multi-center evaluation, larger independent cohorts, subgroup-aware calibration, real-world scanner and acquisition shifts, prospective human–AI workflow studies, and quantitative validation of explanation maps. Until such validation is performed, the framework should be regarded as an uncertainty-aware CT-slice classification proof-of-concept rather than a clinically validated diagnostic system.
来源:Nature 健康护理主题 · nature.com