AuroXRay — Bringing AI To Medical Imaging

Image processing has proven effective in various disciplines and medical field is certainly no exception. Chest X-Rays have been instrumental in diagnosing thoracic diseases; however, radiology is a complex discipline with the potential for human error in interpretation. AuroXRay tackles these challenges by implementing a state-of-the-art approach to support radiologists in making reliable diagnoses, offering a valuable 'second opinion'.

AuroXRay enables radiologists to perform classification based on specific patient symptoms, allowing for the selection of relevant radiological findings and pathologies tailored to the patient's condition. The project currently leverages the extensive CheXpert dataset, a repository of 224,316 chest X-ray images representing a range of pathologies and radiological finding. Further, it also uses the capabilities of LLMs to provide detailed analysis and explanation of the inference results – interpreting each finding, discussing visual patterns and symptoms for the selected radiological observation — all through precise prompting.

It uses ScoreCAM methodology to generate heatmaps which allows the radiologist to visualize the model’s attention, highlighting regions of an image that the model considers significant. This visualization approach is designed to support the radiologist’s interpretation process, and the entire implementation is inspired from WekaDeeplearning4J (an excellent Weka Implementation For #Deeplearning4J).

Honestly, I never envisioned transforming the initial idea into a full-stack application. The initial concept was to experiment with CheXpert dataset and create a magical SDK which combines the power of Weka, Mulan & Deeplearning4J . The core thought process was to process the entire dataset in iterative subsets and during each iteration, features are extracted using pre-trained transfer learning model & then it is appended to the disk in ARFF and then finally train the model using Weka, Deeplearning4J & Mulan. I never expected it to evolve into such a comprehensive implementation, but lately with Spring 3.0 and Angular 17 compelled me to evolve it into something better.

Dataset Overview

The CheXpert dataset is a comprehensive collection of chest x-rays, encompassing multiple labels for each image, with a total of 224,316 images. Previous studies treated it as a multi-label classification problem and evaluated it by considering only five labels using a smaller dataset of just 234 images but encountered the problem of 'Label Sparsity'. In multi-label classification, this can lead to overfitting because of treating missing values as U_ZEROs.

To mitigate this, we approached the dataset as a single label (binary classification) problem, dynamically (on the fly) removing instances with uncertain or missing labels. The only exception was the 'Support Devices' label, where missing values were treated as 0s to mitigate data imbalance.

This approach draws an inspiration from the Binary Relevance implementation, which in this project is termed as “Sparse Binary Relevance” and essentially enable radiologists to perform classification based on specific patient symptoms.

Evaluation

The binary classification results depict varied performance across different labels, with strong AUROC and F1-scores for key conditions like Pleural Effusion, Consolidation, Cardiomegaly, and Edema. The AUROC values range from 0.7991 for Pneumonia to 0.9556 for Pleural Other, indicating model’s excellent to distinguish between positive and negative cases.

The F1-score reflects a balance between precision and recall, which is crucial in medical imaging where both false positives and false negatives have implications. For this reason, recall (True Positive Rate, TPR) is prioritized in the training process, ensuring the model captures a high proportion of actual cases — a critical factor in clinical settings where missing a diagnosis can be costly.

Further, the False Positive Rate (FPR) for Lung Opacity is relatively, because of data imbalance and it’s a radiological finding associated with various underlying conditions, including consolidation, pneumonia, and pleural effusion. Given these limitation and potential redundancy, Lung Opacity was excluded from the final model.

The epitome is that model demonstrates an excellent performance across range of thoracic diseases and findings, with AUROC and F1-scores a key indication of its classification & diagnostic efficacy.