Alzheimer's Predictive Model
2024
Machine Learning
Spring
CS 4641
Content copied, from our github page:
Dataset Link:
Contributers:
Jessica Kalloor
Felipe Oliveira
Rithvik Ram
Shashaank Shankar
Ashna Gupta
Introduction
Machine learning aids in predicting Alzheimer’s disease in its early stages. Vector machines, decision trees, random forest, and CNNs are some methods used [3]. Studies have taken a computer vision approach, wherein CNNs are used to detect abnormalities in MRI and PET scans of the brain. Classifiers are also used to diagnose patients into Alzheimer’s vs healthy vs mild cognitive impairment (MCI), where a balanced accuracy of 90.6% was achieved [1]. Other techniques involve using data from psych evaluation and common brain tests. Regression and classification analyses are used to understand the distribution of features, often reaching over 80% accuracy. [2]
The Alzheimer’s Prediction Dataset includes information such as age, education, socioeconomic status, and exam scores. The ‘Group’ column serves as the target variable, to help us train the algorithm to classify a sample patient into an Alzheimer’s vs non-Alzheimer’s group.
Problem Definition
We aim to develop a method to identify Alzheimer’s Disease (AD) since the exact cause is undetermined. Research has shown that machine learning algorithms can be used to efficiently weigh features based on their likelihood in existing in AD patients. Our algorithm hopes to reach 80%> accuracy in diagnosis in order to be considered as a potential intervention before further development of the disease.
Data Preprocessing:
We first preprocessed the raw dataset by deleting rows that had 6 missing cells or more. This threshold was decided based on previous studies (Bogdanovic et. al, 2022) that used similar Alzheimer’s/Dementia datasets. By removing these rows, we can ensure that our analysis is based on more reliable and complete data.
For the patients that still had empty values for certain features, we used an average interpolation technique. This first finds data points in the same labeled category, and calculates the mean value for that feature with the missing data. The empty cell is then replaced with the mean value.
To run our supervised model, we also had to convert categorical variables into numerical values. More specifically, we changed males to ‘0’, and females to ‘1’, and the target labels as 0 to 4, based on the severity of the neurodegenerative disease (“CN” = 0, “EMCI” = 1, “LMCI” = 2, “SMCI” = 3, “AD” = 4). Once we had a complete, comprehensible dataset, we needed to reduce our 18 selected features to smaller, low-dimensional data.
We were first curious to see what features are highly correlated between each other. We created a covariance heatmap using the Seaborn package to visualize which features are more important, and which are redundant to the dataset.
We then ran a Principal Component Analysis (PCA) for dimensionality reduction. PCA allows us to reduce the number of features in our dataset while preserving most of the variance in the data. This is good for speeding up computation, reducing overfitting, and simplifying interpretation. We first created a principal components chart that indicates the amount of components necessary to maintain 80% variance.
We then used the sklearn package to run a PCA using 7 components based on the chart above.
Our reduced dataset included the features “AGE”, “PTEDUCAT”, “APOE4”, “ADS11”, “WholeBrain”, “TAU”, and “Entorhinal”. Below is an example subset of what our data looked like, including our target “DX_bl” and “PTID” as meta.
ML Algorithm: Neural Network
With Neural Networks we can look at the confusion matrix to show how accurate our predictions were for each class. For each class we can look at the prediction accuracy: CN(0) - 91.5%, EMCI(1) - 80.6%, LMCI(2) - 89.6%, SMC(3) - 80.0%, AD(4) - 85.9%. Comparing these numbers from the other models we tested the data with, it can be seen that the average accuracy is highest with Neural Network. This is further proved with our metrics.
Neural Network Metrics
AUC - 0.975:
The AUC of 0.975 demonstrates exceptional model performance in distinguishing between the different classes of Alzheimer’s progression in our dataset, from normal cognition to severe Alzheimer’s disease. This high AUC is particularly significant in Alzheimer’s diagnosis because it indicates that the model has high sensitivity (ability to correctly identify patients with the disease) and specificity (ability to exclude those without the disease). Features such as hippocampal volume and FDG-PET readings, which are known to correlate with Alzheimer’s progression, likely contribute significantly to this high AUC. This reduces both the number of missed diagnoses and (incorrect diagnoses, which are critical in a disease where early detection can substantially alter patient management and outcomes. To further improve AUC, we can refine feature selection based on our PCA to emphasize brain regions most affected early in the disease, such as the entorhinal cortex and fusiform gyrus.
​
CA - 0.863:
With an accuracy of 86.3%, the neural network shows strong capability in correctly identifying the stage of Alzheimer’s disease across our dataset with nonlinear and complex patterns. In the clinical context, this means the model can reliably differentiate between stages such as mild cognitive impairment and more severe forms. This accuracy is especially valuable when distinguishing between closely related stages like early mild cognitive impairment (EMCI) and late mild cognitive impairment (LMCI), where clinical interventions might differ significantly. With a relatively high CA to rely on, medical facilities using this model can more reliably plan appropriate interventions, therapies, and support systems tailored to the severity of the disease, thereby potentially slowing its progression and improving the quality of life for patients.
​
F1 Score - 0.862, Precision - 0.862, Recall - 0.863:
The F1 Score for our neural network model shows quality results. This high precision is essential in clinical settings to avoid over-diagnosis, where the implications of a false positive, including creating stress and payment for treatments, are significant.
Given that precision is high, it suggests effective utilization of features such as APOE4 genetic markers and MOCA test scores, which are strongly indicative of Alzheimer’s presence or absence.
Recall of 0.863 indicates that the model successfully identifies 86.3% of all actual Alzheimer’s cases. As mentioned before, high recall is vital for a condition like Alzheimer’s, where early detection can lead to more effective management and potentially delay the progression of the disease. To boost recall, refining the input layer to include more detailed clinical history or expanding the feature set to include additional biomarkers like tau protein levels could be beneficial.
Our high F1 score reflects the model’s robustness in handling the trade-offs between missing genuine cases and mistakenly diagnosing healthy individuals.
References
Work Cited [1] Diogo, V.S., Ferreira, H.A., Prata, D. et al. Early diagnosis of Alzheimer’s disease using machine learning: a multi-diagnostic, generalizable approach. Alz Res Therapy 14, 107 (2022). https://doi.org/10.1186/s13195-022-01047-y [2] Kavitha, C., Mani, V., Srividhya, S. R., Khalaf, O. I., & Tavera Romero, C. A. (2022). Early-Stage Alzheimer’s Disease Prediction Using Machine Learning Models. Frontiers in public health, 10, 853294. https://doi.org/10.3389/fpubh.2022.853294 [3] Zhao, Z., Chuah, J. H., Lai, K. W., Chow, C. O., Gochoo, M., Dhanalakshmi, S., Wang, N., Bao, W., & Wu, X. (2023). Conventional machine learning and deep learning in Alzheimer’s disease diagnosis using neuroimaging: A review. Frontiers in computational neuroscience, 17, 1038636. https://doi.org/10.3389/fncom.2023.1038636