During my internship at CyPsi lab, I worked on a project that at first seemed like a standard image classification task: detecting diabetic retinopathy from retinal fundus images. It did not stay that simple for long. The more I worked on it, the more it became clear that this problem sits at the intersection of machine learning, data limitations, and real-world constraints.
Diabetic retinopathy is a complication of diabetes that can lead to blindness if not detected early. Screening at scale is difficult because it requires trained specialists and consistent evaluation. That makes it a good candidate for automation. At the same time, it comes with challenges that do not show up in typical computer vision tasks. The data is heavily imbalanced, the differences between classes are subtle, and datasets are relatively small.
Starting with CNN Baselines
I started with convolutional neural networks using transfer learning. Models like ResNet and EfficientNet are widely used and perform well on many vision tasks, so they were a natural starting point. They worked reasonably well for binary classification, but once I moved to multiclass severity prediction, the limitations became clear. The models struggled to distinguish between adjacent stages of the disease, and they were biased toward the majority class. Severe cases, which are the most important to detect, were often missed.
Instead of switching models immediately, I focused on improving the pipeline. Preprocessing turned out to be more important than I initially expected. Retinal images vary a lot in terms of lighting and quality, so I used domain-specific techniques to enhance important features and remove noise. I also implemented on-the-fly data augmentation rather than static augmentation. This allowed the model to see new variations of the same images in every epoch, which improved generalization without increasing storage requirements.
Handling Class Imbalance
Handling class imbalance had the biggest impact. I used weighted random sampling to ensure that minority classes appeared more frequently in training batches, and focal loss to focus learning on harder examples. This led to a significant improvement in recall for severe diabetic retinopathy cases. The jump from around thirty percent to over sixty percent made it clear how much imbalance was affecting the model.
Moving to Vision Transformers
Even after these improvements, there were still limitations. Convolutional models tend to focus on local features, while retinal diagnosis often depends on patterns spread across the entire image. This motivated a shift to Vision Transformers. Unlike CNNs, transformers can model relationships across the whole image from the beginning, which makes them better suited for capturing global structure.
The challenge with Vision Transformers is their size. Large models have hundreds of millions of parameters, which is not ideal for small medical datasets. To make this practical, I used Low-Rank Adaptation, or LoRA. Instead of fine-tuning the entire model, LoRA keeps most of the pretrained weights frozen and trains only a small number of additional parameters. This reduced the number of trainable parameters drastically while still allowing the model to adapt to the task.
The results showed strong agreement with clinical labels, especially when evaluated using Cohen's Kappa, which is more appropriate for this kind of ordinal classification. I also explored a three-class setting that groups disease severity into clinically meaningful categories. This improved stability and made the system more aligned with real-world screening workflows.
Interpretability and Uncertainty
Another important part of the project was understanding what the model was actually learning. I used Integrated Gradients to visualize which regions of the image contributed to the predictions. In many cases, the model focused on lesion-rich areas such as microaneurysms and hemorrhages, which aligns with how clinicians approach diagnosis. I also used Monte Carlo Dropout to estimate uncertainty. Predictions with higher uncertainty were more likely to be incorrect, which suggests a way to flag cases that should be reviewed by a human.
Lessons Learned
One of the most valuable parts of this experience was learning how to think beyond model performance. Accuracy alone is not enough in a medical setting. Reliability, interpretability, and the ability to handle edge cases matter just as much. I also wrote a full research paper for the first time during this project. That process forced me to think more carefully about how to structure experiments, compare methods, and present results clearly.
There are still several directions to explore. I have started experimenting with ordinal-aware loss functions, since disease severity is inherently ordered. I am also testing different image resolutions to better capture fine details, and looking into ways to improve generalization across datasets.
This project changed how I approach machine learning problems. It is not only about choosing the right model. It is about understanding the data, identifying the real bottlenecks, and building systems that can work reliably outside a controlled environment.