Overview
This X-Ray AI model is a Supervised Fine-Tuned Model (SFT), trained to classify an X-ray image into one of two categories; “Fractured” or “Not Fractured”. A Supervised Fine-Tuned Model is a pre-trained machine learning model that has been further trained on a specific, labeled dataset to improve its performance on a particular task. This process adapts a general-purpose model to a more specialized application. My X-Ray AI is trained off of a Gemini model, and repurposed and tuned to classify X-ray images.
Instructions
Upload an x-ray image from your machine. The AI will predict your image as either "Fractured" or "Not Fractured"
How I Built It
In order to train an AI model, I first needed a dataset that the model would train off of. Luckily for me, there were many public databases with labeled images that I could use. I was able to collect around 4000 labeled images and put them into a csv file. I downloaded them and moved them into an index in the format of a JSON file (You will need to convert the file from a csv to a JSON, my code will be in Sources). Next, I split my data into a training, validation, and testing set. I allocated ~80% of your data for training, and ~10% for both validation and testing. The training dataset is how the model trains and improves, the validation set gives an unbiased evaluation of the models performance on unseen data, and the testing set tests your model’s performance after it has trained to make sure it’s at the level you want it to be at.
Next, I had to train my model. I initially wanted to train my model from scratch. I began by using the AutoML platform, a system designed to train an AI model without the user needing to write any code. However, this path was filled with unexpected system errors that emerged. Despite my persistent attempts to troubleshoot, the issues prevailed.
Undeterred, I moved to a more hands-on coding approach by using Python on Jupyter Notebook. My aim was to write the model's structure and training routines directly from my notebook. Unfortunately, I also encountered significant roadblocks in the form of persistent debugging challenges. Despite many meticulous efforts to find and correct errors in the code, I was unable to resolve the underlying issues.
Following this, I explored the possibility of fine-tuning an existing model within the Jupyter Notebook, hoping to circumvent the troubles of building it from scratch. However, this strategy was met with another hurdle: I could not obtain access to the pre-trained Google models that were needed for fine-tuning. The inability to access these resources prevented me from using this method.
Eventually, I found a viable path by using the console to tune a gemini model. This method allowed me to easily tune a gemini model and achieve a functional and performant model.
The final step was to create a website to host the model. I wrote a python app that utilizes an API to access the model and retrieve a response. I then used html code to style the website how I wanted.
Results
In order to measure the model's performance, I needed to compare its accuracy to something so I could measure its performance. I first decided to use the Gemini 2.5 flash model, which was the pre-trained model that mine was tuned off of, to see how much my model improved from training. The results are listed above. In order to evaluate the model, I had to write a short python script that sends a dataset to both my model and Gemini, and records and grades their responses. I used my testing dataset of around 500 images, unseen by my model and Gemini, to ensure an unbiased, and fair comparison. The results show that my model performed much better than Gemini, meaning that the model’s training was successful in improving its performance. The next thing I wanted to do was see if my AI could perform better than a radiologist.
The main purpose of the AI is to simulate a radiologist, where radiologists can use this to assist them in their work. I found that the average accuracy of a radiologist is between 83%-95%. The results show that my model is 87% accurate, which is in the middle of the range. This means my model well simulates that of a professional radiologist.
What's Next
Some next steps to improve and utilize the are AI are:
- Further improve model accuracy by expanding the training dataset with different types of X-ray images.
- Explore the use of the model in remote or underserved areas, particularly in third-world countries where access to specialized medical professionals may be limited.
- Investigate the potential for multi-label classification to detect and identify different types of fractures, beyond the current "Fractured" or "Not Fractured" categories.
About the Creator
My name is Eliott Lo, I am a 13 year old who lives in California. I wanted to build this project because I have a genuine interest in AI and wanted to create something of value to the world. I want to utilize this model to assist healthcare professionals in third-world countries to try and make healthcare more accessible.
Sources
Built on Gemini Vertex AI.
You can find all my code on GitHub, here
Datasets: I trained my model on FracAtlas’ data ( here); Other datasources: X-Ray Detection
