AiSEPTRON Phase One — Study Results

The first phase of the AiSEPTRON study has been published and the findings are quite exciting: it shows that we can use Ai to predict what might happen in children with sepsis. Jacob Shaw, one of the members of the patient and public involvement (PPI) advisory group who contributed to the study, gives his lowdown on what it showed...

Background

So what is AiSEPTRON all about? It's a study that aims to develop a machine learning model to help predict the outcomes of children that come into hospital with suspected sepsis (a serious possible result of infections). AiSEPTRON phase one was a retrospective study, which means that it looked at historical data rather than following patients through the hospital in real-time.

Why is AiSEPTRON important?

Sepsis causes a large number of deaths in UK hospitals and worldwide, and it is important that doctors detect it quickly to increase the chances of successful treatment.

What we need to know as early as possible is: which children have sepsis or not, which ones are going to need tests and treatment, and which ones will go on to become really unwell and need further treatment or stay in hospital. However, there is no single test for sepsis, and predicting it in children is notoriously difficult - especially using information from when they come to A&E where the vast majority usually present.

What data was used?

This study looked at information from the electronic health records of children under 16 years coming into hospital between 1 January 2018 and 31 December 2019. The location used was the A&E at St Thomas’ Hospital in London which manages complex and specialised cases (tertiary care), but not children with very severe injuries (major trauma).

A total of 46553 patient attendances to A&E were looked at. After excluding those who came to hospital because of accidents or injuries, as well as any incomplete records, that left a total of 35795 patient attendances suitable for analysis.

How was the model built?

This large pool of data was used to develop 4 predictive models:

Model 1: triage model - this used all 35795 attendances to predict which children might need antibiotics using the information taken at triage (when they first came in).

Model 2: antibiotic model - this used information from 4700 of the attendances who went on to have blood tests to predict which ones might need antibiotics.

Model 3: critical care model - this used information from 155 eligible attendances who were more unwell to predict which might need antibiotics and treatments like fluids and ventilation (usually in intensive care).

Model 4: serious infection model - this used information from 443 eligible attendances to predict who might need antibiotics and a stay in hospital longer than 2 days.

4 different machine learning (Ai) methods were applied to these models to see which was best at predicting the outcome in question.

What was the best model in the end?

The best machine learning method was called XGBoost. The best model was the triage model (model 1) because it had the largest dataset and had the best predictive accuracy. It could predict whether a patient would need antibiotics over 80% of the time simply by using information collected when they first came to hospital.

So when combined with human input, this model could be used to better predict which children would need antibiotics and get them the right treatment sooner - hopefully leading to better outcomes as a result.

Are there any caveats though?

The study used a very large dataset of people which will help to make the models more accurate. It was also the first study of its kind, which makes it very important in the drive to treat sepsis in children as effectively as possible.

However, the data only came from one hospital so it is difficult to know how well it would apply to other hospitals, particularly ones that treat different demographics. The size of the dataset was much smaller for very unwell children (because these represent a minority of cases) which limits the accuracy of the model in comparison to using information from all children when they first come in.

The model also had a tendency to overestimate how sick children would become, which could lead to them receiving treatment they did not actually need.

So what next?

In March 2024, the project received a grant from the Evelina London Children's Charity to begin phase two of the study. Now we are working with much more information from hospitals around the UK to see how well the predictive models work when looking at a much larger dataset.

The PPI group are involved in this too and look forward to hopefully sharing more exciting findings as they become available!