Revealing the Accuracy of Patient Risk Models

2020-02-02

After a heart attack or stroke, doctors often use risk models to help guide treatment. These models calculate the patient's risk of death based on factors such as age, symptoms, and other characteristics. While these models are useful in most cases, they cannot accurately predict outcomes for all patients, which can lead to doctors choosing ineffective or unnecessary treatments that endanger the patient's life. Colin Stultz, a professor of electrical engineering and computer science at MIT and a cardiac expert at Massachusetts General Hospital, said, "Every risk model is evaluated on a dataset of some patients, and while it may have high accuracy, it is not 100% accurate in practice. For some patients, the model may provide incorrect answers, which can be catastrophic." To address this issue, Stultz and his colleagues have developed a method to determine the trustworthiness of a specific model's results. Researchers say this can help doctors choose better treatment options for patients. To illustrate how this method works, the researchers selected the widely used risk model called the GRACE risk score. GRACE is a global registry of acute coronary events primarily used to develop risk models and assess the risk of death in patients with acute coronary syndrome within six months. The researchers' new technique generates an "unreliability score" ranging from 0 to 1. The higher the score for a given risk model prediction, the more unreliable the prediction is. The unreliability score is based on risk predictions generated by a specific model, such as the GRACE risk score, compared to predictions generated by different models trained on the same dataset. If the models produce different results, the risk model prediction for the patient may be unreliable. One significant advantage of this method is that it derives a formula that tells doctors how much inconsistency exists between two predictions without having to build an entirely new model based on the original dataset, saving valuable medical time and, more importantly, avoiding privacy concerns. The researchers are now designing a user interface that doctors can use to assess the reliability of a patient's GRACE score. In the long run, they also hope to improve the reliability of risk models through easier data retraining.

RECENT AI NEWS

RECENT AI TOOLS