Google AI Assists Doctors in Making Diagnoses

2023-08-04

Under the joint efforts of Google DeepMind and Google Research, a paper has been published promising to provide safer medical imaging interpretations with a 25% reduction in false positive rates. AI has changed the landscape of many industries, but its application in the medical field has been progressing slowly. The application of AI in this industry faces challenges such as data privacy, security, lack of interoperability, and lack of regulation, which limit its application. AI models are prone to errors, just like humans. Google Research proposed a question: what would be the error rate when combining predictive AI models with the expertise of clinical doctors? In July of this year, Google DeepMind and Google Research launched Complimentary-driven-Deferral-to-Clinical-Workflow (CoDoC), a system that combines human expertise with predictive AI to maximize accuracy. This system can determine whether AI models are more accurate than clinical doctors' diagnoses. Comprehensive testing of CoDoC using multiple real-world datasets showed that combining human expertise with predictive AI results in higher accuracy. They found that in a breast X-ray imaging dataset, the false positive rate decreased by 25%, and more importantly, no true positive cases were missed. This published paper represents a significant advancement in the collaboration between AI and clinical doctors. It is expected to improve the accuracy of diagnosing diseases with binary outcomes. Their dataset focuses on using X-ray mammography for breast cancer screening and chest X-ray for tuberculosis triage. Avoiding AI obstacles in medicine is crucial when using AI tools in a healthcare environment. Knowing when to say "I don't know" is a key challenge addressed in this paper, which then empowers clinical doctors. Alan Karthikesalingam from Google Health, who participated in this research, said, "If CoDoC is used in conjunction with AI tools and radiologists' judgments, and CoDoC helps determine which opinion to use, the accuracy of the results will be better than using humans or AI tools alone." The CoDoC model does not require patients' medical images for diagnosis, which respects patient privacy. During the training dataset, each case only requires three inputs. The first is the output of the hospital's existing predictive AI credibility score (0 indicates the absence of disease, 1 indicates the presence of disease). The second is the output of non-AI expert clinical workflows, and the last is historical "real" data. The system is compatible with any proprietary AI model and does not require access to the model's internal workings or training data. To apply the CoDoC paradigm to existing predictive AI systems, researchers will follow the method described in the paper, which is to train a CoDoC-like model using the output of their own existing predictive AI system. The working principle of the predictive model is to learn by comparing the accuracy of the predictive AI model with the accuracy of doctors' interpretations and then examine how accuracy changes with the credibility scores of the predictive AI model. After training, CoDoC is placed in a hypothetical clinical workflow and works together with predictive AI and doctors. When the predictive AI model evaluates new patient images, its associated credibility score is inputted into CoDoC. By using AI as a reliable tool for confirming diagnoses, doctors can have confidence in their own diagnoses even in extreme cases, which was previously not possible. "The advantage of CoDoC is that it can interoperate with various proprietary AI systems," said Krishnamurthy Dvijotham from Google DeepMind. Inviting developers for testing, validation, and building To help researchers build on their work, maintain transparency, and ensure the use of safer AI models in the real world, they have open-sourced the code for CoDoC on Github. So far, this work is still theoretical, but researchers say it demonstrates the potential of AI systems to adapt and improve medical imaging performance in different populations, clinical environments, medical imaging devices used, and types of diseases.

RECENT AI NEWS

RECENT AI TOOLS