Explainable AI (XAI) aims to provide explanations for how an Artificial Intelligence (AI) system arrived at a decision. Providing explanation for decisions taken, fosters trust in the human users of such systems. Trust is essential for such systems to be deployed for a wide range of use cases in domains like Healthcare, Banking, Insurance or even Military.
To understand this better, consider a case in Healthcare where a Machine Learning (ML) system is asked to predict whether a patient is at risk of developing heart disease or not. The patient data used for this purpose may be composed of various patient attributes like food habits, work habits, exercise routine, past medical conditions, hereditary markers, genetic markers, results of laboratory tests taken periodically etc. When the available data is rich and spans many dimensions, typically Deep Learning (DL) models are called into play as they perform better and give a greater degree of prediction accuracy.
When the prediction results are examined by a Physician, a very pertinent question may be raised — Why did the model take the decision that it did i.e. why did the model predict that a given patient, say A is at risk of developing heart disease while another patient, say B is not. This is where Deep Learning models encounter a problem i.e. there is no explanation that may be interpreted from such a model by the human user. The lack of transparency in the model mechanics is why such models are called Black Box models. It would not be amiss to say that Black Box models would find it hard to get accepted in Healthcare systems because of their opaqueness.
What would build Trust?
In this use case, explanations similar to the following made available to the Physician may have helped.
· Patient A has a sedentary lifestyle with minimal physical activity.
· Patient A has close family members that have had heart ailments and related conditions.
· Patient A has been under medication multiple times in the past 5 years and the lab tests have shown variations in blood composition.
How to build a more explainable system?
Note that for training a DL system, a lot of historical patient data is required that has been collected possibly over the past several years. This data represents patients who have been diagnosed with heart disease as well those who haven’t been.
Imagine if the available patient data that spans multiple dimensions could be grouped into buckets that have some semantic meaning. These buckets along with the specific data dimensions that fall into each bucket could be —
Lifestyle factors — food habits, work habits, sleep quantity, exercise routines, smoking habits, alcohol intake, stress levels
Medical factors — conditions like hypertension or diabetes, results of lab tests, past medical history, known allergies, known sensitivities
Genealogical factors — heart ailments within the close family, hereditary factors, genetic markers etc.
In addition to the DL model that takes all the available patient data across multiple dimensions as input, consider that there are Component DL models. Each Component DL model is fed patient data pertaining to one semantic bucket only. In this use case, there will be 3 Component DL models, one each catering to Lifestyle factors, Medical factors and Genealogical factors. Now, the results of all the Component DL models are compared for each prediction. The prediction is a score that indicates the risk of developing heart disease. This is typically expressed as a probability between 0 and 1.
Note: All the data dimensions listed above are not necessarily equally important from a data modeling perspective. For example, a smoking habit may be considered more important for determining risk of heart disease than say sleep quantity. The weights attached to signify the importance of each data dimension may be tuned at the time of modeling.
To illustrate this use case better, consider that a score of 0.6 and above may be taken to mean that the patient is at high risk of developing heart disease and anything lower may be taken to mean that risk is minimal.
The table above offers an explanation for why a certain patient is deemed at risk of heart disease. With this at hand, the Physician may be able to explain the reason for risk to the Patient and even recommend certain actions that may help reduce that risk.
For example, the Physician may recommend more physical activity and adequate sleep for Patient A whereas the Physician may recommend alternate available treatments for Patient B considering that patient’s medical history. Based on the above results, the Physician may engage in a more detailed conversation with Patient C regarding the patient’s family history. Even though Patient D may not be at risk overall, the Physician may message the patient to watch his or her lifestyle factors more closely as they may bump up the risk if left unregulated.
Will a Machine Learning system replace a Physician?
Absolutely not. A Machine Learning system is a tool in the Physician’s toolkit and does what it does best — crunch big data to arrive at insights. This tool may save some valuable time for the Physician. That saved time may perhaps translate into a much better interaction between the Patient and the Physician and foster the Doctor-Patient relationship.
Acknowledgements:
Review inputs: Dr. T. R. Gopalan.
Feature photo: Robina Weermeijer on Unsplash
Note: If you are curious about what kinds of data may be collected about a patient and how it may support a physician in clinical decision making, check out the You+AI Podcast episode with Dr. T. R. Gopalan on The Doctor-Patient Relationship.