Explaining AI with the Right Level of Abstraction

Identified Problem

The increasing reliance on complex neural networks in artificial intelligence raises concerns about transparency and trust, especially in critical sectors like healthcare. These AI models, while powerful, are often seen as “black boxes,” making their decision-making processes difficult to understand. Human beings are better suited to understanding simple models, making it difficult to trust AI models that prioritize performance over interpretability.

AI Solution

The research team at Singapore Management University (SMU) has developed a solution to explain AI models with the right level of abstraction, making them more transparent and trustworthy. The solution involves techniques to help explain how AI models make decisions, using methods such as decision trees and probabilistic models.

Key Features

• Uses decision trees to explain decisions made by convolutional neural networks in predicting conditions like diabetes.
• Utilizes probabilistic models to explain decisions made by more complex recurrent neural networks.
• Provides an interpretability score indicating the percentage of a model’s decisions that can be explained.
• Allows users to explore how individual decisions are made by clicking on individual cases.
• Techniques also help detect adversarial texts by identifying abnormalities in confidence scores.

Benefits of AI Solution

• Makes AI models more transparent and understandable.
• Enhances trust in AI systems, particularly in high-stakes areas like healthcare.
• Helps ensure that people have a role in overseeing AI decisions.
• Safeguards against the possibility of AI leading to disastrous outcomes, whether accidental or intentional.

Impact of AI Solution

• Enables the application of AI in areas where transparency is critical, such as healthcare.
• Promotes the responsible development and deployment of AI technologies.
• Ensures better outcomes by enabling oversight of AI decision-making.