Explain Your ML Model Predictions with a Visualization

Training a Machine Learning model is easy. But once someone asks you to explain the predictions of your ML model, it becomes a challenge. You always know how it works, but you can’t articulate how or why your model made a particular prediction for a specific instance. So, in this article, I’ll introduce you to a tool you can use to explain your ML model predictions with a visualization.

What’s the Need to Explain the Predictions of an ML Model?

We’ve all been there. You’re beaming with pride over your TensorFlow or PyTorch creation, maybe a deep neural network that’s crushing a classification task. When your model says “yes” or “no,” “cat” or “dog,” “fraud” or “no fraud,” it does so with confidence. But, except for the label and the features, we don’t know how the model is making predictions.

This lack of transparency isn’t just an academic curiosity; it has real-world consequences. This is why model interpretability is no longer optional.

Introducing LIME: To Explain Your ML Model Predictions with a Visualization

LIME isn’t about dissecting your entire complex model. That’s often impossible or computationally prohibitive. Instead, LIME takes a brilliantly pragmatic approach: it focuses on explaining individual predictions. It’s local because it explains around a specific data point, and interpretable model-agnostic because it can explain any black-box model.

Here’s how LIME works in a Nutshell:

1. You choose an instance you want to explain.
2. LIME creates many slightly modified versions (perturbations) of that instance.
3. It then asks your model to predict the outcome for all these perturbed instances.
4. Around your original instance, LIME weights these perturbed samples by their proximity to the original and then fits a simple, interpretable model (like a linear regression or decision tree) to the black-box model’s predictions.
5. The coefficients or rules from this simple, local model become your explanation.

Confused? Don’t be. Let’s walk through a practical example that will make it.

A Practical Example: Explaining a Text Classifier

Let’s say we’ve built a sentiment classifier. It takes a piece of text and tells us if it’s positive or negative. Our goal here will be to understand why our model classified a specific review as positive.

Here’s an example of model training for text classification on a dummy dataset:

import numpy as np
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.pipeline import make_pipeline

# For LIME
# Install lime: pip install lime
import lime
import lime.lime_text
import matplotlib.pyplot as plt
import seaborn as sns

# Suppress warnings for cleaner output
import warnings
warnings.filterwarnings('ignore')

# 1. Dummy Dataset for Sentiment Classification
# Let's create a very simple dataset to illustrate
data = {
    'text': [
        "This movie was fantastic! Loved every moment.", "Absolutely brilliant film, a must-watch.",
        "Terrible acting, waste of time and money.", "Could have been better, somewhat boring.",
        "Great storyline and amazing visuals.", "Worst experience ever, never again.",
        "Highly recommend, truly engaging.", "Mediocre plot, but decent special effects.",
        "So happy with this purchase, perfect!", "Disappointed with the quality, very bad.",
        "An absolute joy to watch!", "This product broke quickly, very unhappy.",
        "What a gem! Exceeded expectations.", "Boring and uninspired, avoid it.",
        "Fantastic service, thank you!", "Poor delivery and damaged item."
    ],
    'sentiment': [
        'positive', 'positive', 'negative', 'negative', 'positive', 'negative',
        'positive', 'negative', 'positive', 'negative', 'positive', 'negative',
        'positive', 'negative', 'positive', 'negative'
    ]
}
df = pd.DataFrame(data)

# 2. Train a Black-Box Classifier (e.g., RandomForest with TF-IDF)
X = df['text']
y = df['sentiment']

vectorizer = TfidfVectorizer(stop_words='english')
classifier = RandomForestClassifier(random_state=42)

# Create a pipeline for convenience
model_pipeline = make_pipeline(vectorizer, classifier)

# Train the model
model_pipeline.fit(X, y)

print(f"Model accuracy on training data: {model_pipeline.score(X, y):.2f}")

# Let's check a prediction
test_text = ["This was an amazing film! So good."]
print(f"Prediction for '{test_text[0]}': {model_pipeline.predict(test_text)[0]}")
# Our model predicts 'positive'

Now that we have our ML model (a RandomForest in this case, representing any complex model), let’s use LIME to explain a specific prediction:

# 3. Use LIME to explain a prediction
# We need a predict_proba function that LIME can use
# Our pipeline already has this!
predictor = model_pipeline.predict_proba

# Create a LIME explainer
class_names = model_pipeline.classes_
explainer = lime.lime_text.LimeTextExplainer(
    class_names=class_names
)

# The instance we want to explain
text_to_explain = "This movie was absolutely fantastic! I loved every single moment and would recommend it to everyone."

# Generate the explanation
# num_features: max features to include in explanation
# num_samples: number of perturbed samples to generate
explanation = explainer.explain_instance(
    text_to_explain,
    predictor,
    num_features=5,
    num_samples=1000 # More samples generally lead to better explanations
)

print(f"\nExplaining: '{text_to_explain}'")
print(f"Predicted class: {model_pipeline.predict([text_to_explain])[0]}")
print("Explanation for positive sentiment:")

# Visualize the explanation
explanation.show_in_notebook(text=text_to_explain)

You’d get an interactive HTML visualization in the output. Words highlighted in green would contribute to the “positive” prediction, and words highlighted in red would contribute to the “negative” prediction (if any). The intensity of the colour indicates the strength of the contribution. Here’s the output I got:

So, this is how you can explain the predictions of your ML model with a visualization.

Final Words