How to create a Responsible AI Dashboard after an AutoML classifcation job for a tabular dataset?

Hello Nasreddine!

Thank you for posting on Microsoft Lear.

You're absolutely right especially we are talking about a compatibility requirements like the MLflow model with sklearn flavor.

AutoML jobs in Azure create a model in a proprietary format (not always compatible with the expected sklearn flavor for RAI). The Responsible AI dashboard currently requires:

• A registered model in MLflow format
• With scikit-learn or compatible interface (take as an example predict method)

Tabular data

As a workaround you can download the Best Model from AutoML :

In the Azure ML Studio:

Go to your AutoML Run

Click on the Best model

Register it (if not already)

Download the model locally to inspect its format

Then you can convert the model to a Sklearn-Compatible Wrapper if you need it :

Usually, AutoML best models are pipelines (like AutoMLPipelineWrapper) :

from azureml.core import Workspace, Dataset
from azureml.core.model import Model
import joblib


model_path = Model.get_model_path("your_automl_model_name")
automl_model = joblib.load(model_path)


class WrappedModel:
    def __init__(self, model):
        self.model = model

    def predict(self, X):
        return self.model.predict(X)

wrapped_model = WrappedModel(automl_model)

You can use raiwidgets or responsibleai packages:

from raiwidgets import ResponsibleAIDashboard
from responsibleai import RAIInsights

rai_insights = RAIInsights(model=wrapped_model,
                            train=X_train,
                            test=X_test,
                            target_column='your_target',
                            task_type='classification')

rai_insights.explainer.add()
rai_insights.counterfactual.add()
rai_insights.error_analysis.add()
rai_insights.causal.add()

rai_insights.compute()

ResponsibleAIDashboard(rai_insights)

You can run this in a Jupyter Notebook, or host it in Azure Machine Learning Compute.