Skip to main content
Version: v1.1.1

Shapley values support

Shapley value support in MLOps requires a model created with H2O-3 or Driverless AI version 1.10 or later, and is available for both MOJO and Python pipeline artifacts.

The following steps describe how to enable and request Shapley values.

Step 1: Enable Shapley values when deploying a model​

For DAI experiments​

If the Driverless AI MOJO pipeline artifact type (dai/mojo_pipeline) is selected when deploying a model, the MOJO Scorer (dai-mojo-scorer) supports all Shapley contribution types:

  • Original features: Shapley values for features or columns that existed as part of the original dataset or experiment.
  • Transformed features: Shapley values for features or columns that have been transformed by Driverless AI.
  • All features: Shapley values for both original and transformed features.
note

You can skip this step if you're using a DAI Python pipeline.

For H2O-3 MOJO experiments​

If the H2O-3 MOJO artifact type (h2o3/mojo) is selected when deploying a model, the H2O-3 MOJO Scorer (h2o3-mojo-scorer) supports Shapley values for transformed features.

note
  • H2O MLOps supports Shapley TRANSFORMED values in H2O-3 for MOJO experiments.
  • Shapley values are supported only for the following model types:
    • DRF
    • GBM
    • XGBoost
  • Shapley values are not supported for the following models:
    • Multinominal classification models
    • Binominal models with Binominal Double Trees parameter set
    • GLM models
Note

Enabling Shapley contributions in h2o3-mojo-scorer loads the MOJO file multiple times (once per enabled contribution type), which increases memory usage proportionally. To configure Shapley support, set the SHAPLEY_ENABLE and SHAPLEY_TYPES_ENABLED environment variables on the deployment. For full configuration details, see H2O-3 MOJO Scorer.

Step 2: Request Shapley values in a curl request​

By default, Shapley values aren't returned in a curl request. To get Shapley values (that is, the Shapley type enabled in the preceding step), you must include the requestShapleyValueType argument in the curl request and set the value to ORIGINAL, TRANSFORMED, or ALL.

Note

  • The MOJO Scorer supports all Shapley contribution types (ORIGINAL, TRANSFORMED, ALL). Set requestShapleyValueType to the type you need. For H2O-3 MOJO models, only TRANSFORMED is supported.

  • The following steps describe how to check which Shapley values have been enabled:

    1. Copy the endpoint URL of the deployment.
    2. In the endpoint URL, replace /score with /capabilities.
    3. Paste the endpoint URL in a browser window. One to three different terms are displayed that indicate whether the deployment supports Shapley values for original features and/or transformed features: [ SCORE, CONTRIBUTION_ORIGINAL, CONTRIBUTION_TRANSFORMED ]

  • To try this using the H2O MLOps Python client, see View scorer capabilities.

ORIGINAL​

ORIGINAL enables the ability to generate Shapley values for features or columns that existed as part of the original dataset or experiment.

Example usage:

"requestShapleyValueType": "ORIGINAL"

TRANSFORMED​

TRANSFORMED enables the ability to generate Shapley values for features or columns that have been transformed by DAI.

Example usage:

"requestShapleyValueType": "TRANSFORMED"

note
  • By default, this value is set to NONE, which is the equivalent of not providing the requestShapleyValueType argument in the curl request.
  • To try this using the H2O MLOps Python client, see Shapley values.

The following is a sample curl request and response with Shapley values enabled for original features:

curl -X POST -H "Content-Type: application/json" -d @- <SCORING_ENDPOINT_URL> << EOF
    {
    "fields": [
        "LIMIT_BAL",
        "SEX",
        "EDUCATION",
        "MARRIAGE",
        "AGE",
        "PAY_0",
        "PAY_2",
        "PAY_3",
        "PAY_4",
        "PAY_5",
        "PAY_6",
        "BILL_AMT1",
        "BILL_AMT2",
        "BILL_AMT3",
        "BILL_AMT4",
        "BILL_AMT5",
        "BILL_AMT6",
        "PAY_AMT2",
        "PAY_AMT3",
        "PAY_AMT4",
        "PAY_AMT5",
        "PAY_AMT6"
    ],
    "rows": [
        [
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0",
        "0"
        ]
    ], "requestShapleyValueType": "ORIGINAL"
    }
    EOF

    {
        "featureShapleyContributions":
        {
            "contributionGroups":
            [
                {
                    "contributions":
                    [
                        [
                            "0.3031580540597976",
                            "0.05037104158009451",
                            "0.01197491002508829",
                            "-0.09613404645427222",
                            "0.03349942127192829",
                            "-0.19629869420775475",
                            "-0.05457586577961132",
                            "-0.016488709633310006",
                            "0.03022179422131117",
                            "-0.010334700480366232",
                            "-0.001831092307318766",
                            "0.24078515169214335",
                            "0.059130207887999234",
                            "-0.03856367964221913",
                            "-4.1371086615778267E-4",
                            "0.02549452684520287",
                            "-0.012600605263304759",
                            "0.17373018794999764",
                            "0.19368473683713824",
                            "0.1887846519524733",
                            "0.08455862402217218",
                            "0.04817053716929957",
                            "-1.4823175495435195"
                        ]
                    ]
                }
            ],
            "features":
            [
                "contrib_LIMIT_BAL",
                "contrib_SEX",
                "contrib_EDUCATION",
                "contrib_MARRIAGE",
                "contrib_AGE",
                "contrib_PAY_0",
                "contrib_PAY_2",
                "contrib_PAY_3",
                "contrib_PAY_4",
                "contrib_PAY_5",
                "contrib_PAY_6",
                "contrib_BILL_AMT1",
                "contrib_BILL_AMT2",
                "contrib_BILL_AMT3",
                "contrib_BILL_AMT4",
                "contrib_BILL_AMT5",
                "contrib_BILL_AMT6",
                "contrib_PAY_AMT2",
                "contrib_PAY_AMT3",
                "contrib_PAY_AMT4",
                "contrib_PAY_AMT5",
                "contrib_PAY_AMT6",
                "contrib_bias"
            ]
        },
        "fields":
        [
            "default payment next month.0",
            "default payment next month.1"
        ],
        "id": "f0395bc4-47d0-11ec-b7eb-fad6d6e23f65",
        "score":
        [
            [
                "0.6144353",
                "0.38556466"
            ]
        ]
    }
Feedback