Endpoint fails with the model generated by Automated ML

Question

Endpoint fails with the model generated by Automated ML

Hajime 6

I have loaded bankmarketing_train.csv to get a dataset and auto generated a model to predict "y" field value with AutoML.
Voting Ensemble model was generated as the best model and tested its behavior after deployed to the endpoint.

Schema is generated like this for the endpoint.

Tried with the endpoint test feature in ML Studio. It worked and responded an expected output (left side in the fig below).
But my python REST call fails with 502 Bad Gateway(right side)

Using the REST plug-in for VSCode, I have requested as below. This also failed with the same response status code.

POST http://d8e9f6ad-4112-4417-97c0-01b4246b284a.japaneast.azurecontainer.io/score  
Content-Type: application/json  
Authorization: Bearer === My correct key here ===  
  
{"data": [{"age": 87, "campaign": 1, "cons.conf.idx": -46.2, "cons.price.idx": 92.893, "contact": "cellular", "day_of_week": "mon", "default": "no", "duration": 471, "education": "university.degree", "emp.var.rate": -1.8, "euribor3m": 1.299, "housing": "yes", "job": "blue-collar", "loan": "yes", "marital": "married", "month": "may", "nr.employed": 5099.1, "pdays": 999, "poutcome": "failure", "previous": 1}]}

Investigated in the App Insight and queried the exceptions.
I found this end point tries to convert 'yes' to int value. Of course it fails.
114083-%E3%82%B9%E3%82%AF%E3%83%AA%E3%83%BC%E3%83%B3%E3%82%B7%E3%83%A7%E3%83%83%E3%83%88-2021-07-13-003243.png

The value 'yes' is set to 'loan' and 'housing". Both are defined string value in the swagger.json for this endpoint.

What do you think?
Am I missing something?
Is this a bug with the endpoint?

Rohit Mungi 47,716 Reputation points Microsoft Employee Moderator

2021-07-13T06:47:44.697+00:00

@Hajime I think you could try to check what is the schema provided on the consume tab(next to test tab) of the endpoint to call via REST?
Ideally, if the same works on UI test tab it should work similar for REST API calls.
Hajime 6 Reputation points

2021-07-13T07:08:53.527+00:00

Hello @Rohit Mungi ,
Thank you for your reply.
Yes, I already have tried that several times. But the result has been the same.

Is there any place to find the generated request with the 'test' tab?
Rohit Mungi 47,716 Reputation points Microsoft Employee Moderator

2021-07-13T08:11:58.59+00:00
I could see my failed request when I try to fail it and the data passed in deployment logs tab.

Could you also try to use the input data as mentioned in the consume tab with proper single and double quotes for key and values in JSON? I am not sure if this could be causing the issue as seen in your request above.

data = { "Inputs": { "WebServiceInput0": [ { 'CulmenLength': "39.1", 'CulmenDepth': "18.7", 'FlipperLength': "181", 'BodyMass': "3750", }, ], }, "GlobalParameters": { } } body = str.encode(json.dumps(data))

4 answers

Your answer

Rohit Mungi 47,716 Reputation points Microsoft Employee Moderator

2021-07-13T06:47:44.697+00:00

@Hajime I think you could try to check what is the schema provided on the consume tab(next to test tab) of the endpoint to call via REST?
Ideally, if the same works on UI test tab it should work similar for REST API calls.
Hajime 6 Reputation points

2021-07-13T07:08:53.527+00:00

Hello @Rohit Mungi ,
Thank you for your reply.
Yes, I already have tried that several times. But the result has been the same.

Is there any place to find the generated request with the 'test' tab?
Rohit Mungi 47,716 Reputation points Microsoft Employee Moderator

2021-07-13T08:11:58.59+00:00

I could see my failed request when I try to fail it and the data passed in deployment logs tab.

Could you also try to use the input data as mentioned in the consume tab with proper single and double quotes for key and values in JSON? I am not sure if this could be causing the issue as seen in your request above.

data = { "Inputs": { "WebServiceInput0": [ { 'CulmenLength': "39.1", 'CulmenDepth': "18.7", 'FlipperLength': "181", 'BodyMass': "3750", }, ], }, "GlobalParameters": { } } body = str.encode(json.dumps(data))

Answer 1

Deleted

This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

1 deleted comment

Comments have been turned off. Learn more

Answer 2

Hajime 6

OK, I will.
Thank you for your support > @Rohit Mungi

0 comments

Answer 3

I got a response from MS together with a workaround. I have confirmed the workaround is effective in this case.

The order of keys should be the same as the one defined in the model schema.

After I update the data definition in my client python file, it starts working.

data = {"data":
        [
          {
            "age": 17,
            "job": "blue-collar",
            "marital": "married",
            "education": "university.degree",
            "default": "no",
            "housing": "yes",
            "loan": "yes",
            "contact": "cellular",
            "month": "may",
            "day_of_week": "mon",
            "duration": 971,
            "campaign": 1,
            "pdays": 999,
            "previous": 1,

Thank you.

Answer 4

We just ran into this issue as well. Found out that the order of properties in the "data" structure must match exactly with the order of columns in the dataset.

This does not make sense to me, since the property names are passed in the json, why would the order matter also?

On top of that, we use C#, and the example code generated in the Azure ML Portal looks like:

        using (var client = new HttpClient(handler))

        {
            // Request data goes here
            var scoreRequest = new Dictionary<string, List<Dictionary<string, string>>>()
            {
                {
                    "data",
                    new List<Dictionary<string, string>>()
                    {
                        new Dictionary<string, string>()
                        {

This uses a Dictionary for the properties, but the order of elements in a C# Dictionary is undefined and it does not guarantee the order when reading elements from it.
So the generated/example C# code is prone to errors.

Is this a known issue? Or am I missing something here?

Thanks in advance!