Microsoft-Decision-1 context limit does not match the documented 32K

Question

Microsoft-Decision-1 context limit does not match the documented 32K

Billy Zhou 50

Hello Microsoft Support,

1.Issue

The Microsoft Foundry catalog lists a 32,768-token context window for Microsoft-Decision-1. In my test, a request with 130,030 repeated characters succeeds and reports 16,354 input tokens, while increasing the input by just 10 characters causes the request to fail with an error saying the question prompt exceeds the 32,768-token context limit.

This appears inconsistent with the reported token usage. Could you clarify how the endpoint calculates the context limit, and whether that calculation includes tokens not included in 'usage.input_tokens'?

2.Test method

I sent requests to the 'systemone' endpoint using the same payload and fixed question criteria. The only variable was the number of repeated 'A' characters appended to the question instructions: 130,020, 130,030, and 130,040.

Repeated A charactersResult130,020Success; usage.input_tokens: 16,352130,030Success; usage.input_tokens: 16,354130,040HTTP 422The 130,040-character request returned:

{"detail":"invalid model request: Question prompt exceeds the 32768-token context limit including the decision token"}

3.Test code (The endpoint, deployment name, and API key are masked):

import requests

ENDPOINT = "https://<resource-name>.services.ai.azure.com"

API_KEY = "<redacted>"

DEPLOYMENT_NAME = "<redacted>"

def mock_context(length: int) -> str:


if length <= 0:

    return ""

return "A" * length
```def main(query, length):

```python
mock = mock_context(length)

payload = {

    "model": DEPLOYMENT_NAME,

    "state": query,

    "questions": {

        "parity": {

            "type": "choice",

            "instructions": (

                "Classify the input as odd, even, or other. "

                "Use odd/even only when the input clearly refers to an integer; "

                "otherwise use other. This is extraneous context, please ignore it: "

                f"{mock}"

            ),

            "criteria": {

                "odd": "The input is an odd integer.",

                "even": "The input is an even integer.",

                "other": (

                    "The input is not clearly an integer or is unrelated "

                    "to odd/even classification."

                ),

            },

        }

    },

}

response = requests.post(

    f"{ENDPOINT}/providers/microsoft/v1/systemone",

    headers={"Content-Type": "application/json", "api-key": API_KEY},

    json=payload,

    timeout=60,

)

if not response.ok:

    print(

        f"length: {length} status: {response.status_code}. "

        f"response body: {response.text}"

    )

    response.raise_for_status()

response_json = response.json()

print(f"length: {length}, token usage is {response_json['usage']['input_tokens']}")
```if __name__ == "__main__":

```r
main("1", 130020)

main("1", 130030)

main("1", 130040)
```**Logs:** 

length: 130020, token usage is 16352

length: 130030, token usage is 16354

length: 130040 status: 422. response body: {"detail":"invalid model request: Question prompt exceeds the 32768-token context limit including the decision token"}

0 comments

Answer accepted by question author

0 additional answers

Your answer

Answer 1

Hi Billy Zhou & thx for join me at Q&A platform,

an interesting reproduction, and the 10-character difference between a successful request and HTTP 422 makes it particularly useful for investigating the limit. I agree that the reported usage.input_tokens doesn't explain the rejection. Your successful request reports 16,354 input tokens, which is roughly half the advertised 32,768-token context window.

One possibility worth investigating is that the endpoint's request validation and its reported token usage aren't using exactly the same counting method. There's a numerical clue here. Your 130,040 repeated characters correspond to approximately 32,510 tokens if a conservative four-characters-per-token estimate is being used. That's very close to the 32,768 limit once the fixed question instructions, criteria, and internal formatting are included.

Meanwhile, the successful response reports around 16,354 actual input tokens. That could indicate a difference between an estimated pre-validation token count and the tokenizer used for execution or usage reporting. However, this is a hypothesis, not confirmed Microsoft-Decision-1 behavior. Another possibility is that the service constructs an expanded internal question prompt or reserves additional context that isn't reflected in usage.input_tokens.

The error's reference to the decision token doesn't, by itself, explain a difference of approximately 16,000 tokens.

I'd suggest two targeted comparisons before escalating further. First, keep the total character count approximately the same but replace the repeated A characters with ordinary natural-language text or less repetitive content. If the failure threshold stays near the same character count while reported token usage changes substantially, that would support the possibility of a character-based or estimated-token validation limit.

Second, keep the same repeated text but move it from questions.parity.instructions into state, leaving the question instructions short. If the acceptance threshold changes significantly, that would suggest the endpoint applies additional processing or limits specifically to question prompts.

Neither test requires changing the deployment or region. The public Microsoft-Decision-1 documentation describes the Decision API and its token usage response, but I couldn't find a documented explanation of how the 32,768-token validation limit relates to internal prompt construction or usage.input_tokens.

I'd raise this with Microsoft Foundry model support and include the three exact payloads, deployment and model version, UTC timestamps, and request IDs. The specific engineering question should be whether context validation uses a different tokenizer or token estimate from usage accounting, and whether additional internal tokens are included in the context limit but excluded from the reported input usage.

Until Microsoft confirms the implementation, I wouldn't assume that usage.input_tokens can be used to predict the maximum accepted request size for this model.

Ur test provides a good, reproducible case for either clarifying the documented limit or identifying an inconsistency in the endpoint's validation logic.

rgds,

Alex

p.s. if my answer is help - pls mark it as an answer

Billy Zhou 50 Reputation points

2026-10-28T01:26:52.5333333+00:00

It works after I put the mocked context into the "state" field. Thanks for your reply.