Responses API intermittently never responds (no headers at all) when the request contains a PDF input_file

Silvan Kurth 0 Reputation points
2026-11-18T15:23:10.28+00:00

Some streaming calls to the Azure OpenAI Responses API that include a PDF as input_file get no response at all. No HTTP status and no response headers arrive, the connection stays open, and nothing comes back within 150 seconds. Sending the identical request again right away succeeds in about 1-3 seconds. Without the PDF the same requests never hang.

Setup

  • Microsoft Foundry / Azure OpenAI resource in Switzerland North
  • Endpoint: https://{resource}.services.ai.azure.com/openai/v1/responses
  • Deployments: gpt-6.1-terra (main tests), also seen with gpt-6-luna
  • Happens both directly against the Foundry endpoint (Entra ID token) and through API Management. APIM gateway logs show the request was forwarded within ~20 ms and the backend never answered (BackendTime ~100 s, ResponseCode 0, ClientConnectionFailure after our client gave up).

Minimal request

POST /openai/v1/responses

{

"model": "gpt-6.1-terra",

"input": [{ "type": "message", "role": "user", "content": [

{ "type": "input_file", "filename": "sample.pdf", "file_data": "data:application/pdf;base64,..." },

{ "type": "input_text", "text": "How many pages does the document have? Answer with one number." }

]}],

"reasoning": { "effort": "high" },

"stream": true,

"store": false

}

This follows the documented format for PDF input. We checked that the PDF bytes arrive unchanged (byte-identical to the original file).

Measurements (sequential requests, client timeout 130 s)

  • PDF A (Word for Microsoft 365 export, 2 pages, 1 image, ~555 KB): 16 of 200 requests hung (8 %)
  • PDF A with a 30 s pause between requests, single client, no other load: 2 of 40 hung, so this does not look like a rate limit (a rate limit would return 429 right away)
  • PDF A re-written with pypdf (clean structure, 140 KB): 1 of 40 hung
  • PDF B (PDFlib invoice, 2 pages, text only, 33 KB): 1 of 50 hung
  • PDF C (DevExpress report, 1 page, 3 images, 82 KB): 0 of 50 hung
  • Small generated text-only PDF (10 KB): 0 of 286 hung, including requests sent at the same time as PDF A
  • Large requests without any PDF (up to ~313k tokens / 1 MB): 0 of 280 hung
  • Other attachment types through the same gateway (JPG, PNG, DOCX): 0 of 40 hung

Sometimes several PDF requests in flight at the same moment hang together, even with different PDFs, while requests one second earlier or later succeed. That looks like an intermittent stall in the backend's PDF processing.

Example hung requests (UTC start time, x-ms-client-request-id we sent)

  • 2026-11-18T10:44:56.031Z abdddc9f-01ea-47aa-8ab4-5269e28cd59c
  • 2026-11-18T10:44:56.207Z 169b6ad3-6c4c-4194-9fb9-a925b52386f6 (different PDF, same moment)
  • 2026-11-18T11:56:35.073Z 62adfad7-156c-4e71-a3a8-2b1edb8cfc11 (30 s pause run)

Questions

  1. Is this a known issue with PDF input_file processing in the Responses API?
  2. Shouldn't the service end such requests with an error status instead of holding the connection open without any response?
  3. Would uploading the PDF via the Files API and passing file_id avoid this, or is the same processing path used?

Current workaround: we cancel streaming requests that receive no response headers within 15 s and retry, which makes it mostly invisible to users.

I can share the reproducing PDF privately if someone from Microsoft wants to investigate.

Azure OpenAI in Foundry Models
0 comments No comments

1 answer

Sort by: Most helpful
  1. Oluwayemi Aina 0 Reputation points
    2026-11-18T17:36:03.0533333+00:00

    Based on the evidence you collected, this does not appear to be a client-side or rate-limiting issue. The symptoms point more strongly towards an intermittent issue within Azure OpenAI's backend PDF-processing pipeline.

    A few observations support this:

    • APIM logs confirm that requests reach Azure OpenAI successfully, with approximately 20 ms forwarding time.
    • Failed requests return neither response headers nor conventional 4xx or 5xx errors.

    The same request often succeeds immediately on retry.

    The issue is predominantly associated with PDFs, not DOCX, images, large prompts, or ordinary text.

    Multiple PDF requests hanging simultaneously suggest a shared backend dependency.

    The official file-input documentation confirms that PDFs undergo additional processing, including text extraction and page-image processing.

    1. Is this a known issue?

    From my findings, there is no publicly documented Microsoft advisory describing this exact behaviour.

    However, the symptoms suggest a possible backend ingestion or PDF-processing bottleneck rather than a model inference issue. Given the reproducible timestamps, request IDs, failure rates, and APIM traces, there is sufficient evidence to justify a Microsoft support investigation.

    2. Should the service return an error?

    Yes. Ideally, the request should return a successful response, a structured 4xx/5xx error, or a timeout.

    Keeping a connection open for several minutes without returning headers is abnormal production API behaviour and warrants investigation.

    3. Would using file_id help?

    Possibly, but it is not guaranteed.

    If the problem occurs during Base64 decoding or request-body handling, file_id could help. However, if the bottleneck is PDF parsing, text extraction, or page rendering, both approaches may encounter the same issue.

    I would recommend testing both approaches using the same PDF, prompt, and model.

    4. Recommended next steps

    1. Explicitly test PDF detail: low to assess the impact of page-image processing.
    2. Compare file_data and file_id using identical requests.

    Test with at least 100 runs per approach to establish a meaningful failure rate.

    Raise a Microsoft support ticket with UTC timestamps, deployment name, Switzerland North region, request IDs, APIM traces, and a sample PDF.

    My final assessment

    My strongest suspicion is an intermittent backend PDF-processing bottleneck. The fact that multiple PDFs can hang simultaneously while requests immediately before and after succeed makes a client-side issue less likely.

    I would prioritise escalating this to Microsoft while testing file_id as a potential workaround. This should help establish whether the problem occurs during request ingestion or deeper within the PDF-processing pipeline.

    Note: This remains a working hypothesis, not a confirmed Microsoft service defect. Based on the evidence collected, this does not appear to be a client-side or rate-limiting issue. The symptoms point more strongly towards an intermittent issue within Azure OpenAI's backend PDF-processing pipeline.

    Was this answer helpful?

    0 comments No comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.