An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
Based on the evidence you collected, this does not appear to be a client-side or rate-limiting issue. The symptoms point more strongly towards an intermittent issue within Azure OpenAI's backend PDF-processing pipeline.
A few observations support this:
- APIM logs confirm that requests reach Azure OpenAI successfully, with approximately 20 ms forwarding time.
- Failed requests return neither response headers nor conventional 4xx or 5xx errors.
The same request often succeeds immediately on retry.
The issue is predominantly associated with PDFs, not DOCX, images, large prompts, or ordinary text.
Multiple PDF requests hanging simultaneously suggest a shared backend dependency.
The official file-input documentation confirms that PDFs undergo additional processing, including text extraction and page-image processing.
1. Is this a known issue?
From my findings, there is no publicly documented Microsoft advisory describing this exact behaviour.
However, the symptoms suggest a possible backend ingestion or PDF-processing bottleneck rather than a model inference issue. Given the reproducible timestamps, request IDs, failure rates, and APIM traces, there is sufficient evidence to justify a Microsoft support investigation.
2. Should the service return an error?
Yes. Ideally, the request should return a successful response, a structured 4xx/5xx error, or a timeout.
Keeping a connection open for several minutes without returning headers is abnormal production API behaviour and warrants investigation.
3. Would using file_id help?
Possibly, but it is not guaranteed.
If the problem occurs during Base64 decoding or request-body handling, file_id could help. However, if the bottleneck is PDF parsing, text extraction, or page rendering, both approaches may encounter the same issue.
I would recommend testing both approaches using the same PDF, prompt, and model.
4. Recommended next steps
- Explicitly test PDF detail: low to assess the impact of page-image processing.
- Compare file_data and file_id using identical requests.
Test with at least 100 runs per approach to establish a meaningful failure rate.
Raise a Microsoft support ticket with UTC timestamps, deployment name, Switzerland North region, request IDs, APIM traces, and a sample PDF.
My final assessment
My strongest suspicion is an intermittent backend PDF-processing bottleneck. The fact that multiple PDFs can hang simultaneously while requests immediately before and after succeed makes a client-side issue less likely.
I would prioritise escalating this to Microsoft while testing file_id as a potential workaround. This should help establish whether the problem occurs during request ingestion or deeper within the PDF-processing pipeline.
Note: This remains a working hypothesis, not a confirmed Microsoft service defect. Based on the evidence collected, this does not appear to be a client-side or rate-limiting issue. The symptoms point more strongly towards an intermittent issue within Azure OpenAI's backend PDF-processing pipeline.