How does Microsoft Foundry Agent Service handle conversation state?

Wade Gausden 166 Reputation points
2026-09-20T06:26:02.6433333+00:00

The question is from a small knowledge check on the training "Develop AI agents with Microsoft Foundry and Visual Studio Code"

User's image

The answer in this case is option C, that the Responses API automatically manages conversation context.

However, and I could be mixing up resources or terms, in the training one step earlier it has the following :

User's image

In the Lab, we use a Conversation_Id between turns to track state, which seems to work just fine. So........ I guess the API is "stateless" as we have to pass the conversation id back each time, but then it's also confusing in the documentation to say we must store conversation history when that isn't realllyyy the case.

Foundry Agent Service
Foundry Agent Service

A fully managed platform in Microsoft Foundry for hosting, scaling, and securing AI agents built with any supported framework or model

0 comments No comments

2 answers

Sort by: Most helpful
  1. Venkata Tarun Kumar Mavillapalli 0 Reputation points
    2026-11-18T12:12:06.6833333+00:00

    Hi Wade, did you manage to clarify this? If not, the key detail in your second screenshot is “Agent Application endpoints.”

    When you reuse a conversation ID in the lab, Foundry stores the conversation history and uses it for subsequent turns. Your application retains the ID to identify that conversation; it doesn’t need to store and resend the full transcript. Passing an ID on each request doesn’t make the history client-managed.

    The statement in your screenshot describes the legacy Agent Application endpoint. Microsoft documents that endpoint as supporting only stateless POST /responses, with /conversations unavailable. For that endpoint, the client needs to maintain the history for multi-turn interactions.

    That explains the apparent contradiction: the lab uses service-managed conversation history, while the screenshot describes a limitation of a particular published endpoint.

    These two Microsoft references explain the distinction:

    The second article is now marked as the legacy publishing experience, so that restriction should be read in that specific context.

    Was this answer helpful?

    0 comments No comments

  2. AI answer

    2026-09-20T06:26:27.6333333+00:00

    Both statements can be true because Foundry Agent Service separates responses, conversations, and sessions.

    • A response is one unit of execution.
    • A conversation persists history across turns.
    • A session persists sandbox state such as $HOME and uploaded files.

    For Responses protocol agents:

    • Conversation continuity does not come from agent_session_id.
    • It comes from either previous_response_id or a conversation ID.
    • If a conversation ID is used, the platform stores the message history for that conversation and reuses a stable session automatically.

    That is why the lab works when passing conversation_id between turns: the conversation history is being persisted by the service under that conversation ID.

    The part that says the Responses API is stateless refers to the call pattern, not to whether the platform can store history. A single responses.create call does not automatically know prior turns unless the application supplies a threading mechanism such as:

    • previous_response_id, or
    • conversation

    So the practical distinction is:

    • Without previous_response_id or conversation_id: each call is effectively standalone.
    • With conversation_id: the platform-managed conversation history is reused across turns.

    For Invocations protocol agents, the platform does not store conversation history; the container code manages state itself. In that model, the statement about storing conversation history in the client or application is accurate.

    So the clean interpretation is:

    • The quiz answer is correct for the Responses-based conversation model: Foundry can automatically manage conversation context when a conversation is used.
    • The “stateless Responses API” wording means each request must still reference prior context explicitly; the service does not infer it from the session alone.
    • The “store conversation history in your client” guidance applies when using stateless response chaining or when not using platform-managed conversations.

    AI-generated content may be incorrect. Read our transparency notes for more information.

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.