Custom federated connector (MCP): image tool results reach the model double base64-encoded as text

Michael Anthony 0 Reputation points
2026-11-04T07:43:18.8066667+00:00

Summary

When a custom federated connector's MCP tool returns an image, Microsoft 365 Copilot does not pass it to the model as an image. The model gets the image data as text, and that text has been base64-encoded a second time. Copilot therefore can't see or describe any image returned by an MCP tool.

The same MCP server and tool work correctly in Copilot Studio: the model receives the image and can describe it.

Environment

  • Microsoft 365 Copilot Chat, with a custom federated connector (Copilot > Connectors > Create a new connector > Connect to MCP server)
  • Remote MCP server on Azure Functions (MCP extension), Streamable HTTP transport
  • The same server connected to a Copilot Studio agent, for comparison

What the MCP server returns

A standard MCP CallToolResult with a text block followed by an image block:

{

  "content": [

    { "type": "text", "text": "{ ...attachment details... }" },

    { "type": "image", "mimeType": "image/png", "data": "iVBORw0KGgoAAAANSUhEUg..." }

  ]

}

The data field is base64-encoded once, as the MCP specification requires. We checked the raw tool result directly: it starts iVBORw0KGgo, the base64 of the PNG file signature.

Steps to reproduce

  1. Connect the custom federated connector in Copilot Chat (Settings > Sources > Connect).
  2. Ask Copilot to retrieve an image through a tool that returns an MCP image block (in our case, a PNG attachment of about 8 KB).
  3. Ask Copilot to describe the image.
  4. Then ask Copilot to quote only the first 10 characters of the image data it received, without running any code.

Expected result

The model receives the image as an image and describes it, as it does in Copilot Studio with the same server and tool.

Actual result

  • Copilot says it received a long base64 text string, not an image, and can't view it.
  • Asked for the first 10 characters of the data, Copilot replies aVZCT1J3ME.
  • aVZCT1J3MEtHZ28 is the base64 of iVBORw0KGgo. So the model receives our already base64-encoded data encoded a second time.
  • The text the model receives is about 1.78 times the image size (4/3 × 4/3). An 8,231-byte PNG becomes about 14,600 characters.
  • Tool calls returning several images time out: about 800 KB of images became about 1.4 MB of text.

Comparison

| Client | Same MCP server and tool | Result |

|---|---|---|

| Copilot Studio agent | Yes | Image received as an image; model describes it correctly |

| Microsoft 365 Copilot (custom federated connector) | Yes | Image received as double base64-encoded text; model can't see it |

A similar open report in another Copilot client, where MCP tool images reach the model as base64 text: https://github.com/microsoft/copilot-for-eclipse/issues/465

The federated connector documentation doesn't say which MCP tool-result content types (text, image, resource) are supported.

Questions

  1. Is this a known issue with custom federated connectors?
  2. Are MCP image content blocks supposed to be supported in federated connector tool results?
Microsoft Copilot | Microsoft 365 Copilot | Development
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest

  1. AI answer

    2026-11-04T07:44:30.6566667+00:00

    Treat this as a Microsoft 365 Copilot federated-connector host limitation or defect and open a Microsoft support case; do not base64-decode the payload before returning it, because your current CallToolResult already has the expected single-encoded MCP image representation.

    Answers

    1. Is this a known issue?
      It is not currently documented as a known Microsoft 365 Copilot extensibility issue. The published known-issues page does not list MCP image blocks being converted to text or encoded twice ^1^. There is an open report for the same general failure mode in GitHub Copilot for Eclipse—an MCP image result reaches the model as base64 text rather than image input—but that concerns a different Copilot client and does not establish Microsoft 365 Copilot behavior ^2^.
    2. Are image content blocks supported in federated-connector tool results?
      The current custom federated-connector documentation confirms that these connectors invoke tools exposed by an MCP server, but it does not explicitly document image as a supported model-visible tool-result content type ^3^. Therefore, image-block support for this specific route cannot currently be confirmed from the published contract. Copilot Studio accepting the same response establishes that the server produces a usable result for that client. It does not guarantee that the Microsoft 365 Copilot federated-connector host implements the same content-type handling.

    Submit a Microsoft 365 support case with:

    • Tenant and connector identifiers, but no credentials or image contents containing sensitive data.
    • UTC timestamps for two minimal reproductions.
    • The prompt and resulting Copilot response.
    • The raw MCP response captured at the server boundary.
    • HTTP response headers and Streamable HTTP transport details.
    • A tiny synthetic PNG rather than production data.
    • The comparison result from Copilot Studio.
    • These byte/string checkpoints:
      • PNG bytes begin 89 50 4E 47 0D 0A 1A 0A.
      • The server’s single-base64 value begins iVBORw0KGgo.
      • Copilot-observed text begins aVZCT1J3ME....
    • Payload sizes before encoding, after the required MCP encoding, and as observed by the model.
    • Confirmation that a text-only result from the same tool succeeds.

    Ask support to confirm:

    • Whether content[].type: "image" is supported for custom federated connector tool results.
    • Whether the connector pipeline serializes data as ordinary text before constructing model input.
    • Whether this behavior has an incident or bug ID.
    • Whether any supported capability declaration is required for multimodal tool output.

    Safe interim workaround

    Until Microsoft confirms support, avoid returning image base64 through this connector:

    1. Return a concise text description, OCR output, or extracted metadata in the model-visible content.
    2. Keep the binary in your source system and return a stable identifier or access-controlled reference for retrieval outside model vision.
    3. Limit the tool to one small result and paginate collections to avoid the observed payload expansion and timeouts.

    For MCP apps with widgets—not necessarily the same custom federated-connector path—Microsoft recommends separating model-visible summaries from UI-only data: put concise information in content and widget-specific data in _meta. It also warns against excessively large content or structuredContent responses ^4^. This may provide a presentation workaround, but it does not make the image available to the model’s vision input.

    Do not return raw PNG bytes in the JSON data field or pre-decode the required base64. That would make the MCP response invalid rather than compensate reliably for client-side handling.

    References

    1. Known Issues in Microsoft 365 Copilot Extensibility
    2. #465 [Bug] MCP tool results containing image content are sent to the model as Base64 text instead of as images
    3. Set up custom federated connectors - Microsoft 365 Copilot connectors
    4. Troubleshoot MCP apps in Microsoft 365 Copilot
    AI-generated content may be incorrect. Read our transparency notes for more information.

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.