Information retrieval not working in my agents.

Mats van Borre 0 Reputation points
2026-10-07T08:49:16.0366667+00:00

Hi,

I am building several knowledge-based agents in Microsoft Copilot Studio using connected document sources.

Across multiple agents, I am seeing the same recurring problem: the correct information exists in the connected sources, but Copilot Studio does not consistently retrieve all relevant passages needed to answer a question completely.

The issue appears to be retrieval completeness rather than answer generation itself.

Typical pattern:

  • The user asks a question that requires information from one or more connected documents.
  • Copilot retrieves some relevant passages.
  • Other relevant passages that are also present in the connected sources are not retrieved.
  • The model then produces an incomplete, overly cautious, or incorrect answer based on the partial context it received.
  • If the missing concept is searched more directly or with different terminology, the relevant passage can often be found.

This behaviour occurs across different agents and different document sets, not just one specific source.

I have already tried:

  • detailed descriptions for knowledge sources;
  • instructions to search synonyms and alternative terminology;
  • instructions to perform additional searches when the first search is insufficient;
  • instructions to consider all relevant roles, conditions, exceptions and applicability;
  • instructions not to stop after finding only one part of the answer;
  • stricter source-use and comparison instructions.

These changes sometimes improve results, but they do not reliably solve the retrieval issue.

In the activity trace, I can often see that Copilot generates a refined knowledge query, but that query retrieves only part of the information needed. Sometimes the correct source is searched, but the relevant passage from that source is still not returned.

My questions are:

  1. Is there a supported way to improve retrieval completeness in Copilot Studio?
  2. Can the automatically generated/refined knowledge query be controlled or influenced more directly?
  3. Can an agent be forced to perform multiple independent knowledge searches for one question and then combine the retrieved results?
  4. Is there a way to increase the number or diversity of retrieved chunks?
  5. Can Copilot Studio be configured to make sure multiple relevant knowledge sources contribute to the answer instead of stopping after the first strong match?
  6. Would Azure AI Search provide more control over retrieval behaviour than the standard Copilot Studio knowledge system?
  7. Are there recommended Microsoft architectures or best practices for knowledge-based agents where completeness across multiple documents is important?

The main issue is that the information is present in the knowledge sources, but the retrieval layer does not consistently provide all relevant context to the model.

I would like to understand whether this is an expected limitation of the current Copilot Studio knowledge retrieval system, or whether there are configuration options or recommended patterns that can improve this.

Thanks.

Microsoft Copilot | Other
0 comments No comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.