Azure RAG fails to analyze full 10K feedback dataset—often returns "No info." Need help ensuring complete data coverage in results.

Vishav Singh | CHECKER 0 Reputation points
2025-05-12T04:32:12.29+00:00

We are developing a solution that enables users to search and analyze large datasets, such as 10,000 feedback reports containing branch info, question/answer data, scores, and open-ended comments. We’ve implemented Azure OpenAI with a Retrieval-Augmented Generation (RAG) model, using both vector and non-vector indexes, semantic, keyword, and hybrid search. While the index is successfully created, responses often return “No information available,” raising concerns about whether the model is retrieving and analyzing the entire dataset or only a subset. We’ve tested with various prompt formats and configurations but still face the same issue. The goal is to ensure that users can perform deep insight analysis across the full dataset without missing data or receiving inaccurate responses. We are seeking support to validate that our implementation enables complete dataset coverage and to help improve the reliability of the returned insights.

I was thinking that the problem is on my code level but when when I tried to use same indexing in Azure Chat Playground, getting the partially responses. Same result as I get using my coding.

@ms support team or @anyone can help us to check and fix the issue before we decide to move to another solution?

Regards

Vishav Deep Singh

Foundry Tools
Foundry Tools

Formerly known as Azure AI Services or Azure Cognitive Services is a unified collection of prebuilt AI capabilities within the Microsoft Foundry platform

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Deleted

    This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

    4 deleted comments

    Comments have been turned off. Learn more

  2. Vishav Singh | CHECKER 0 Reputation points
    2025-05-21T08:00:17.87+00:00

    @JAYA SHANKAR G S

    We are implementing your recommendation to leverage the Azure OpenAI Assistant for improved response generation, and we've certainly observed that it delivers superior responses compared to our current Chat Completion RAG model.

    However, we're encountering significant challenges with the Azure OpenAI Assistant, primarily related to file management and search accuracy:

    File Size and Token Limitations: A major hurdle is managing file size and token limitations. We are forced to develop custom logic to validate content even for single lines before writing, as failing to do so leads to indexing failures and disrupts the assistant's search functionality. This adds considerable overhead to our development process.

    Vector Store Limitations: The current limit of 10,000 files per vector store presents a significant scaling issue. Once this limit is reached for a project, we would need to create and manage additional vector stores for new files. Unfortunately, the Azure OpenAI Assistant can only be linked to a single vector store at a time, making it impractical for projects with large document sets.

    Agent Search Behavior: We explored using Agent, which offers the ability to link multiple indexes. However, the Agent's search behavior is currently problematic. It tends to retrieve information "out of the box," returning responses that are not confined to the linked knowledge index documents. For instance, when asked "How many questions do we have in Japan that has parking space?", the Agent provided a live count in the thousands, despite our linked index documents only containing one hotel in Japan.

    We need to understand if there are any specific settings or configurations for the Agent that would restrict its search to only the linked index documents, thereby preventing it from fetching information from external sources.

    Addressing these challenges is critical for the successful and scalable implementation of the Azure OpenAI Assistant. We look forward to your guidance on these matters.

    Regards

    Vishav Deep Singh

    Was this answer helpful?

    0 comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.