Azure AI Foundry Agent Hallucinating When AI Search Index Connected

My current pipeline consists of using the azure document intelligence read model to process large pdf documents, whose data is then moved into an azure AI Search Index, which I then want to directly connect to an AI foundry agent to then be queried upon. The search index is functioning perfectly with clean, well-structured data, but the foundry agent consistently hallucinates document names and information instead of using the actual search results from the index.

Index Setup:

Fields:

• chunk_id (String, key)
• source_document (String, searchable, filterable)
• blob_path (String, searchable, filterable)
• document_id (String, filterable)
• page_numbers (String, searchable, filterable)
• start_page (Int32, filterable, sortable)
• end_page (Int32, filterable, sortable)
• chunk_text (String, searchable)
• main content field
• total_pages_in_document (Int32)
• processed_date (DateTimeOffset)

Semantic Configuration: Enabled

• Title field: source_document
• Content fields: chunk_text, page_numbers

Through direct testing, I can see that the index is functioning correctly, and is able to find the correct documents and correct content, it is only when connected directly to the foundry agent that it stops working.

Field Mappings in Azure AI Foundry:

• Content field: chunk_text
• Title field: source_document
• URL field: blob_path

Would also appreciate guidance on:

1. The correct settings/configuration to enforce strict grounding
2. How to prevent the agent from using its training data when a search index is connected
3. Best practices for pharmaceutical use cases requiring perfect accuracy
4. Whether we should be using a different Azure service/API for this use case

If more context is needed please let me know.