API call to RAG-based service

Question

API call to RAG-based service

Steven Kent 40

I built a RAG-based OpenAI service that uses data I uploaded to AI Portal. It works when I am testing from the chat playground.

Now I want to make the same exact thing callable from Postman. I am able to call the service from Postman. The problem I am having is the API call does not take advantage of the RAG pattern I set up.

How do I get the API that I call to do exactly what the chat playground is already doing?

0 comments

2 answers

Your answer

Answer 1

Deleted

This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

1 deleted comment

Comments have been turned off. Learn more

Answer 2

Hello Steven Kent,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that your RAG-based OpenAI service works in the chat playground but does not function as expected when called via API (Postman).

By following the below steps, you will be able to resolve it.

Step 1: Compare the API Request in the Playground and Postman

Open the browser’s DevTools (F12 in Chrome) or any of your browser and go to the Network tab.
Make a request in the chat playground and find the actual API request being sent to OpenAI.
Copy the full request payload, including headers, and compare it with your Postman request.

Step 2: Ensure RAG-Specific Parameters Are Present

If you are using Azure OpenAI’s on-your-data feature, the API request must include:

"data_sources": [
  {
    "type": "azure_search",
    "parameters": {
      "endpoint": "YOUR_AZURE_SEARCH_ENDPOINT",
      "index_name": "YOUR_INDEX_NAME",
      "authentication": {
        "type": "api_key",
        "key": "YOUR_SEARCH_API_KEY"
      }
    }
  }
]

Ensure that your payload in Postman includes this section.

Step 3: Check for Missing System Messages or Parameters

Some RAG-based services require system messages or specific parameters. For an example:

"messages": [
  {"role": "system", "content": "You are an AI assistant retrieving relevant knowledge."},
  {"role": "user", "content": "What are my available health plans?"}
]

NOTE: Copy the exact structure of messages from the playground.

Step 4: Validate Headers and API Version

Make sure you are using the correct API version (e.g., api-version=2024-10-21) and include all required headers:

{
  "Authorization": "Bearer YOUR_OPENAI_API_KEY",
  "Content-Type": "application/json"
}

Step 5: Debug the API Response

Compare the API responses from Postman and the chat playground.
If the API response is different, check for missing retrieval evidence (e.g., lack of "context" or "documents" in the response).
If the response does not contain retrieved documents, your RAG setup might not be properly configured.

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.