How to add a vectorizer for a search index in C#

Question

How to add a vectorizer for a search index in C#

Thibault Verlinde 100

Hiya

I'm currently working on a project that indexes a certain amount of data. This goes flawlessly when I create the index, with vectors, inside the portal.
Whenever I create such index with a C# program, it succeeds, but without the vectorizer. There isn't also any info on how/where you add a Vectorizer inside the new index with that code.

Thanks in advance!

EDIT: I found the answer myself, but for some reason i can't accept my own answer. See the answer posted by me to find the solution.

My code:

//Initialisation of new index
var indexName = _srchClient.IndexName;
var vectorSearchConfigName = "vectorconfig-" + _options.ProjectName;
var vectorSearchProfileName = "vectorprofile-" + _options.ProjectName;
//var semanticConfigName = "semanticconfig-" + _options.ProjectName;
var index = new SearchIndex(indexName)
{
    VectorSearch = new()
    {
        Algorithms =
        {
            new HnswAlgorithmConfiguration(vectorSearchConfigName)
        },
        Profiles =
        {
            new VectorSearchProfile(vectorSearchProfileName, vectorSearchConfigName)
        }
    },
    Fields =
    {
        new SimpleField("id", SearchFieldDataType.String) { IsKey = true },
        new SearchableField("content") { AnalyzerName = LexicalAnalyzerName.StandardLucene },
        new SimpleField("sourcePage", SearchFieldDataType.String) {IsFacetable = true},
        new SimpleField("sourceFile", SearchFieldDataType.String) {IsFacetable = true},
        new SearchField("embedding", SearchFieldDataType.Collection(SearchFieldDataType.Single))
        {
            VectorSearchDimensions = 1536,
            IsSearchable = true,
            VectorSearchProfileName = vectorSearchProfileName
        }
    },
    //When using free tier, this can't be used!
    //SemanticSearch = new()
    //{
    //    Configurations =
    //    {
    //        new SemanticConfiguration(semanticConfigName, new()
    //        {
    //            ContentFields =
    //            {
    //                new SemanticField("content")
    //            }
    //        })
    //    }
    //}
};
// Create a new search index
await _srchIndexClient.CreateIndexAsync(index);

0 comments

2 answers

Your answer

Answer 1

Deleted

This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

Comments have been turned off. Learn more

Answer 2

Hey there Thibault Verlinde

Thats a good question thanks for using QandA platform

So i guess you need to define and configure the vectorizer within the SearchIndex object before creating the index.

Lemme modify your code to include a vectorizer, below

var indexName = _srchClient.IndexName;
var vectorSearchConfigName = "vectorconfig-" + _options.ProjectName;
var vectorSearchProfileName = "vectorprofile-" + _options.ProjectName;

var vectorizer = new SearchIndexVectorizer
{
    Name = vectorSearchConfigName,
    Source = "/document/content", // Specify the source field for vectorization
    Tokenizer = new LexicalTokenizer(LexicalTokenizerName.StandardLucene),
    OdataType = "#Microsoft.Azure.Search.StandardAnalyzer"
};
var index = new SearchIndex(indexName)
{
    VectorSearch = new()
    {
        Algorithms =
        {
            new HnswAlgorithmConfiguration(vectorSearchConfigName)
        },
        Profiles =
        {
            new VectorSearchProfile(vectorSearchProfileName, vectorSearchConfigName)
        }
    },
    Fields =
    {
        new SimpleField("id", SearchFieldDataType.String) { IsKey = true },
        new SearchableField("content") { AnalyzerName = LexicalAnalyzerName.StandardLucene },
        new SimpleField("sourcePage", SearchFieldDataType.String) {IsFacetable = true},
        new SimpleField("sourceFile", SearchFieldDataType.String) {IsFacetable = true},
        new SearchField("embedding", SearchFieldDataType.Collection(SearchFieldDataType.Single))
        {
            VectorSearchDimensions = 1536,
            IsSearchable = true,
            VectorSearchProfileName = vectorSearchProfileName
        }
    },
    Vectorizers = { vectorizer } // Add the vectorizer to the index
};
// Now ccreate a new search index
await _srchIndexClient.CreateIndexAsync(index);

If this helps kindly accept the answer thanks much.

Thibault Verlinde 100 Reputation points

2024-03-07T15:19:21.3933333+00:00

Hiya @Azar

Thanks for your quick answer!
I'm currently working on the nuget package of Azure Search Documents, version 11.6.0-beta.3, but I don't have any implementation of SearchIndexVectorizer... My only option is a AzureOpenAIVectorizer, but with only a name, what is also weird.
FYI: I'm not using a skillset or anything.

I read that it has to do something with AzureOpenAIParameters, but I'm not sure if that's the case/how to implement it.
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Ajay Kumar N 28,216 Reputation points Microsoft Employee Moderator

2024-03-11T11:17:17.5233333+00:00

Thibault Verlinde, Glad to know that you had figured out the issue. Thanks for sharing the solution with the community.

As Azar mentioned, since the Microsoft Q&A community has a policy that the question author cannot accept their own answer. They can only accept answers by others, please consider to Accept the answer summarized, it will benefit the community find the answers quickly. Much appreciate your feedback and update.