Embedding model migration best practices guide for Foundry IQ (Azure AI Search)

Note

Foundry IQ (Azure AI Search) provides direct indexing and query APIs alongside higher-level knowledge source and knowledge base APIs for managed retrieval. Use the Azure portal, REST APIs, or Azure SDKs to configure these capabilities, or connect agents to knowledge bases in the Microsoft Foundry portal.

An embedding model migration changes how your Foundry IQ (Azure AI Search) application represents documents and queries as vectors. You might migrate because a model retires or because your requirements for relevance, language coverage, latency, throughput, cost, or data residency change. Evaluate replacement models against your workload rather than assuming that a newer model is better.

This best practices guide helps you choose a migration approach, regenerate embeddings, validate results, and switch production traffic. For most production migrations, build and test a separate target index while retaining the existing index. For a same-index migration, add a new vector field and retain the old field rather than overwriting it; keep both fields through validation and the rollback window. Your source data, freshness requirements, available capacity, and tolerance for interruption determine how to populate and switch to the new index or field.

Prerequisites

  • An existing vector or hybrid-search application, including access to its index schema, ingestion configuration, and query configuration.
  • Authoritative source data that you can reprocess, or a complete export of the indexed content needed for migration.
  • A replacement embedding model deployment with verified document and query interfaces. For models in Microsoft Foundry, review the current retirement schedule.
  • Permissions to manage indexes and read or write documents, plus authorization to call the replacement model or custom endpoint.
  • Search capacity and model quota for the migration workload, including any temporary duplicate indexes or vector fields.
  • Representative evaluation queries, expected access decisions, and business acceptance criteria.
  • A plan for concurrent source changes, cutover, and rollback.

Choose a migration approach

First, decide where to store replacement vectors and where to obtain the content to embed.

Approach Choose when Requirements and tradeoffs Cost considerations
Rebuild a separate index from source data You use built-in indexers, or need source replay or new chunking/enrichment. New index and indexer for indexer pipelines; independent validation and source-change catch-up. Full-corpus embeddings, source reads, applicable paid enrichment, and duplicate index capacity.
Re-embed exported index content into a separate index You manage push ingestion and can export suitable text or chunks completely. Verify schema eligibility and export consistency. The samples require frozen writes during backup. Replacement embeddings, export storage, adapter hosting, and duplicate index capacity.
Add a second vector field to the existing index You manage push ingestion and want to share content while retaining old vectors. New field/profile, bounded backfill, and coordinated writers. See the second-field procedure. Replacement embeddings, duplicate vectors, backfill writes, and possible scale-out. Dual-write adds ongoing embedding work.

Use the first two approaches for the separate-index procedures, or follow the second-field procedure for a same-index migration. In that approach, keep the old and new fields side by side through validation and the rollback window. A vector field's dimensions can't be changed after the field is created, so create a new field or index with the replacement model's dimensions. See Update or rebuild an index for supported schema changes and the sample workflows for current and planned sample coverage.

If Foundry IQ built-in indexers populate your index, use Rebuild from authoritative source data. This migration path requires a full source rebuild into a new index using a new indexer. The export and second-field alternatives in this guide are for application-managed push ingestion, not migration through an existing indexer pipeline.

An identical backup and restore can help rehearse recovery, but it doesn't migrate an embedding model. It only copies existing vectors.

Choose how to handle concurrent changes

Separately, decide how to keep content current while generating replacement embeddings.

Strategy Choose when Required controls Cost considerations
Pause writes A maintenance window or delayed freshness is acceptable. Stop applicable indexers, push writers, updates, deletes, and schema changes. Account for upstream changes before resuming normal operation. Avoid simultaneous old/new generation during the pause, but budget for catch-up processing and the business effect of delayed freshness.
Build from a consistent source snapshot and replay changes You need a short cutover window and have a durable change stream. Track a starting watermark and replay creates, updates, deletes, and permission changes. Reconcile the target before cutover. Snapshot storage, change capture, replay, and reconciliation. Repeated changes can require additional embeddings.
Maintain old and new ingestion paths Continuous freshness justifies the extra engineering and capacity. Generate model-specific vectors for both indexes or both fields. Coordinate writes to the same document and handle retries, partial failures, deletes, and permission changes. Both models process applicable changes. Include extra writes, retries, retained capacity, and reconciliation effort.

These choices are ingestion architecture choices, not automatic migration features. Search exports aren't transactional snapshots, and the migration samples don't capture and replay concurrent changes. Writing to two indexes isn't an atomic operation.

For two fields in one index, dual-write means keeping both embeddings current for each changed document. It doesn't require repeatedly embedding the entire corpus. Use a bounded backfill with progress and consistency controls.

For built-in indexers, parallel ingestion means retaining the old pipeline while a new indexer populates the new index from source. It doesn't mean backfilling a second field through the existing indexer.

Keeping queries online against the old index doesn't necessarily keep that index current. Define both your availability requirement and your acceptable data delay.

Estimate migration costs

Separate one-time backfill costs, temporary overlap costs, and steady-state operating costs. No approach is always cheapest: sharing text and metadata in one index reduces duplication, but two vector fields still consume vector capacity.

  1. Establish the current production baseline and the expected migration and rollback periods. Separate additional migration spending from costs you already incur for the existing application.
  2. Measure a representative pilot before estimating the full corpus. Record source volume, generated chunks, provider-billed embedding usage, retries, index size, and elapsed processing time. Use the actual provider's billing units; don't assume every model bills by tokens.
  3. Estimate the initial rebuild or backfill. Include replacement embeddings, source reads or export, applicable parsing and paid enrichment, artifact storage, adapter hosting, and data transfer. For a new indexer, budget for the full source load rather than assuming the old enrichment cache eliminates that work. Failed or repeated processing can still incur charges.
  4. Estimate temporary overlap and validation. Include any additional Search capacity, old/new ingestion for changed content, cache misses and warm-up, evaluation queries, applicable premium query features, and rollback retention. Allow for retries and reconciliation. Throttling can extend the overlap period and time-based resource costs.
  5. Estimate steady-state costs after cutover. Compare the replacement model's ongoing document and query embedding usage, Search capacity, vector storage, and adapter costs with the baseline. Retire old resources only after acceptance and the rollback window.

Apply the billing model correctly:

  • Dedicated Search: Charges depend on provisioned capacity and how long you retain it. A second index or vector field doesn't automatically double the Search bill or add a per-index, per-field, or per-write fee. Check whether existing capacity can handle the peak workload; additional replicas, partitions, or another service can increase costs. Model calls and other billable services remain separate.
  • Serverless Search, where supported: Compute and indexed storage are usage-billed. Include indexing and query work, not just stored vectors. Check current tier limits and networking restrictions rather than assuming support for every migration scenario.

Use Search cost guidance, capacity planning, and current Foundry IQ pricing. Obtain model rates from the selected deployment's pricing information. Set a budget and alerts, monitor actual usage during migration, and investigate unexpected reprocessing before expanding the workload.

After deleting old vectors, check whether you can reduce replicas or partitions while meeting your workload requirements. Storage cleanup alone doesn't lower Dedicated Search charges. Include any remaining artifact or cache retention in the post-migration budget.

Define the replacement embedding contract

For each vector field, document the settings that indexing and querying must use together:

  • Model identity, version, deployment, endpoint, and API contract.
  • Output dimensions and an appropriate similarity metric.
  • Input preprocessing, chunking, truncation, and normalization.
  • Provider-specific document and query task settings, where required.
  • Authentication, network access, and regional availability.

Changing a vectorizer or deployment name doesn't regenerate indexed vectors. To change embedding spaces, generate document embeddings again from the intended text, images, or other source inputs. Equal vector dimensions don't establish compatibility between models.

Keep each vector field compatible with the model used to generate its query vectors. For a same-index migration, store replacement embeddings in a separate field even when the dimensions match, so you can compare results and keep the old vectors available for rollback. For integrated vectorization, select a supported skill and vectorizer pair. If the replacement requires a custom adapter, validate both its indexing skill and query vectorizer contracts.

If you also change chunking, compare that change separately where practical so you can distinguish its effects from the model change. Review compression and similarity settings rather than assuming that settings tuned for the old model remain appropriate.

Handle caches during migration

Inventory caches in the ingestion and query paths. Switching a model, vector field, or index alias doesn't automatically invalidate application-managed caches.

  • Indexer enrichment cache (preview): This cache belongs to its indexer, not to the application query path. Don't assume you can transfer it to a new indexer. Follow the reprocessing considerations; cached old-model vectors don't replace generating new embeddings.
  • Application-managed document embeddings: Reuse an entry only when its input content and embedding contract match. Include the source revision or content hash and contract version in the cache key. Changed chunking or preprocessing can invalidate previously cached embeddings.
  • Application-managed query embeddings: Key entries by query input and embedding contract, not query text alone. A deployment name or endpoint can stay unchanged while its model changes. Keep document and query task settings distinct where the provider requires them.
  • Application-managed search results or answers: Account for the target index generation, vector field, query configuration, filters, and authorization context. Invalidate affected entries when source content or permissions change. Don't serve cached content unless the requesting user's current access remains valid.

Use an explicit embedding-contract version that covers model identity/version, dimensions, task settings, and preprocessing. Keep old and new application cache namespaces separate during overlap and rollback.

If a model changes behind a custom skill endpoint without a detectable skillset change, cached outputs can remain stale. Follow the documented skill invalidation and reprocessing procedure for affected skills and their dependencies. This procedure doesn't replace the new-indexer source rebuild required for the indexer migration path.

Budget for cache storage, replacement-model calls on misses, and warm-up traffic. Test both cold and warm paths before cutover; a warm cache must not hide an incompatible model, stale document, or incorrect access decision.

Plan for Cohere Embed v3 retirement

The Foundry model retirement schedule lists November 15, 2026, as the deprecation date for Cohere-embed-v3-english and Cohere-embed-v3-multilingual. Both models retire on February 15, 2027. The listed replacements are Cohere-embed-v5-fast and Cohere-embed-v5-pro. Check the schedule for updates when planning your migration.

Complete and test the migration before retirement, allowing time to resolve failures and rehearse rollback while the old model remains available.

  1. Identify ingestion pipelines and query paths that use either retiring model, including client-side query embedding generation.
  2. Evaluate the replacement models against representative queries and your quality, latency, throughput, language, and cost requirements. Deploy the selected replacement in Microsoft Foundry and verify its document and query interfaces and output dimensions.
  3. Choose a migration approach. Built-in indexer pipelines require a full source rebuild with a new index and indexer. For push ingestion, prefer a separate target index or add a second field with consistency controls. Regenerate document vectors; don't copy v3 vectors and only change the query model.
  4. For push ingestion with the Python or .NET samples, configure the selected fast or pro model and the authenticated embedding Function. These samples use a Custom Web API vectorizer; a replacement listed in Foundry doesn't imply support in an existing native Search skill or vectorizer.
  5. Complete data, access, and retrieval testing, then switch traffic. Update ongoing ingestion and query embedding generation together.

Choose a v5 variant

Select a variant based on the workload you test, not just the model name. For capabilities and workload guidance, see Cohere Embed V5 in Microsoft Foundry.

Variant Workload priorities Example deployment name
Embed v5 Fast Latency-sensitive interactive search and responsive agent experiences. cohere-embed-v5-fast-sls
Embed v5 Pro Retrieval quality for applications where finding the most relevant information is the priority. cohere-embed-v5-pro-sls

Fast and Pro share an embedding space, but that compatibility doesn't extend to existing v3 vectors. Regenerate v3 document embeddings for the v5 migration. Validate dimensions, document/query task settings, relevance, latency, and cost for your selected configuration.

Call the Cohere provider endpoint

The direct Cohere embedding request uses this endpoint pattern: POST https://<foundry-resource>.services.ai.azure.com/providers/cohere/v2/embed.

Replace <foundry-resource> with your Foundry resource name. Set the request's model value to your deployed model's deployment name, such as cohere-embed-v5-fast-sls or cohere-embed-v5-pro-sls. Both variants use the same provider route. The names in the preceding table are examples, not required deployment names.

Follow the migration samples for the request payload, API version, and authentication configuration. Use the document task when indexing and the query task when generating query vectors.

This provider endpoint isn't the Search Custom Web API vectorizer endpoint. In the sample workflow, keep the vectorizer pointed at the authenticated Function's query route. The Function adapts the Search request to the Cohere provider request.

Prepare the target and source data

Prepare the configuration and consistency controls before starting the migration.

  1. Inventory your index fields, document keys, parent-child relationships, vector profiles, compression, analyzers, semantic configuration, and scoring profiles. Identify applications, indexers, knowledge sources, and other consumers that depend on them.
  2. Create a migration record with the source and target index or vector field names, embedding contract, expected schema changes, synchronization strategy, and acceptance criteria.
  3. Configure target authorization and network access. Distinguish the migration operator, the Search identity calling an embedding endpoint, and any intermediary identity calling the model. A successful local model call doesn't prove that Search has access.
  4. Establish the write freeze, source snapshot, or change-replay boundary selected for your migration.
  5. Define the target schema. For a separate-index migration, reserve a new index name and create it through your selected workflow. For the samples' export-and-restore workflow, leave the target nonexistent: restore creates it and refuses to overwrite an existing index. For a second-field migration, plan the additions to the existing schema instead.

Don't delete the production index to make room without a separate, reviewed availability and recovery plan. Protect exported documents and migration artifacts with access controls and retention policies appropriate to the source content. Don't commit them to source repositories.

Index definitions returned by GET or LIST aren't credential backups. Recreate required dependencies and supply target authentication explicitly. For index management tasks, see Manage search indexes.

Preserve document identity and access controls

Keep document identities and their associated content, metadata, and permissions together. If chunking or key generation changes, account for obsolete parent and child records rather than assuming that new writes remove them.

For protected content, retain the supported connector or trusted ingestion flow that supplies permissions. Copying a field with a permission-like name doesn't recreate connector-derived permission provenance. Verify the requirements of your document-level access flow before selecting an export-and-push migration.

Don't assume the general migration samples support every permission-enabled or generated knowledge-source index. If you can't establish a supported, complete transfer, rebuild through the appropriate source ingestion flow. Validate effective access before exposing the target to applications.

Check export eligibility

If you migrate from an existing index instead of authoritative source data, check every field required to recreate the documents.

  • You must be able to retrieve the required content. An index export can't recover source text or metadata that it can't read.
  • You can't recover vectors with stored: false through document retrieval. Changing retrievable doesn't make them exportable.
  • The sample's paging requirements must match the key field and corpus size. Python and .NET don't have identical paging eligibility rules.
  • The export must include the complete intended document set, not just documents visible to a restricted query identity.

For fields with stored content, changing retrievability might be supported, but it also changes what queries can return. Don't broadly expose sensitive fields merely to satisfy a sample. See Vector storage options for the distinction between stored and retrievable.

Generate and load replacement embeddings

Follow the procedure for your selected approach. Keep production reads on the existing index and old vector field until the replacement passes validation. Only ingestion paths configured for the replacement embedding contract should write replacement vectors.

Rebuild from authoritative source data

Use this process for Foundry IQ built-in indexers. Also use it for application-managed ingestion when source data and metadata are replayable or parsing, chunking, or enrichment must change.

For an indexer-based migration, changing the existing vectorizer or model deployment doesn't backfill replacement embeddings. Create a new pipeline that processes the full source corpus. Keep the existing index and pipeline available until the replacement passes validation.

If the replacement model doesn't have a supported built-in embedding skill and vectorizer pair, keep the built-in indexer and add a custom integration to the new pipeline:

  • Indexing: Add a Custom Web API skill to the new skillset. Its endpoint generates document embeddings with the replacement model. Map its output to the new vector field through output field mappings or index projections, as appropriate.
  • Querying: Configure a Custom Web API vectorizer in the new index and reference it through the vector field's profile. Its endpoint generates compatible query embeddings for text vector queries. If your application sends query vectors directly, update its embedding generator instead.

Implement adapter endpoints that satisfy the skill and vectorizer request and response contracts. Don't point either component at a raw model API unless that API implements the required Search contract. Use the same replacement embedding contract for both paths, with the appropriate document and query task settings.

Configure authentication and network access separately for Search-to-adapter and adapter-to-model calls. Validate document embedding generation and query vectorization through the actual Search identity before starting the full load.

  1. Create the new target index with the replacement vector field and profile. Keep production queries on the existing index.
  2. If embeddings are generated in a skillset, create a separate skillset for the replacement model and review its index projection targets. Don't change a shared production skillset in a way that switches the old pipeline to the new model.
  3. Configure document embedding generation and query vectorization with the replacement contract. Keep source permissions associated with the corresponding documents and chunks. For indexer-based integrated vectorization, follow Set up integrated vectorization. For application-generated vectors, follow Generate embeddings.
  4. For built-in indexers, create a new indexer with its source connection, new target index, replacement skillset where applicable, and field/output mappings. Set disabled to true during setup; otherwise, creation starts a run immediately.
  5. Validate a representative test load before starting the full load. For built-in indexers, use a separate test pipeline with a limited source scope. Keep the full-source migration indexer disabled until you're ready for the full load.
  6. Process the full corpus through the new pipeline. For built-in indexers, plan reprocessing, enable and run the migration indexer, and monitor its execution history, errors, and warnings. For push ingestion, inspect per-document indexing results. Resolve failures before declaring the load complete.
  7. Apply the source changes required by your synchronization plan, including deletions and permission changes.

The target is ready for acceptance testing when the intended corpus is loaded and changes through your agreed synchronization point are accounted for.

Plan indexer reprocessing

A full source rebuild is more than copying documents into a new index. Plan the initial processing work, subsequent changes, and recovery from interrupted runs.

  • Initial load: The new indexer has its own processing state. Budget for source reads, parsing, chunking, applicable enrichment, and replacement embeddings across the full intended corpus. Configure the replacement model before this load; don't wait for source documents to change before regenerating their vectors.
  • Incremental runs: After the initial load, ordinary runs use the source's supported change-detection behavior. They aren't requests to regenerate every vector. Verify how updates, deletions, and permission changes reach the new index before relying on its schedule.
  • Enrichment caching: The enrichment cache is a preview feature, managed per indexer. Don't assume the old indexer's cache transfers to the new pipeline or makes the rebuild incremental. Follow the cache handling guidance when model inputs or skill behavior change.
  • Retries and resets: Don't assume exactly-once processing or one model call per chunk. Interruptions and retries can repeat work and consume model quota and budget. Run and monitor the migration indexer to continue processing; don't routinely reset or recreate it to retry failures. A reset followed by a run requests full reprocessing.
  • Chunk and deletion reconciliation: Reprocessing can change chunk counts or keys if parsing or chunking changes. Check for missing and obsolete child records and retain their correct permissions. Resetting and rerunning doesn't remove orphaned documents whose source records no longer exist; use the supported deletion flow.
  • Pipeline overlap: While both indexers remain active, source changes can invoke both models and applicable enrichment. Include this overlap, throttling, and repeated work in the migration cost estimate.

Enrichment caching isn't a backup or a migration-completion ledger. Repeated interruptions or skill failures can increase reprocessing on large sources. Validate a representative workload before assuming caching reduces the rebuild's cost or duration.

Add a second vector field in the same index

Use this procedure for application-managed push ingestion, not built-in indexer pipelines. Indexer users must rebuild from source with a new indexer.

Keep the old and new vector fields together while you populate and test the replacement. This approach avoids creating a second complete index, but backfill competes with production traffic and doesn't guarantee uninterrupted service.

  1. Confirm that your ingestion flow supports the schema and writer changes. Don't assume a generated knowledge-source index supports arbitrary customization. Check capacity for both vector sets and review partial-update risks before writing any documents.
  2. Add the new vector field and a compatible vector profile, algorithm configuration, and vectorizer where applicable. Use the replacement model's dimensions and similarity settings. Existing documents initially have null values in the new field.
  3. Keep production queries on the old vector field and compatible query model. Configure either a write pause or coordinated dual-write for new and changed documents. During dual-write, generate both embeddings from the same authoritative content revision and handle failures explicitly.
  4. Backfill the new field from the intended text for each existing document. Use the controls in Make backfill bounded and restartable. Preserve unrelated content, permissions, and old vectors.
  5. Reconcile all intended document keys and source revisions through the agreed synchronization boundary. Resolve missing vectors, failed writes, deletions, and permission changes. Matching document counts alone doesn't establish coverage.
  6. Complete data, access, and retrieval testing against the new field before switching production queries.
  7. Switch the same-index query path while retaining the old path for the agreed rollback window. Keep old vectors current during that window, or maintain a tested catch-up path.
  8. After the rollback window and acceptance, stop generating old embeddings. Optionally clear old vector values through a tested document-update process. The old field remains in the schema until you rebuild the index.

Don't try to disable the old field by changing its searchable attribute. Changing that attribute or removing the field requires a rebuild. Leaving the field defined doesn't require querying it.

Make backfill bounded and restartable

Separate the historical backfill from ongoing ingestion. Process the initial corpus once, handle new and changed content through the ongoing path, and reconcile before cutover. Continuous source changes might require more than one reconciliation pass.

Use progress tracking to avoid unnecessary embedding calls, and writer coordination to prevent stale updates. These application-managed controls solve different problems.

Control How to use it Limitation
Migration version and readiness Track the embedding-contract version and source revision used for the new vector. Process missing or stale work; record completion only after a successful vector write. A Boolean such as vectorV2Ready alone doesn't prove freshness after a source change. A marker isn't a lock.
Deterministic checkpoint Persist a stable sortable/filterable key checkpoint or an authoritative source cursor for a bounded pass. Retain failed keys for retry before declaring the pass complete. A document key isn't automatically a creation watermark. New or modified records behind the cursor need replay or reconciliation. Search paging isn't a transactional snapshot.
Source revision or content hash Compare the content and embedding contract used for the vector with the current source state. Skip already current, unchanged records. A revision check before writing doesn't prevent a change between the check and the write. Include all embedding inputs and preprocessing in the comparison.

Store progress in an external durable ledger or suitable index fields. If you filter on index markers, configure their field types and filterability for those queries. Don't mark work complete while its vector write is still pending or failed.

Coordinate historical and ongoing writers for each document key. For example, route both through one ingestion owner that serializes changes and rejects stale source revisions. A marker alone can't stop an old embedding request from completing after a newer write and overwriting it.

Don't rely on batch action ordering, retries of an unchanged stale payload, or an assumed document-level conditional-update guarantee. If you can't establish reliable ordering and reconciliation, use a write pause or a separate-index approach with an appropriate synchronization plan.

For historical updates to existing records, consider merge, which fails when a record no longer exists. A stale mergeOrUpload or upload can recreate a deleted record. Coordinate deletes and permission changes with the same ingestion flow, and inspect every document's indexing result.

Use the migration samples

The following reference samples demonstrate re-embedding exported text into a new target index. Follow their READMEs for dependencies, authentication, configuration, commands, and validation.

These samples use application-managed push workflows. They aren't migration tools for an existing built-in indexer pipeline. For built-in indexers, follow the full source rebuild procedure with a new index and indexer.

Sample Export-based workflow Same-index second-field workflow
Python embedding model migration Back up an eligible frozen source index, validate an identical restore, regenerate a selected vector field, and restore to a new target. Includes an embedding Function and an optional Blob ingestion pipeline example. Planned at this sample location. Follow the second-field procedure; check the README for available implementation.
.NET embedding model migration Create a complete baseline with the backup/restore tool, prepare re-embedded artifacts, and optionally restore them to a new target. Uses a separately deployed embedding Function. Planned at this sample location. Follow the second-field procedure; check the README for available implementation.

The export-based workflows use authenticated document and query embedding routes. Their fast and pro selections are sample-specific model configurations, not generic switches for arbitrary providers. Validate any adaptation against the replacement model's actual interfaces. Their procedure is:

  1. Review the selected sample's schema, key, storage, and authentication prerequisites. Run its offline checks before cloud operations.
  2. Freeze all applicable source writers during backup. A command-line acknowledgment doesn't stop indexers or other applications.
  3. Create a complete baseline in a fresh output directory. Reject partial exports or incomplete artifacts as migration input. If you resume source writes afterward, capture and replay those changes before cutover.
  4. Validate the embedding Function's document and query routes, including authentication and measured output dimensions.
  5. Generate replacement vectors and load a fresh target index. Use new artifact directories and target names as required by the sample. Treat nonzero exits and per-document failures as failures, not completion.
  6. Configure future ingestion to generate compatible vectors. The samples don't automatically migrate your existing indexers, data sources, skillsets, aliases, or external model resources.

The export-based samples regenerate one selected top-level vector field from one top-level text field. Don't assume they handle nested or multimodal fields, every permission flow, or arbitrary projection schemas. They also change the selected vector profile's compression configuration. Review the target's algorithm and storage settings.

The optional Python pipeline is a full-document Blob example, not a reconstruction of your existing chunking, projections, or security configuration. Use it only when its ingestion contract matches your requirements.

Validate data, access, and retrieval

Before making changes in production, test your selected migration approach in a nonproduction environment using a representative sample of your own data and queries. Validate retrieval quality, access, performance, cutover, and rollback against acceptance criteria that you define in advance.

Test in stages: start with a small, representative dataset, validate the fully populated target, and rehearse cutover and rollback. Run load tests against your expected traffic. Offline sample tests help check tool behavior, but they don't replace end-to-end testing with your deployed model, identities, network configuration, and application.

Check data and configuration

  • Compare the target schema with the intended changes. Review unrelated fields, profiles, and dependencies for unintended differences.
  • Reconcile document keys, parent-child associations, expected nonvector values, deletions, and freshness through the agreed synchronization point.
  • Check that replacement vectors are populated, finite, and have the expected dimensions. Resolve failed uploads and missing embeddings.
  • Verify that both new documents and updates use the replacement embedding contract.
  • For an indexer rebuild, verify the full initial source load, subsequent change detection, and recovery from an interrupted run. Reconcile projected chunks and source deletions rather than treating an indexer reset as cleanup.
  • For a second-field migration, verify that readiness corresponds to the current source revision and embedding contract, not just a populated vector.
  • For application-managed backfill, interrupt and resume processing. Verify that committed progress skips completed, unchanged work and that failed records remain retryable. Incomplete work can run again; don't treat this as an exactly-once guarantee.
  • Test an update and a deletion while a historical embedding request is in flight. Confirm that stale work can't overwrite a current vector, falsely mark it ready, or recreate deleted content.
  • For a same-index backfill, test records added or changed behind a checkpoint. Verify reconciliation and preservation of old vectors and permissions, including applicable stored: false updates.

For an identical baseline copy, compare content fidelity, including vectors. For a model migration, old and new vectors aren't expected to be numerically equal.

Check access and query execution

  • Test authorized and unauthorized identities with representative protected content. Preserve the supported permission flow; don't substitute an unfiltered or elevated fallback.
  • Exercise query-time vectorization through the actual Search identity and network path.
  • Verify client-generated query vectors, selected fields, profiles, and embedding caches. Test cache invalidation and versioning with cold and warm caches, including content and permission changes.
  • Test applications and dependent retrieval consumers, not just direct requests to the target index.

Evaluate quality and operations

  • Compare relevance against judged queries and business tasks, including important languages, filters, and query types.
  • Test vector, hybrid, and semantic query behavior used by your application. Review score-based thresholds rather than assuming raw vector scores are comparable across models.
  • Measure latency distributions, throughput, throttling, storage, and embedding usage under representative load.
  • Confirm that the target meets your freshness, access, quality, and operating-budget requirements.

If checks fail, keep production traffic on the existing path while you resolve the cause. Don't switch merely because the full load has finished.

Switch traffic and retain a rollback path

Coordinate the change across ingestion and querying. Switching an index name alone isn't sufficient when the application generates query vectors.

  1. Reach the agreed write or change-replay boundary and complete acceptance testing.
  2. For a direct-index cutover, switch the application's index target, query embedding configuration, selected vector field, and caches as a coordinated deployment. For an alias-based cutover, follow Switch a secondary index with an alias; don't treat the alias and model changes as atomic.
  3. For an indexer-based migration, keep the new indexer targeting the new physical index and configure its ongoing schedule as needed. Update knowledge sources through their supported target configuration.
  4. Confirm production reads and writes reach the intended target. Test a new document, an update, a deletion, and an applicable permission change.
  5. Monitor the agreed relevance and operational signals during the rollback window.
  6. Retire obsolete ingestion paths and model dependencies only after acceptance and retention requirements are satisfied. For a separate-index migration, retire the old index. For a second-field migration, retain the active index and follow the old-vector cleanup guidance.

Retain a compatible query path as well as an old index. If the old model retires or becomes inaccessible, the old index alone might not provide a usable rollback. Rehearse rollback while the old model remains available.

Account for writes accepted after cutover before redirecting traffic back. Rolling back an alias doesn't copy those writes to the old index.

Switch the same-index query path

For a second-field migration, retain the index name and change the application's query configuration. An index alias doesn't select a vector field.

  1. Complete acceptance testing and final reconciliation for the new field.
  2. Switch vectorQueries.fields to the new vector field together with its compatible query embedding model. For text vector queries, verify the vectorizer associated with the new field's profile. Update caches and all dependent query paths.
  3. Test production reads and ongoing writes, including updates, deletions, and permission changes. Continue maintaining both embedding paths during the rollback window, or retain a tested way to catch up the old field.
  4. If rollback is required, reconcile the old field and restore its compatible query configuration together. The old model must still be available.
  5. After acceptance and the rollback window, retire old-model writes and perform any planned old-vector cleanup.

Clearing old vectors removes that immediate rollback path. Test cleanup with the actual vector collection type, API, and update behavior, and preserve the new field and unrelated content. Physical deletion isn't immediate; monitor index statistics before reassessing capacity.

Switch a secondary index with an alias

If your application already references an index alias, keep that alias name when rebuilding the index. Create and validate a secondary index on the same Foundry IQ service, then change the alias mapping to point to it. Supported application calls continue using the same name without changing their index-name references.

  1. Confirm that the application's supported document operations reference the alias, not the physical index name. If they use the physical name, configure them to use the alias before relying on an alias-based cutover.
  2. Create, populate, and test the secondary index while the alias still points to the existing index.
  3. Pause model-specific queries and push writers that use the alias. Drain in-flight requests and queued retries, then complete final synchronization.
  4. Update the existing alias to replace its index target with the secondary index.
  5. Keep traffic paused for at least 10 seconds after the update to allow alias propagation. Configure client-side embeddings, selected vector fields, and caches for the replacement index. Validate requests through the alias before resuming queries and push writes. Keep traffic paused if validation fails.

An alias update can take up to 10 seconds to propagate. Switching client-generated embeddings immediately can send replacement-model vectors to the old index during that interval. Apply the same pause, drain, propagation wait, and validation sequence when rolling an alias back.

If you can't pause traffic, use application routing that pins each query or push write to a physical index and its matching versioned embedding configuration. Keep that pairing through retries. Don't rely on a mutable alias to switch the index and client-generated embeddings atomically.

An alias is scoped to one Foundry IQ service. If the target index is on a different service, update the application's service endpoint separately; an alias can't redirect requests across services.

An alias maps to one index at a time; it doesn't distribute requests across old and new indexes or provide a transaction across resources. It also doesn't regenerate vectors or change an application's embedding model.

Indexers and Foundry IQ index knowledge sources can't use an alias as their target. Update those concrete index references separately.

Consider same-index migration risks

For a second-field migration, keep old and new embedding spaces in their respective fields during backfill and validate how queries handle missing vectors. The existing field's dimensions can't be changed after creation; define the replacement dimensions on the new field. Review other field properties for changes that require rebuilding the index.

Review partial-update behavior before backfilling. For existing vector fields with stored: false, omitting vectors from an applicable merge or mergeOrUpload update can lose vector data. See Vector storage options.

This risk applies when populating a new field or later clearing an old one. Preserve or regenerate every affected vector required by the update. If your ingestion flow can't do so, use a different migration approach.

Retain both vector fields through validation and the rollback window. Adding a second field preserves old vectors, but still requires consistency controls, access validation, and a tested rollback plan.

Troubleshoot migration issues

Symptom Action
Export fails or required fields are missing. Check retrievability, vector storage, query authorization, and the sample's paging requirements. Rebuild from source if a complete export isn't possible.
Vector generation or queries report dimension errors. Measure the replacement model's output and check the field definition, profile, and query generator together. An existing field's dimensions can't be changed; create a new field or index with the required dimensions. For a same-index migration, retain the old field through the rollback window. Don't pad or truncate vectors unless the model explicitly supports that configuration.
Document counts match but content is stale or records are missing. Reconcile keys and field values through the synchronization boundary. Check deletes and changes made during export or backfill.
Backfill repeatedly embeds unchanged documents. Track successful work by embedding-contract version and source revision. Check checkpoint persistence, failed-key retries, and missing or stale marker selection.
New vectors are stale even though records are marked ready. Coordinate historical and ongoing writers per key. A marker doesn't prevent stale writes. Reconcile against current source revisions before cutover.
Deleted records reappear during backfill. Prevent stale upserts from recreating deleted content. Coordinate deletes with backfill and consider updates that require an existing document.
The Function works locally but Search queries fail. Verify Search-to-Function authentication, audience, caller authorization, network access, and the Function-to-model connection. See Custom Web API vectorizer.
Only some documents load successfully. Inspect per-document results, resolve errors, and retry according to your ingestion plan. See Troubleshoot indexers for indexer-based loads.
Access decisions differ from the source. Keep the target out of production. Review the supported permission ingestion flow and document associations before retesting representative identities.
Retrieval quality regresses. Check model consistency, task settings, preprocessing, chunking, similarity metric, and compression. Reevaluate with representative queries before changing production traffic.