set CF_UPDATE_FLAG_ENABLE_ON_DEMAND_POPULATION trigger search indexer

Hi @Hangkun Ung ,

Thank you for the detailed description. I built a minimal Cloud Files provider (following Build a Cloud Sync Engine that supports placeholder files) and reproduced this on Windows 11 25H2 (build 26200). In short, this is expected behavior. It is how the Windows Search indexer tracks changes on NTFS, not a defect in the Cloud Files API.

Why it happens:

1. CfUpdatePlaceholder with CF_UPDATE_FLAG_ENABLE_ON_DEMAND_POPULATION marks the directory as partially populated (CF_UPDATE_FLAGS). On disk, this changes the directory's placeholder reparse data and attributes: it gets FILE_ATTRIBUTE_OFFLINE | FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS (File Attribute Constants).
2. NTFS records those changes in the USN change journal. In my test, each directory produced USN_REASON_BASIC_INFO_CHANGE | USN_REASON_REPARSE_POINT_CHANGE.
3. On NTFS volumes, Windows Search learns about changes through the USN change journal. Any changed item inside the crawl scope is queued to be indexed again. From Indexing process in Windows Search:

   "For sources such as NTFS, there is only a single crawl and everything else is handled by notifications from the USN Change Journal." "Every URL from notifications providers that falls within the crawl scope is added to a queue…"

What I observed on 20 placeholder directories:

• No action (baseline): no USN records; no directories re-gathered by Search.
• CfUpdatePlaceholder(..., CF_UPDATE_FLAG_NONE): USN_REASON_REPARSE_POINT_CHANGE on each directory; 20/20 directories re-gathered.
• CfUpdatePlaceholder(..., CF_UPDATE_FLAG_ENABLE_ON_DEMAND_POPULATION): USN_REASON_BASIC_INFO_CHANGE + USN_REASON_REPARSE_POINT_CHANGE on each directory; 20/20 directories re-gathered.
• Directory opened later: FETCH_PLACEHOLDERS was completed with CF_OPERATION_TRANSFER_PLACEHOLDERS_FLAG_DISABLE_ON_DEMAND_POPULATION. The attributes were cleared again and the directory was re-gathered a second time.

Points worth noting:

• The behavior is not specific to this flag. Any CfUpdatePlaceholder call on a directory changes it on disk and causes it to be indexed again, even with CF_UPDATE_FLAG_NONE.
• With your boot-time pattern, every directory is indexed again twice: once when you set the flag, and once when the directory is populated again and marked full.
• The indexer handles partial directories conservatively. It does not enumerate them, so it does not trigger FETCH_PLACEHOLDERS and does not hydrate anything. In my test the cost was limited to directory-level re-gathering, with no feedback loop. However, files inside a directory are not searchable while that directory is partial.

Recommendations:

1. Do not re-enable on-demand population on every directory at startup. Instead, keep a change cursor or delta token from your service. At startup, ask the server for changes since the last cursor, and apply only those changes: CfCreatePlaceholders, CfUpdatePlaceholder with new metadata, deletes and renames. Indexing activity then matches the number of real upstream changes.
2. If you prefer to load directory listings lazily (the CF_POPULATION_POLICY_FULL model), set CF_UPDATE_FLAG_ENABLE_ON_DEMAND_POPULATION only on directories the server reports as changed. If your service has no delta API, compare a per-directory ETag or hash.
3. Avoid calls that change nothing. Check the current state first with CfGetPlaceholderInfo or the FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS attribute. Skip directories that are already partial, and placeholders whose metadata has not changed.
4. Follow the documented pattern of keeping directories fully populated (CF_OPERATION_TRANSFER_PLACEHOLDERS_FLAGS, and the CloudMirror sample, which creates directories with CF_PLACEHOLDER_CREATE_FLAG_DISABLE_ON_DEMAND_POPULATION). Have your provider push upstream changes into placeholders rather than using FETCH_PLACEHOLDERS as a periodic refresh.

You could remove the sync root from the index with the Crawl Scope Manager, but users would then lose Windows Search over their files, so I would not recommend it.

I hope this helps. If you found my response helpful or informative, I would greatly appreciate it if you could follow this guide so others with the same issue can find it.

Thank you.