Supported user-mode NTFS directory flush and durability of newly published name bindings

pepe papo 0 Reputation points
2026-11-02T06:30:09.2+00:00

Subject: Supported user-mode NTFS directory flush and durability of newly published name bindings

We are evaluating the durability requirements of a local Windows application running as a standard, non-administrative user on NTFS.

The application synchronizes a temporary file, atomically replaces an existing JSON file, and acknowledges success only after completing the required persistence operations. On first use, it also creates a new directory under an existing application profile directory.

We are seeking authoritative clarification of the following:

Is NtFlushBuffersFile or NtFlushBuffersFileEx(Flags=0) officially supported for an NTFS directory handle opened from an ordinary user-mode process through Ntdll.dll? What exact NtCreateFile access mask, create options and share mode are required?

How should the write/append access requirements of the flush API be reconciled with the NtCreateFile documentation restricting FILE_WRITE_DATA and FILE_APPEND_DATA when opening directories?

Does successful completion of a supported flush on a directory guarantee persistence of a preceding same-volume file replacement within that directory, assuming compliant storage hardware and drivers?

When a new child directory has just been created, does flushing that child directory also persist its newly created name binding in the parent? Or must the parent directory be flushed separately? Is an additional ancestor-directory flush required?

Which documented Windows 10/11 NTFS and storage conditions are required before an application can treat the complete file-content and namespace publication as durable across a system crash?

We distinguish API-call success, filesystem crash consistency, and physical power-loss guarantees. We are not reporting a reproduced data-loss defect.

If the described procedure is not officially supported, please identify the supported non-administrative alternative, or confirm the limitation and its relevant documentation.

Relevant references:

NtCreateFile

NtFlushBuffersFileEx

  • MS-FSA §2.1.5.7Subject: Supported user-mode NTFS directory flush and durability of newly published name bindings We are evaluating the durability requirements of a local Windows application running as a standard, non-administrative user on NTFS. The application synchronizes a temporary file, atomically replaces an existing JSON file, and acknowledges success only after completing the required persistence operations. On first use, it also creates a new directory under an existing application profile directory. We are seeking authoritative clarification of the following:
    1. Is NtFlushBuffersFile or NtFlushBuffersFileEx(Flags=0) officially supported for an NTFS directory handle opened from an ordinary user-mode process through Ntdll.dll? What exact NtCreateFile access mask, create options and share mode are required?
    2. How should the write/append access requirements of the flush API be reconciled with the NtCreateFile documentation restricting FILE_WRITE_DATA and FILE_APPEND_DATA when opening directories?
    3. Does successful completion of a supported flush on a directory guarantee persistence of a preceding same-volume file replacement within that directory, assuming compliant storage hardware and drivers?
    4. When a new child directory has just been created, does flushing that child directory also persist its newly created name binding in the parent? Or must the parent directory be flushed separately? Is an additional ancestor-directory flush required?
    5. Which documented Windows 10/11 NTFS and storage conditions are required before an application can treat the complete file-content and namespace publication as durable across a system crash?
    We distinguish API-call success, filesystem crash consistency, and physical power-loss guarantees. We are not reporting a reproduced data-loss defect. If the described procedure is not officially supported, please identify the supported non-administrative alternative, or confirm the limitation and its relevant documentation. Relevant references:
Windows development | Windows API - Win32

2 answers

Sort by: Most helpful
  1. Tony Thach (WICLOUD CORPORATION) 1,445 Reputation points Microsoft External Staff Moderator
    2026-11-02T07:21:51.73+00:00

    Hi @pepe papo , and thanks for posting your question.

    Thank you also for the additional details. Separating the lease and revision logic from crash durability, and stating a clear acceptance requirement, makes it much easier to answer from the documentation rather than from observed behavior.

    Summary

    Based on the public Microsoft documentation I could find, there is no documented non-administrative API sequence that guarantees the complete file-content and namespace durability you describe, particularly for the newly created workspaces directory. File flushing and MoveFileExW are documented building blocks, but I could not verify that their combination provides that complete guarantee.

    1. Supported user-mode directory flush

    • The native routine is callable from user mode: the ZwFlushBuffersFileEx page says user-mode callers should use the name NtFlushBuffersFileEx. It returns STATUS_ACCESS_DENIED when the handle has neither write nor append access, and FLUSH_FLAGS_FILE_DATA_SYNC_ONLY is documented as not valid with directory handles.
    • NtCreateFile says not to specify FILE_WRITE_DATA or FILE_APPEND_DATA (or generic rights) for a directory. File Access Rights Constants shows that, for a directory, the same values mean FILE_ADD_FILE (0x2) and FILE_ADD_SUBDIRECTORY (0x4). This explains the shared bit values, but does not establish a documented access contract for flushing a directory.
    • I could not find public Microsoft documentation that specifies the access mask, create options, sharing mode and supported versions for this purpose.

    2. Replacement: atomicity versus durability

    • Atomicity: Microsoft's guidance on alternatives to Transactional NTFS describes writing the data to a new file and then replacing the original as the common approach when changes must be either completely applied or not applied at all, and names ReplaceFile as one method. That guidance addresses atomicity, not durability. ReplaceFileW also documents REPLACEFILE_WRITE_THROUGH as not supported.
    • Durability: MoveFileExW documents MOVEFILE_WRITE_THROUGH as "The function does not return until the file is actually moved on the disk." This is the strongest documented statement for this step. The documentation does not separately define crash or power-loss semantics for the destination name binding, and I could not find an additional supported operation that closes that gap.
    • Python's os.replace does not expose a write-through option. If you want the documented MOVEFILE_WRITE_THROUGH behavior, call MoveFileExW directly, for example (sketch):
    import ctypes
    from ctypes import wintypes
    
    MOVEFILE_REPLACE_EXISTING = 0x1
    MOVEFILE_WRITE_THROUGH = 0x8
    
    kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
    kernel32.MoveFileExW.argtypes = (wintypes.LPCWSTR, wintypes.LPCWSTR, wintypes.DWORD)
    kernel32.MoveFileExW.restype = wintypes.BOOL
    
    # tmp_path was written, flushed (os.fsync) and closed before this call
    if not kernel32.MoveFileExW(tmp_path, final_path,
                                MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH):
        raise ctypes.WinError(ctypes.get_last_error())
    

    3. First-time Profile\workspaces creation

    CreateDirectoryW has no write-through option. I could not find a documented non-administrative operation that guarantees persistence of a new directory entry, or documentation stating whether the parent or any ancestors must be flushed. MS-FSA 2.1.5.7 says a flush "ensures that the directory structure is persisted to stable storage", but it does not define that scope for a child entry held by its parent.

    4. Storage and failure boundaries

    • Windows 10/11, local NTFS: MS-FSA Appendix A (note 80) says that behavior is only implemented by NTFS: "Other file systems return STATUS_SUCCESS and perform no other action."
    • Standard user: a volume-wide FlushFileBuffers requires administrative privileges, so it is not available to your process.
    • Filter drivers: I could not find public Microsoft documentation that defines durability guarantees in the presence of third-party filesystem filter drivers.
    • Storage cache: NtFlushBuffersFileEx with Flags = 0 asks the underlying storage to flush its cache. The result still depends on the controller and device honoring that request.
    • OS crash versus power loss: I could not find public Microsoft documentation that defines these two cases separately for namespace changes.

    5. Limitation and recovery-oriented design

    Within the public documentation, I could not find a non-administrative sequence that provides the requested guarantee. A recovery-oriented design avoids depending on that gap:

    • At startup, treat a missing workspaces directory or bookmark file as "not yet initialized" rather than as corruption, and recreate it.
    • After a restart, validate the bookmark (parse, revision, referenced workspaces) and fall back to the last valid state if it does not validate.
    • Because the bookmark is operational state separate from the evidence store, keep it rebuildable from the evidence data where possible.
    • For structured data that needs transactional updates, the same Microsoft guidance points to the Extensible Storage Engine (ESE), which is included in Windows.

    These are design suggestions, not a Microsoft-confirmed durability guarantee. If you need a formal statement beyond the public documentation for a specific Windows build and storage stack, a Microsoft support request is the right channel for that.

    References

    If this instruction is applicable to your situation, I would greatly appreciate it if you could follow the instruction here so others experiencing similar behavior can benefit from it as well. If you have any follow-up questions, feel free to add a comment.

    Was this answer helpful?

    1 person found this answer helpful.

  2. AI answer

    2026-11-02T06:31:23.0966667+00:00

    The published Windows contracts do not provide a supported non-administrative recipe for opening an NTFS directory and flushing it with NtFlushBuffersFile[Ex]; use a documented file flush followed by MoveFileExW(..., MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH) for the replacement, but do not claim that this proves complete parent/ancestor namespace durability after power loss.

    Why the native directory-flush sequence is not documented as supported

    NtFlushBuffersFileEx with Flags == 0 is documented to write cached file data and metadata and synchronize underlying storage. The documentation recognizes directory handles indirectly by stating that FLUSH_FLAGS_FILE_DATA_SYNC_ONLY is invalid for directory handles. It also says user-mode callers should use the NtFlushBuffersFileEx name rather than ZwFlushBuffersFileEx.^1^

    However, there is no documented access mask that satisfies both sides of the operation:

    • The flush API documents STATUS_ACCESS_DENIED when the handle has neither write nor append access.^2^
    • NtCreateFile explicitly says not to specify FILE_WRITE_DATA or FILE_APPEND_DATA for a directory. It also says generic rights cannot be used for a directory.^3^
    • The Win32 alternative, FlushFileBuffers, requires a handle opened with GENERIC_WRITE; that does not resolve the directory-access conflict.^4^

    Consequently, the documentation supplies no exact supported directory combination of:

    • DesiredAccess;
    • CreateOptions;
    • ShareAccess; and
    • flush API.

    FILE_DIRECTORY_FILE is the appropriate directory type option and can be combined with synchronous I/O and FILE_WRITE_THROUGH; if synchronous I/O is selected, SYNCHRONIZE is required. Those rules describe opening a directory but do not confer the write/append access required by the documented flush contract.^3^

    A standard user can obtain a directory handle with CreateFileW and FILE_FLAG_BACKUP_SEMANTICS; the flag does not require backup privilege merely to open a directory, although privilege is needed to override normal security checks.^5^ The documented list of operations accepting directory handles does not include FlushFileBuffers.^6^

    Observed success with another access mask or with FILE_WRITE_ATTRIBUTES would therefore be implementation behavior, not the requested published contract.

    What MS-FSA establishes—and what it does not

    MS-FSA defines an abstract flush operation over a DataFile or DirectoryFile. For a directory stream, it flushes metadata associated with that directory, ensures the directory structure is persisted, and flushes underlying physical storage.^7^

    That specification does not define:

    • how an ordinary user-mode application obtains a qualifying Windows handle;
    • an NtCreateFile access mask that overcomes the documented restriction;
    • that flushing directory D includes changes owned by its parent;
    • the required parent/ancestor sequence for an NTFS rename or creation.

    A child’s newly published name is an entry in its parent’s directory table.^8^ Therefore, flushing only the child cannot be treated as a documented guarantee that its parent-held name binding is durable. The available contracts also do not establish that flushing the parent is sufficient, or that an additional ancestor flush is required. There is no supported user-mode directory-flush sequence with which to implement either proposition.

    Likewise, a successful directory flush cannot be cited as a documented guarantee for a preceding replacement when the prerequisite user-mode directory open itself has no documented compatible access contract.

    Supported application-level replacement

    For an existing JSON file:

    1. Create the temporary file on the destination volume.
    2. Write all content.
    3. Call FlushFileBuffers on the temporary-file handle and require success.
    4. Close handles as required by the replacement operation.
    5. Call:
    BOOL ok = MoveFileExW(
        tempPath,
        finalPath,
        MOVEFILE_REPLACE_EXISTING |
        MOVEFILE_WRITE_THROUGH);
    
    if (!ok) {
        DWORD error = GetLastError();
        // Do not acknowledge publication.
    }
    

    MOVEFILE_REPLACE_EXISTING replaces the destination, subject to ACL requirements. MOVEFILE_WRITE_THROUGH says the function does not return until the move is performed on disk, although its explicit flush guarantee is phrased for a move implemented as copy-and-delete.^9^ This is the documented non-administrative publication mechanism, but its contract does not explicitly promise the complete parent/ancestor crash-durability model posed in the question.

    There is no equivalent documented flag that turns CreateDirectory plus parent/ancestor publication into that complete model.

    Storage and power-loss boundary

    Even a successful normal flush depends on the complete storage stack correctly honoring cache synchronization. Devices may use volatile write-back caches, may lack cache-flush or write-through support, or may instead have power-protected cache.^10^

    STORAGE_WRITE_CACHE_PROPERTY exposes, among other fields:

    • WriteCacheEnabled;
    • WriteThroughSupported;
    • FlushCacheSupported;
    • NVCacheEnabled.^11^

    That query mechanism is documented as unsupported for RAID and flash devices, so it cannot establish a universal end-to-end power-loss guarantee on common configurations.^10^

    Accordingly:

    • API success: available for file flushing and write-through replacement.
    • Filesystem crash consistency: NTFS supplies its recovery behavior, but the cited public contracts do not define the requested full namespace publication sequence.
    • Physical power-loss durability: requires compliant filesystem, filter, storage, controller, firmware, and cache behavior; API success alone cannot prove those properties.

    If a formal guarantee for a particular Windows build, NTFS version, filter stack, and storage configuration is required, it must be obtained through Microsoft Developer Support or replaced by an application design that can recover and validate publication after restart.


    References

    1. ZwFlushBuffersFileEx function (ntifs.h)
    2. NtFlushBuffersFileEx function (ntifs.h)
    3. NtCreateFile function (ntifs.h)
    4. FlushFileBuffers function (fileapi.h)
    5. CreateFileW function (fileapi.h)
    6. Directory Handles
    7. [IP Address] Server Requests Flushing Cached Data
    8. About Directory Management
    9. MoveFileExW function (winbase.h)
    10. Querying for the Write Cache Property
    11. STORAGE_WRITE_CACHE_PROPERTY structure (ntddstor.h)
    AI-generated content may be incorrect. Read our transparency notes for more information.

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.