Hi @pepe papo , and thanks for posting your question.
Thank you for laying out the full publication procedure and the acceptance requirement. Having each step makes it possible to check them one by one against the documentation.
I noticed this question covers the same topic as your earlier one, with more detail about your current implementation. To avoid repeating the same analysis, this answer takes a different approach for you: it checks your current Python procedure step by step and focuses on recovery-oriented options.
Summary
Within the public Windows documentation, I could not find a non-administrative API sequence that guarantees the destination name binding, or the new workspaces entry, survives an operating-system crash or power loss. The documented pieces cover flushing file data, atomic replacement and a write-through move. The remaining gap is best handled with a recovery-oriented design.
Your current procedure against the documentation
- Flush step: in Python,
os.fsync()calls the CRT_commit()on Windows, which is documented to force the operating system to write the file to disk. Callf.flush()first so Python's own buffers are included. - Replace step:
os.replaceperforms the replacement, but it has no parameter forMoveFileExWflags, so it cannot requestMOVEFILE_WRITE_THROUGH. If you want that flag, callMoveFileExWdirectly. - Lease and revision: as you noted, these coordinate cooperating writers. They do not change what is recovered after a crash.
Sketch for the flush and replace steps:
import ctypes
import os
from ctypes import wintypes
MOVEFILE_REPLACE_EXISTING = 0x1
MOVEFILE_WRITE_THROUGH = 0x8
kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
kernel32.MoveFileExW.argtypes = (wintypes.LPCWSTR, wintypes.LPCWSTR, wintypes.DWORD)
kernel32.MoveFileExW.restype = wintypes.BOOL
# tmp_path must be in the same directory as final_path
with open(tmp_path, "xb") as f:
f.write(payload)
f.flush()
os.fsync(f.fileno())
if not kernel32.MoveFileExW(tmp_path, final_path,
MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH):
raise ctypes.WinError(ctypes.get_last_error())
1. Directory flush for a standard user
The public documentation does not reconcile the two requirements: the native flush routine fails without write or append access, while NtCreateFile advises against requesting FILE_WRITE_DATA or FILE_APPEND_DATA for a directory. I could not find a documented access mask, create options, sharing mode or version range for this purpose, so I would not base an acceptance requirement on it.
2. Atomicity versus durability
- Atomicity: Microsoft's guidance on alternatives to Transactional NTFS describes writing to a new file and then replacing the original as the common approach when changes must be either fully applied or not applied at all, with
ReplaceFileas one method. - Durability:
MoveFileExWdocuments that, withMOVEFILE_WRITE_THROUGH, the call does not return until the move has actually been performed on disk. That describes when the call returns. It does not describe what is recovered after a crash or power loss, and I could not find an additional supported operation that adds that guarantee.ReplaceFileWdocuments itsREPLACEFILE_WRITE_THROUGHflag as not supported, so it does not close the gap either.
3. First-time Profile\workspaces
Neither CreateDirectoryW nor os.makedirs offers a durability option, and I could not find documentation on whether the parent or any ancestor must be flushed. Because Profile already exists, a practical approach is to create workspaces idempotently at startup (for example os.makedirs(path, exist_ok=True)) and treat its absence as a first-run state rather than as data loss.
4. Boundaries
- Windows 10 and Windows 11: the APIs above are supported on both. I could not find documentation describing different durability semantics between them.
- Standard user: a volume-wide flush with
FlushFileBuffersrequires administrative privileges, so it is not available to your process. - Filter drivers: I could not find public documentation that defines durability guarantees with filesystem filter drivers in the I/O path, so they belong inside your acceptance boundary.
- Storage cache: a flush depends on the controller and device honoring it. The
CreateFileWdocumentation notes that not all disk hardware supports write-through. - OS crash versus power loss: I could not find public documentation that defines these two cases separately for namespace changes.
5. Supported alternative and recovery design
As far as the public documentation goes, I can confirm this limitation. These designs do not depend on it:
- Keep the previous valid bookmark as a backup copy (
ReplaceFileWcan create one through its backup file parameter). On startup, load the newest copy whose JSON parses and whose revision and workspace references validate. - Treat a missing or invalid bookmark as an explicit recovery state, and rebuild it from the evidence store where possible.
- If you need a documented transactional commit, Microsoft points to the Extensible Storage Engine (ESE), which is included in Windows.
A binding statement for a specific Windows build, filter stack and storage configuration would need to come through a Microsoft support request rather than a Q&A answer.
References
If this instruction is applicable to your situation, I would greatly appreciate it if you could follow the instruction here so others experiencing similar behavior can benefit from it as well.