How can I create a Visual Studio 2022-style custom title bar in a C++/Qt Win32 application?

Hello @LoveC ,

I put together a small Qt 6 + Win32 prototype on Windows 11 to explore this, so the notes below are based on what I observed in that test rather than a definitive statement.

One point that may be worth checking first: the classic "DWM custom frame" trick (returning 0 from WM_NCCALCSIZE) appears to have worked on Windows 7 Aero, but on Windows 10/11 it seems the system no longer draws the Min/Max/Close buttons once the standard frame is removed. In my test the top-right area came up empty. This suggests that keeping the system-drawn caption buttons AND placing widgets on the title-bar row AND avoiding both owner-draw and a frameless window may not all be possible at once on Win11. It might be relevant that VS2022, Windows Terminal and the browsers all seem to paint their own button glyphs rather than relying on the system.

One approach that seemed to work well for me (and stays close to VS2022):

1. Keep the native frame and extend the client over the title bar with WM_NCCALCSIZE + DwmExtendFrameIntoClientArea (this appears to preserve shadow, resize and Snap).
2. Paint just the button glyphs yourself with QPainter.
3. In WM_NCHITTEST, return HTMINBUTTON / HTMAXBUTTON / HTCLOSE over the buttons. The system then seems to handle the actual minimize/maximize/close and show the Win11 Snap Layout flyout on the Maximize button. Return HTCAPTION for the empty area (drag) and HTLEFT/HTRIGHT/... for the edges (resize).

Core of nativeEvent on your QMainWindow (it helped to call DwmDefWindowProc first):

bool MainWindow::nativeEvent(const QByteArray&, void* message, qintptr* result) {
    MSG* msg = static_cast<MSG*>(message);
    LRESULT lRet = 0;
    if (DwmDefWindowProc(msg->hwnd, msg->message, msg->wParam, msg->lParam, &lRet)) {
        *result = lRet; return true;
    }
    switch (msg->message) {
    case WM_NCCALCSIZE:
        if (msg->wParam == TRUE) { *result = 0; return true; } // extend client over title bar
        break;
    case WM_NCHITTEST: {
        int btn = captionButtonAt(msg->hwnd, msg->lParam); // 0=min,1=max,2=close,-1=none
        if (btn != -1) {
            *result = (btn==0)?HTMINBUTTON : (btn==1)?HTMAXBUTTON : HTCLOSE;
            return true; // system does the action + Win11 Snap flyout
        }
        LRESULT ht = hitTestNCA(msg->hwnd, msg->lParam, gTitleBarHeight); // HTCAPTION / edges
        if (ht != HTNOWHERE) { *result = ht; return true; }
        break;
    }
    }
    return QMainWindow::nativeEvent(QByteArray(), message, result);
}

And extending the frame once the window exists:

MARGINS m = { 0, 0, gTitleBarHeight, 0 };
DwmExtendFrameIntoClientArea(hWnd, &m);
SetWindowPos(hWnd, nullptr, 0,0,0,0, SWP_FRAMECHANGED|SWP_NOMOVE|SWP_NOSIZE|SWP_NOZORDER);

hitTestNCA is the helper from the DWM custom frame sample (Appendix C). The icon, menu bar, search box and custom button can be normal Qt widgets in the title-bar layout, and it may help to reserve roughly 3 * buttonWidth on the right so they do not overlap the buttons. Two things that caught me out: the borders may need compensating in WM_NCCALCSIZE when maximized so the top is not clipped, and the hit-test point likely needs dividing by devicePixelRatioF() for high-DPI.

In my testing, checking with SendMessage(WM_NCHITTEST) showed the buttons returning HTMINBUTTON/HTMAXBUTTON/HTCLOSE and the empty area returning HTCAPTION, with the Snap flyout appearing as expected, though your setup may differ. I would be glad to look closer at any part of this. If you found my response helpful or informative, I would greatly appreciate it if you could follow this guide for your confirmation.

Thank you.