Hello,
Starting prominently with the recent feature rollouts, the operating system aggressively prioritizes routing DNS requests over IPv6 anytime it detects that both network stacks are enabled on the adapter. Even if it just senses an automatic local link address starting with fe80, it will attempt to resolve hostnames via IPv6 first instead of gracefully falling back to your functional IPv4 local mapping. In basic peer to peer network environments, your switches and routers are likely not fully configured to manage or intercept these internal IPv6 queries.
Simply disabling it is currently the most practical diagnosis and workaround for simple unmanaged LAN setups like you are dealing with.
Since you service multiple clients with this setup, are all these local server PCs acting as file shares in standard simple workgroups, or do you have any actual Active Directory controllers in place where you might eventually need to leave IPv6 toggled on for network policy communication?
I can run you through how to manually change the Windows network prefix priorities under the hood if you happen to need a scenario where IPv6 stays enabled.