Currently, there is no specific setting in Windows that allows you to prevent the "Location in use" icon from appearing in the taskbar without disabling location services entirely. The icon is designed to inform users when an application is actively using location services, and this behavior is consistent across Windows 10 and Windows 11.
If you want to avoid the icon from showing up, the only option available is to disable location services completely, which you mentioned as the "nuclear solution." Unfortunately, there are no gentler solutions provided in the context that would allow you to manage this icon's visibility without affecting the overall functionality of location services.
You may want to provide feedback to Microsoft regarding this design flaw through the Feedback Hub, as user feedback can influence future updates and changes in Windows design.