If you click allow or deny, it will auto-save your choices on the website and won't ask again next time, but if your settings for cache, cookies, and permissions are deleted or reset every time you close your browser, it will ask again.
If you wanted to manually disable this, please go to your Microsoft Edge.
In the upper right-hand corner, you will see three horizontal dots.
Click on it and select Settings from the dropdown menu.
On the left side, click privacy, search, and services, and find Site permission.
Click it and go to All permissions > Apps on device > Disable "Ask before accessing" or you may add a website to customize behavior.