Scheduling and managing events using Outlook.com calendar
I had the same experience and it was a nightmare. I found this option to work for me.
Go to File – Options – Mail – Tracking and select “Update Tracking information, and then delete responses that don’t contain a comment.”
This will stop you receiving all calendar responses via email but will still track the responses. This way you are not flooded with accepted or rejected invitation responses, and you are still able to see who has responded and their response.