Платформа и инфраструктура облачных вычислений для создания и развертывания приложений и служб, а также для и управления ими через глобальную сеть центров обработки данных, управляемых корпорацией Майкрософт.
Troubleshooting flaky tests in Azure Pipelines: Is "Debug Therapy" the right approach for LLM-based QA?
"Hi everyone,
I'm currently struggling with stabilizing our automated testing suite in Azure DevOps. Since we moved to AI-driven features, our pipeline logs have become massive, and I'm finding it nearly impossible to pinpoint why some integration tests fail only 10% of the time.
I was reading about a concept called "Debug Therapy" (I think I saw it on the testomat.io blog recently) which suggests that we should treat debugging in software testing as a semantic analysis rather than just a trace-log hunt.
Here is my logic (where I think I might be wrong): I’ve configured our pipeline to re-run failed jobs up to 3 times to 'self-heal' the results. My assumption is that if it passes on the 2nd attempt, it's an environmental issue and can be ignored in the final report. However, our lead says this is actually masking technical debt.
Also, I’m trying to synchronize these results back to a central management hub so manual testers can see the automated failures. Does Azure native reporting support this kind of bi-directional sync, or am I overcomplicating the automated testing workflow..?
Would appreciate any advice on whether I should keep the 'retry' logic or if there's a better way to implement this 'therapy' approach for logs."