How does caching work for Frontdoor when a JS challenge is enabled

Two things worth verifying here.

First, with the rule still on, open the site in a browser and let the challenge page complete, then check devtools for the afd_azwaf_jsclearance cookie. That cookie means you're past the challenge. Request the same URL again and look at X-Cache. If it comes back TCP_MISS, and TCP_HIT on the one after, caching is working and the CONFIG_NOCACHE you were seeing belonged to the challenge page rather than your content. The cookie lasts 30 minutes by default, and the challenge reissues if your IP changes, so test from a stable connection.

Second, if that still shows CONFIG_NOCACHE, disable the rule and tell me what X-Cache comes back as. CONFIG_NOCACHE is a specific value. From Front Door caching: "CONFIG_NOCACHE: Request is configured to not cache in the Front Door profile." But it's also what you'd see on a route where caching was never switched on, since caching is opt-in per route.

So if it changes to TCP_MISS, or TCP_HIT or TCP_REMOTE_HIT on a repeat request, the rule really is what switches caching off. If you still get CONFIG_NOCACHE with the rule disabled, the challenge isn't the cause and something in the Front Door config is doing it. The route is the first place to look, then any Rules Engine rule that sets cache behavior.

On documentation, neither the [JavaScript challenge] page nor the Front Door caching page says what happens to caching when a JS Challenge WAF rule is enabled.

If it does turn out to be the rule, you probably don't have to choose between caching and the challenge. WAF policies in Front Door can be associated at profile, domain or route scope, and route-level takes precedence over the other two. So you can keep the challenge policy on the routes that need it and attach a policy without it to the routes you want cached. Tell me what you find and I'll give you the next steps.