If you are playing as a spectator when a map completes, you can’t exit out of the scoring mode to get back to the menu. The ESC key is completely disabled. The only exit at that point is during the warmup and voting for the next map. If I was on a team, I would be able to exit with the ESC key.
