Can't connect AirPods Pro mid movie

When trying to connect my AirPods Pro while watching a movie (even if the movie is paused) no sound will come out of the airpods. In some cases, only turning my Apple TV off and then on again fixed the issue for me.

If I’m stopping the movie and connect my airpods and then playing the movie again, sound will be working as it should.