For anyone else that comes up against this issue (or if I ever forget the solution), I was able to fix it by changing the limit for Internet Upload Speed in Plex from 100 Mbps to 1000 Mbps. My actual upload speed is 102 Mbps but when it was set to 100 Mbps I could not play 4k remuxes through Infuse without getting the HTTP 500 error. I could still play 1080p remuxes but now that it is set to 1000 Mbps all of the 4k remuxes are also playing fine in Infuse.
You can find the input for this in Plex > Settings > Remote Access. Also, once I have upload set to 1000 Mbps, it doesn’t matter whether I have Enable Relay in Plex Network Settings enabled or not, the 4k video will still play fine in Infuse. I actually tried it with upload set to 150 Mbps and 4k files also played fine but they will not play with it set to 100 Mbps.
I think I’ll leave it at 1000 Mbps though just to be on the safe side.