There are some shows that add recaps at the beginning, before playing one or two intros, sometimes resulting in a couple minutes of content that you want to skip over. In Infuse, loading a video and then immediately trying to jump ahead can result in a lot of buffering and waiting. But later on, it can do so without a hiccup.
On the Apple TV (my primary Infuse device), it appears that for TV shows it tries to load the entire file into local storage as fast as possible (deduced by watching the network traffic from my home server). I would speculate that trying to load all this data and then also jump around the video at the same time is too much for it to handle.
So as a possible solution, perhaps Infuse could wait a few minutes before loading the entire file, to allow the user to find the proper starting point?
Edit: Hmm, just realized this might be more appropriate in the “Video Playback” forum since this isn’t really a new feature suggestion.