I was able to link my google drive to infuse, but when I try to play a movie it will take 2 minutes to load before playing on WiFi phone/tablet.
What type of video files are these?
Can you try running a speed test in Infuse to see what kind of speeds you are able to get?
At that speed and file size, you have just enough bandwidth for the entire file in those two minutes. Are you sure you were streaming it and not downloading it?