Same thing for me, i have a netgear 516 on a gigabit switch with apple tv 4 and ipad pro ==> opening a folder with 200 episodes has become extremely slow!
This since the update which has introduced the library function.
Opening the folder with series on my ipad or apple tv takes a minute and more.
Once the folder is open starting to play an episode or switching between series no problem.
It is opening and displaying the content which is slow.
Also a small remark, rewinding on ipad or apple tv 4 is unsyncing external subtitles downloaded via infuse. (Fps of the movie and sub are the same)