When you launch Infuse, go directly to Settings > Library and tell us what the messages below the Movies, TV Shows, and Others, are.
Infuse still has to look at the files to determine if they have been deleted or added to and to determine the duration, codecs etc. even if it’s not fetching metadata from TMDB.
So when you first launch Infuse and go directly to the library settings it only says failed? It doesn’t go through any other messages before that?
Have you tried restarting the ATV? Usually it goes through several steps before you get a fail. It may be you’re atv isn’t able to connect to your server.