Something must be wrong with your installation of ATV Flash. I recommend you do a full reinstall of ATVFlash. On my current setting I have an Apple TV with the latest ATV Flash running XBMC as my default media management. All my files (videos, movies, and photos) are stored on a NAS. Both the NAS and Apple TV are connected on the same switch at 100Mbps. All my compatible files (forget about 1080, MKV without the right encoding, etc.) play without problems on my Apple TV/XBMC. I use Handbrake to convert the files to standard Apple TV format because I can save lots of space. My setting also handles videos faster when they are on m4v format (NAS + Apple TV).
Regarding Nito, I honestly prefer how XBMC handle DVDs rather than Nito. If I’m streaming a DVD9 then the loading to memory takes a long time on Nito than on XBMC. Also, on my NAS setting Nito tends to gets choppy when playing DVD9.
How are you connecting your Apple TV and your NAS? What’s the brand/model of your NAS?
NAS throughput will be affected by many factors: NAS Specs such as memory, CPU, Ethernet speed, OS, users connected, etc. Also you have to take into account the throughput of your switch. In many cases (home settings) a NAS is connected to a Wi-Fi Router on one 10/100BASE-T port and the Apple TV is connected using Wi-Fi. On that Router you could have many users transferring data (using CPU cycles and memory). Your Apple TV could be connecting at 54Mbps using G (not very realistic) or 130Mbps using N (yeah right!) which does not represent the optimal setting. But again, you have to consider all the traffic and users on your network.
I know that my comment looks like a big brainstorm. If reinstalling doesn’t work for your please post your network settings/specs (brand/model of switch and NAS, if they are connected on an Ethernet port on the same switch, etc.)