Faster processing of NAS folders

Our devices are slow at the best of times.
Adding an external NAS storage through WIFI typically does not help in that area.

I therefore have a suggestion for faster processing of folders, hopefully resulting in a much, much faster UI.

a) The process starts by the user adding a media content folder, e.g. “TV Shows”:
This root folder should be considered a generic container, i.e. it is not a specific tv show, but a container for tv shows.
Deleting and re-adding a root folder should remove all related caches.

b) This action happens every time the user navigates up and down the folder hierarchy:
You don’t want to do this on every visit, so keep track of when a folder was last scanned, e.g. perform it once per hour.
Scan the folder for foldernames, filenames and teir timestamps but do not scan into subfolders.
This should be performed on a separate thread, so as not to block the main thread and UI responsiveness.
Process only the ones that differ from your cache, i.e. new / updated.
To be even faster, you should show preliminary information immediately (i.e. only filename, if necessary) and update when changes are received (e.g. nice-name, thumbnail, etc).
Include a “Scan Current Folder Only” option, so you can circumvent that delay of one hour.
Include a “Full Scan From Here” option that does that: scans the currently selected folder and it’s subfolders recursively.

  • Processing a file
    Metadata files should just be handled like today / hidden.
    If a tv show has not been assigned to the file’s folder or any of it’s parent folders (recursively upwards), then try to map the filename to a tv show.
    If a tv show could not be determined then just show the filename so the content can still be viewed.
    Otherwise process the file as part of that show.
    The “S01E02 - The Big Bran Hypothesis - HD TV.mkv” file can therefore be processed as part of TBBT because it is in the “/tv shows/Big Bang Theory/Season 01” folder, which has been mapped to The Big Bang Theory.
    It should be possible to perform manual corrections.
    Cache the end result and store locally so you don’t have to process this file again unless changed / updated.

  • Processing a folder
    Metadata folders should just be handled like today / hidden.
    Mapping a tv show to a folder should enforce that mapping on that folder and all it’s subfolders.
    If a tv show has not been assigned to this folder or any of it’s parent folders (recursively upwards), then you try to map the folder name to a tv show.
    A folder that cannot be mapped should be considered a container and just appear at the top of the list.
    It should be possible to remap / perform manual corrections, which include being able to mark a folder as a container.
    However you should not allow remapping subfolders that are already part of a tv show.
    Cache the end result and store locally so you don’t have to process this file again unless changed / updated.

The above should allow folder structures as follows:

  • /tv shows/archived/Rockford Files/Season 01/metadata/S01E01 - The Kirkoff Case - SD DVD.xml
  • /tv shows/archived/Rockford Files/Season 01/S01E01 - The Kirkoff Case - SD DVD.mkv
  • /tv shows/archived/…etc…
  • /tv shows/Big Bang Theory/Season 01/S01E02 - The Big Bran Hypothesis - HD TV.xml
  • /tv shows/Big Bang Theory/Season 01/S01E02 - The Big Bran Hypothesis - HD TV.mkv
  • /tv shows/…etc…
    I deliberatly mixed “linline” metadata from separate metadata folders (not that I would use both at the same time, but that you should support it).
    I also have an “archived” folder for old tv shows, which results in a shorter and faster active list but makes it possible to view old ones as well.

The most important thing is that the UI should now be lightning fast.
Entering a folder should only incur asking the NAS for a simple directory listing, and then not even on the main UI thread.
Which is as it should be - I should not need to wait for updates that I don’t need.

Then you could build on it…

Add the following to the tv show metadata you store in the cache:

  • a “NewEpisode” boolean value (default = false, as you have probably already seen them if you add existing files)
  • broadcast information from and/or
  • a “Flat Folders” boolean value (SickBeard, SickRage, etc, i.e. is everything in one folder or is it using “Season nn” subfolders)
    Now you should be able to know:
  1. if you need to detect changes to a tv show, i.e. only when NewEpisode=false,
  2. when to expect changes,
  3. where to expect changes, even for a coming season, and
  4. optionally, when you last tried to detect changes, so you don’t check for every viewing session
    For all TV shows, where you determine that a change could have happened that you need to investigate, you make a quick scan for changes and use “Flat Folders” to only check in the expected folder.
    Do this in list order and show information in the UI, so I know that it is happening and where you are in the process.
    Again, using threads to keep UI responsiveness.
    You can clear the NewEpisode when I watch the last (unwatched) episode for a tv show, as you want to keep the value between sessions, if there are any unwatched episodes left.

Now you also have a very fast way of detecting scheduled changes.

I have added this feature as an Infuse suggestion, as I hope coming AppleTVs will be able to run it.
In reality I need this now for ATV 2 (black) :slight_smile: