I’d like to add a small update around this topic, and why this one is still relevant.
There is a feature when adding a new source, such as SMB, called “Pre-Cache Details,” which disables detailed file analysis when adding new files or indexing. It mostly delays that process closer to the moment when the file needs to be played. You can read about that and a similar conversation in this discussion:
Why this discussion is still relevant is because when adding files from just the local computer, via the “Add Local Folder” source on MacOS, there are no advanced options to disable stuff like Pre-Cache Details, so the original post’s described problem is still pertinent.
It would be great if those options were made available to this source as well. Better yet, if this option was made as just a toggle in the “General” section of the settings, which would apply this to all sources by default or not, instead of per source.