Infuse should store the meta data in to the container (MKV/MP4)

I know that editing headers of a file is really quick, but attaching additional files may require remuxing the entire file. Just reading the headers of all the files is the slowest part of infuse (if you have decent internet). Having it read embedded metadata will make it even slower. And then what happens if multiple infuse clients are trying to write at the same time.