Infuse uses info from TMDb for movie collections, and the Metadata Fetching option will need to remain enabled in order for things to be grouped into collections.
Unfortunately, collections cannot be used when only local or embedded metadata is used, but there is a thread the suggestions area for creating manual collections you may consider following.