There’s no way to delete folders from the library (long pressing select should also provide that menu item, not just “use local metadata”.
If you are browsing through the Library, you shouldn’t actually see any folders - only playable items.
If you’re browsing via a folder, Infuse will allow you to delete remote videos (as long as the File Management option is enabled). If you are deleting a video that also happens to be the last item in the folder, Infuse will also remove the folder - but deleting folders themselves is not supported.
I usually browse through the shares and not through the library.
I can delete the movie file, but not the entire directory structure. I should be able to do that by long pressing SELECT (OK) and selecting “Delete folder” which should delete the folder and anything else under it.
This is, unfortunately, a bi-product of how my system downloads these files and extracts them, which can leave a few byproducts such as NFO files, cover images, sample videos and parity check files.