Infuse will include an alphabet scrollbar when browsing in the Library or a Plex server.
Emby and Jellyfin require a small technical change on the server side for Infuse to display these alphabet scrollbars, and we have communicated with the respective teams about this. Emby has expressed interest in making this small change for a future update and once this happens we can enable this feature for Emby.