There is not anything specifically difficult about getting this to work. Yet, everything has to be correct, or it becomes difficult to figure out what is wrong.
It is a very good idea to name the local libraries with some unique identifier that KiCad will sort to the top of the list.
No Local library can be of the same name as any Global library.
On Linux, the USB stick should not have a space in the name.
Funny enough, there is probably a performance hit on 5.1.5. There was a bug pre-5.1.9 that effectively caused KiCad to spam the filesystem for fileinfo constantly in the background. This means network shares and slower filesystems like flash drives would have slow down kicad itself.
