I did start writing a library manager/downloader app, using libgit2 and also some web scraping. It's a prototype, it's not cross platform, and it's not C++ so pretty useless for inclusion in KiCad, but I wanted to get my head around what sort of GUI and operation would look like. It also only works if KiCad is installed in a user location, ie. not Program Files.
For example, one issue is it's hard to tell what the current installed versions are, because the Kicad files lack metadata. I was going to solve that by writing some metadata files, which would be valid after the first download using the app.
git is quite inefficient for the 3D libraries, it's 2.5G of data which even on my super-fast fiber takes a while to download, and even copying on disk is quite slow . A "thin" clone would be better, but in that case one may as well pull the zipped master and just make a note of the head, in order to check for updates.
I don't think there is a really simple way to handle large data blobs like 3D models that is efficient, you have to get into caching, on demand decompression etc.