Windows Nightly too big

Nightly builds are now 670MB. Time for the Windows builds to split into two, application and data
If you agree, please support my wishlist report

The first problem that concerns me is that if there is not a very minimal data set, then it becomes not obvious where everything should go or end up.

I think, even with the nightly alone, one should be able to have symbols for a battery, resistor, capacitor, inductor, diode, and transistor and also have a basic footprint in both TH and SMD styles.

While typing this, I think this could establish a “starting point” for a new library system that has been recently talked about in the forum. A user would then only have to download what they need or want. Reduces the data use-age on both sides.

I am thinking that a new Nightly install would use both installer files. This would give the application and a full data set.
Then you can track the application changes with a much smaller more or less daily download of just the application part. Fetching the updated data can be done less often or when you want a specific change

My concern with that is the fact that I have a post here about me having to use the un-installer to get rid of everything on a specific nightly build.

If it is done as you suggest and it works reliably then it will not be a problem.

In an ideal world, the library files should not have to be re-downloaded every time.

  1. Most of the time, the library files don’t change
  2. When the library does get updated, the majority of the files within the library remain unchanged.

It is a waste of time and bandwidth to re-download the libraries. Not only at install time, but when (some) users load libraries via the GitHub plugin.

I would love to implement a global “library manager” which separates the libraries from the installer. This would be internal to KiCad, I think. It would provide a way of keeping libraries synced with a “server” (be it Git or something else). Using a back-end such as Git means that you only ever need to download the differences in the libraries.

The library manager should also provide a status indicator showing if a library package is out-of-date (and provide a shiny “Update Library” button).

KiCad would ship with (e.g.) three standard libraries:

a) Symbols
b) Footprints
c) 3D models

However they come “empty” but pointing to the KiCad github repo, then KiCad downloads the libs on first run. Then subsequently installs (e.g. nightly builds) don’t overwrite these data.

This is a lot of work and a lot of care needs to be taken to integrate nicely with the new library formats.

Ideally this would be developed concurrently with the ability to store library elements in an external database.

4 Likes

Yeah, I see that jump (almost 20%!) in size a few days ago. Should I read between those lines, and guess that the next major KiCAD release will not offer a truly effective solution to the shortcomings of the current library system? Or is this just a temporary annoyance, caused by a need to carry large segments of both new code and old code, until new features are fully implemented?

Dale

I can’t say. These are all things I’d really like to see solved before 5.0. Hopefully, not delaying the release a ton either.

Some companies firewall policies blocking GIT mean that an option to download a zip file snapshot is still essential

Unless I’m mistaken, the 20% jump is not due to some fancy new code. Even given the beautiful way in which KiCad source is written, you can’t generate 100MB of binary that quickly :stuck_out_tongue:

This significant jump was (probably) due to this PR on the libraries.

It added a metric buttload of 3D models to the symbol libraries. These are shipped with the installer, whereas the footprints are downloaded as zip files from github. And the 3D models are downloaded from github via curl.

THIS MAKES SENSE. (do not question it)

In short, the 3D models are packaged with the installer and 3D models are large.

As an “option”, sure. We could still massively improve on the current functionality by checking to see if a new .zip file needs to be downloaded (or not). And we could treat the entire set of footprints as a single library package, rather than the MORE THAN 100 symbol libraries that currently exist on github.

1 Like

I would prefer a single directory with thousands of files (any hosting limitations?) and an index database with categories/attributes. Searching for a part is a real guessing game at the moment

That sounds like a new library format to me - that’s outside the scope of “complaining about how bad the current format is”

No, the only change is that each footprint or model file needs a hash somewhere.
The categories etc are taken from the existing Keywords field.
The database is sqlite on your own PC.
The drive to single file per symbol/footprint/model has too much inertia to fight. I am trying to figure out how to make that work in a real world of erratic Internet, finite hosting bandwidth and slow hard disk file access

Ah ok, so you’re simply suggesting a single (flattened) directory - I see (I misread your comment initially).

Yes, but maybe split A-Z due to possible hosting limits on number of files

Hi,

I prefer having it in one download. It works fine. If you split it the whole exercise becomes complex.

If it a’int broke don’t fix it.

This is about nightly test versions, not the regular stable production version of the program.
There is a new version every night… the bandwidth costs to KiCAD and people downloading it to test it are not helpful and usually the libraries do not change as much, which is why testers do not want to download it every time they are going to test the new version.

This is still broken and it still needs fixing.

3 Likes

Indeed. Even copying the 3d models from the installer to the hard drive (SSD) takes several minutes, so I don’t want to install the libraries at all with nightly builds. I have nothing against self-contained installer for the stable release.

Another annoying thing with the installer is that it blindly overwrites previous version and don’t let you handle them as separate. Especially we who test and play with versions should be able to install side by side, for example “KiCad 5” and “KiCad nightly” so that there would be no need for complex instructions for how to do it manually (which I have written to the FAQ).

These changes shouldn’t be problematic or difficult at all (relatively speaking), they would just need some volunteer to do them.

2 Likes

I personally open the installer file with 7-zip and replace only the bin and lib directories of my installed version.

1 Like

Most of the size is due to 3d files
If installer can be split in two i.e. main application package with symbol & footprints and separate installer download only packaged 3d files
The 3D part should be optional download… and I’m not talking about only nightly. This should be implemented for regular release too.
There are still plenty of users who are beginners who don’t need 3D view from the start; or some people like me, who design PCBs to be mounted in a panel with plenty of space - in this case I do not need to provide 3d drawing of the panel

1 Like