Why even 7zip at that point if you aren’t compressing? tar is the universal and industry standard for this job.
The idea would be to use an unpacked format for library authoring (i.e. the official library Git repository would be unpacked) and a packed format for distribution (to make KiCad faster). The packed format would likely also be compressed at that point.
As @retiredfeline says, there is really no point to forcing the libraries to be in the same format for editing as distribution. Related: it’s also desired to allow embedding 3D models inside footprint libraries, etc. Distribution libraries should be optimized for performance and file size, meaning they should be single-file and compressed. VCS diffing of these distribution libraries is not a relevant feature in comparison.
4 Likes
That was said earlier – tar doesn’t support random access.
That’s right. The only problem for me is that reloading the footprint libraries takes so long in KiCad when something has changed. I suppose that when the library system is redesigned I can have compressed official library and non-compressed personal library which I can edit easily and keep in a VCS.
Yup, exactly.
Random access on disk is not really needed if the uncompressed library is cached in RAM
I guess it depends on the direction we are going to go with library loading in future since we do need a take a serious look at the increasingly significant loading delays for users that aren’t rocking a 5900X, multi SSD beast like I do.
1 Like
I have nothing in this exercise, but I wonder if the effort to make the changes discussed in this thread are the best use of available resources? I am running KiCad in a Windows 10 Pro virtual machine on a Dell Vostro 3681 (I5-10400 processor) PC running Windows 10 Pro equipped with 32GB ram and a 500GB M.2 NVME and it is so much faster than the previous real PC it ran on it is incredible.
Just thinking that the inevitable hardware advances may negate the need to completely redo the symbol library format. Just my $0.02.
It’s not just about the user’s computer speed, it’s also about download size and the ease of maintaining the official library (and a desire to have harmonized concepts in the symbol and footprint libraries)
Slowing OS software performance seems to be quite capable of matching hardware speed increases.
Add to this that the industry is trying to push everything into the cloud, which means that each file access might cost seconds across the oceans. We have already seen the developers trying to speed up KiCad performance on a LAN
Nah this isn’t a cloud issue. NAS/shared drives in corporate networks are defacto standard for decades. That’s my focus with the fixes. Granted, at my work I’ve long since implemented a 40 gigabit backbone to our Windows Server driven storage and the user switches, so latencies are no issue (anymore…it used to be a 1 gigabit pipe between our entire VMware/SAN cluster and the user switches, I cried everyday). But not everyone in corporate environments is fortunate and they do sometimes get the short end of the stick, even being physically in a building only feet from the server because their IT is…not very on top of things. The fact it fixes cloud drive storage issues are a bonus since it is all related.
Most KiCad developers are not working on networks, so many bottlenecks were overlooked. I still see commits to improve performance. eg https://gitlab.com/kicad/code/kicad/-/commit/765816ccb1268f80fd093e22bba172ac2fa2c9f4
Yes that’s a recent issue in 5.1 that we thought we fixed months ago for network shares.
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.