KiCad will support compressed step and vrml models in the future

I wasn’t but I am now. :wink: As you know, I am a big fan of getting our model library into a smaller size.


Note by mod: Moved from original topic as this spawned a lengthy (but interesting) discussion
I also sadly screwed up the ordering in the move so the true top post is: KiCad will support compressed step and vrml models in the future

5 Likes

Please test the effect on loading times on Windows, it is possible that antivirus engines might try decompressing the entire library on the fly and make KiCad loading and installation even more painful

I’m already using the STEP library with the compressed models in .stpz to load kicad_pcb into FreeCAD and I don’t see any weird behavior both on Win10 or Linux.
I wouldn’t expect any different behavior if the wrl models would be converted and loaded from wrz compressed file format.

1 Like

Wouldn’t that be something where you write an angry letter at your expensive OS or antivir supplier? Slowing down loading of well known data formats just because one is not able to properly parse them is in my opinion the problem of the program that causes it not the problem of the program that uses said data.
Also just because kicad supports it does not mean everyone needs to use it. The normal file formats will continue to be supported so worst case would be you need to itterate over all library files and decompress them using some script.


@Seth_h another possible improvement suggestion: The number of supported 3d model file endings seems to grow with every release. Might it be time to end the use of the filename as an indicator what file to load? (how about using the file name minus extension in the footprint properties? One could still allow file endings to be present for backwards compatibility reasons. For example use it as an indicator for the “preferred” file or completely ignore it. This would need to be smart enough not to die if the filename contains a dot character at any other point then the file ending.)

Like it or not Windows has weaknesses in dealing with opening huge numbers of files. KiCad developers should at least try not to make things worse.
If the compressed version is not slower, I am all for reducing the size, as 3D models will only increase in number

1 Like

In computing there is always a tradeoff between size and computation effort. For 3d models we might just choose the former as a default and offer a script as i described for people who have the hard disc space and want the latter.


Additionally: unlike with symbols and footprints there is no tool that needs to iterate over all files to extract meta information (like the footprint/symbol name) as step or wrl has no “fields” in it. So there really is not such an impact on speed as you might fear.

And I presume that the windows installer is aleady compressed so this will not make a big impact.

Also I think that windows has a problem with a number of files(a lot) and not with their size.

Note by mod: this should have been the top post, sorry

@Seth_h
I’ve seen you are about to support ‘.stpz’ format for KiCad v6 cycle
at the mailinglist

On a side note, I do plan on implementing the standard .stpz, or
compressed STEP[1], format during v6. But this will be in the context
of OpenCascade.

Are you planning to support also VRML compressed file format?
If so it would be a chance to reduce the disk space for packages3d to 1/5 or even 1/10 (5 times or 10 times smaller) of the uncompressed file format.
Libraries could be deployed as compressed wrz and compressed stpz and handled on the fly when using the 3D viewer or STEP exporter…

This may not affect the installer size but it will affect the size on disk. Note that @qu1ck has a patch set at https://gitlab.com/kicad/packaging/kicad-win-builder/merge_requests/86 that will fix the installer size.

It is compressed, so installation time might improve as the files are not being expanded.
This avoids some disk fragmentation, which is known to make running on Windows very slow.

About time. Back to binary. It’s just better! :sunglasses:

@Seth_h
I just tried to transform the installed kicad-packages3D-5.1.5 library to the compressed format and here the result:

kicad-packages3D-5.1.5
wrl,step
11,469 items (and 3 hidden), totalling 5.3 GB
wrl->wrz,step->stpz
11,469 items (and 3 hidden), totalling 834.9 MB

2 Likes

Compression should improve loading time from HDD for sure. Time for loading smaller size file and decompress it should be less than loading much bigger uncompressed file. As we all know small file occupy whole sector on disk so a massive numbers of files can waste some disk space as well. It will be good if Kicad could compress multiple files into one zip file, for example like jar files in java.

Hi all!
For the discussion, please also mind that the 3d viewer has its own binary cached models (not sure how it is behaving at moment) so for loading speed proposes, into 3D viewer, it only matters the first time the step /wrl/or the compressed file is loaded… After that it will use the cached file.

That would make sense only if 3D models used in a project were inside that project. Each jar file is a group of files which belong together. The 3D models library of KiCad isn’t like that. In short, it wouldn’t be useful and not worth it.

Nice, last time I used a 83MB stp model (and others) in same 3D Viewer because wasn’t possible the compressed version.
How to compress stp->stpz? (Can I just zip it and rename to the correct extension?) It is available on last Nightly?

to compress a STEP file you can use FreeCAD >= 0.18
FreeCAD can open and save .stpz compressed STEP file format.

KiCAD doesn’t have direct support for .stpz file format ATM; it will be available in version 6 later on.
ATM you can steel use a STEP compressed models through StepUp… just associate the VRML model to a kicad board and convert it with kSU.

Mainly it is a useful feature for saving disk usage…

@kammutierspule is there a way to delete the 3d cache? I think this should be addressed, considering that kicad has different versions of 3D libraries

I believe it’s in-memory cache so simple kicad restart deletes it.

It is in a specific 3d folder inside kicad profile… it will last forever there

You mean %appdata%\kicad\3d ? That folder stays empty for me, I don’t think it’s used anymore.