Only one core involved in generating footprint info cache

Software
1

This tread was split off from another as it turned out to be (possible) bug related, so some comments may be out of context.

I did not think of this as a bug before, but now I’ll do some more testing to try to get some more symptoms clear before following Seth’s advise to open an issue on gitlab.

For some time now, I’ve been amazed at how few of the applications I use make use of multiple threads. (For example searching though a 1000+ page .PDF datasheets on a single thread).
So if anyone here knows something about general threading issues with Linux (Mint 20), please comment…

-------- 8<------------ 8<------------ 8<------------ 8<------------ 8<----
Here a screenshot of what my PC does when I open the footprint editor.
It takes about 30 seconds to load the libaries and during that time one of the cores is maxed out, and the others are idling.

image
image911×866 117 KB

Apart from opening the Footprint Editor, KiCad works quite well on this old PC. I can even open and work with the A64-Olinuxino, which is a more complicated PCB then I’ll ever design myself.

My PC is a probably around 8 years old I5 with 4 cores / 8 threads, and KiCad’s libraries are on an SSD I bought in 2014. I do not know how this old PC compares to newer hardware, but from the CPU History it sure looks like loading the libraries could benefit from a bit of multi threadedness.

Weird thing though:
If I completely close KiCad, then restart and open the Footprint Editor again, then “all stuff” is read from Cache in Ram and the Footprint Editor starts in 2 or 3s. This only works if KiCad is started with the same project. If it’s started with another project, then there is again a 30s load time.

Application: KiCad
Version: 5.1.6-c6e7f7d~87~ubuntu20.04.1, release build
Libraries:
wxWidgets 3.0.4
libcurl/7.68.0 OpenSSL/1.1.1f zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.2.0) libssh/0.9.3/openssl/zlib nghttp2/1.40.0 librtmp/2.3
Platform: Linux 5.4.0-47-generic x86_64, 64 bit, Little endian, wxGTK
Build Info:
wxWidgets: 3.0.4 (wchar_t,wx containers,compatible with 2.8) GTK+ 3.24
Boost: 1.71.0
OpenCASCADE Community Edition: 6.9.1
Curl: 7.68.0
Compiler: GCC 9.3.0 with C++ ABI 1013

Build settings:
USE_WX_GRAPHICS_CONTEXT=OFF
USE_WX_OVERLAY=ON
KICAD_SCRIPTING=ON
KICAD_SCRIPTING_MODULES=ON
KICAD_SCRIPTING_PYTHON3=ON
KICAD_SCRIPTING_WXPYTHON=ON
KICAD_SCRIPTING_WXPYTHON_PHOENIX=ON
KICAD_SCRIPTING_ACTION_MENU=ON
BUILD_GITHUB_PLUGIN=ON
KICAD_USE_OCE=ON
KICAD_USE_OCC=OFF
KICAD_SPICE=ON

1 Like
2

Yup, the “many separate files” footprint approach is definitely hurting performance. We’re thinking about possible improvements that wouldn’t be too painful for a future version.

1 Like
3

In the screenshot only one CPU core is active at any time for KiCad. The “noise” in the lowest 20% is from Firefox, which is one of the very few applications I use that can use all those CPU cores.

It would seem that loading the libraries with as many threads a the CPU
has would be a significant boost in this single slowest part of KiCad. 30s is excessive, even with so many small files, and from the graph the bottleneck is clearly the single threaded CPU speed, which does all disk I/O in a single thread.

And that while these days we’re living in the age of 64 core 128 threads, Threadrippers.

Are any parts of KiCad multi-threaded? Sometimes I see a short spike on all cores, such as in the first second of opening the Footprint Editor in the screenshot above.

[Edit] Oops, forget this. I goofed up with the search.
A quick search at gitlab.com/kicad does not give any results for either “multi” or “thread”.

1 Like
4

Things should already be faster in v5.99 thanks to https://gitlab.com/kicad/code/kicad/-/merge_requests/411

1 Like
5

We thread many parts of KiCad, including the footprint loading. This is what the same test looks like on my machine. I tried twice, removing the fp-info-cache each time to force the regeneration. If you open a bug report at the tracker, we might be able to figure out the difference and see if there’s something in code that we can address.

image
image1285×1264 212 KB

1 Like
6
1 Like
7

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.