Eeschema UI blocks for ~10s for each component chooser selection

[tai]

I’m having an usability issue with Eeschema - everytime I try to place a new component, it takes ~10s for component chooser dialog to open.

I’m using the latest KiCad 5.1.9 (installed from scratch, after removing prior version and per-user appdata).

This is actually a long-standing issue for me since earlier version of KiCad5. I’ve been staying with KiCad4 due to this issue, but now checking to see if I can migrate.

I traced kicad.exe with Process Monitor, and found huge amount of file accesses to *.lib files every time.

Here’s the screenshot of this disturbing activity:

…

As you can see, loading started at around 0:34:22 and ended at around 0:34:30 - taking 8s.

These library files are placed on recent PCIe NVMe SSD. Also,this PC has 40GB RAM, so these files are presumed to be in memory cache as well. I have no third-party Antivirus software installed except genuine Windows one.

This looks like very inefficient file accesses happening here. This has been a blocker for me to use KiCad5 for years. Is it just me seeing this? Do all other people just wait for ~10s for each component placement? Is there anything I can do about it?

=== Related posts ===
Parts Chooser So slow to load, can it stay open? - Community / Feature Request Chat - KiCad.info Forums

Schematic choose component slow - Schematic / Library Symbols - KiCad.info Forums

=== Version information ===
Application: Eeschema
Version: (5.1.9)-1, release build
Libraries:
wxWidgets 3.0.5
libcurl/7.71.0 OpenSSL/1.1.1g (Schannel) zlib/1.2.11 brotli/1.0.7 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.3.0) libssh2/1.9.0 nghttp2/1.41.0
Platform: Windows 8 (build 9200), 64-bit edition, 64 bit, Little endian, wxMSW
Build Info:
wxWidgets: 3.0.5 (wchar_t,wx containers,compatible with 2.8)
Boost: 1.73.0
OpenCASCADE Community Edition: 6.9.1
Curl: 7.71.0
Compiler: GCC 10.2.0 with C++ ABI 1014

Build settings:
USE_WX_GRAPHICS_CONTEXT=OFF
USE_WX_OVERLAY=OFF
KICAD_SCRIPTING=ON
KICAD_SCRIPTING_MODULES=ON
KICAD_SCRIPTING_PYTHON3=OFF
KICAD_SCRIPTING_WXPYTHON=ON
KICAD_SCRIPTING_WXPYTHON_PHOENIX=OFF
KICAD_SCRIPTING_ACTION_MENU=ON
BUILD_GITHUB_PLUGIN=ON
KICAD_USE_OCE=ON
KICAD_USE_OCC=OFF
KICAD_SPICE=ON

[paulvdh]

I am having the same issue (I think), except that it takes about 30s for me on my slower PC.

In this thread there is a screenshot I made from the task manager, which shows footprints are loaded via a single CPU thread (after an initial short burst), while Seth_h posted a screenshot in which all CPU threads do their best to get the job done quick.

Only one core involved in generating footprint info cache

This tread was split off from another as it turned out to be (possible) bug related, so some comments may be out of context. I did not think of this as a bug before, but now I’ll do some more testing to try to get some more symptoms clear before following Seth’s advise to open an issue on gitlab. For some time now, I’ve been amazed at how few of the applications I use make use of multiple threads. (For example searching though a 1000+ page .PDF datasheets on a single thread). So if anyone her…

Can you verify that it also runs on only one thread on your system?
Your link to the “Parts Chooser So slow” may be another issue, related to the network. (Unless the network thing causes KiCad to drop out of the multi-threaded mode for loading symbols.

Application: Eeschema
Version: 5.1.9-73d0e3b20d~88~ubuntu20.04.1, release build
Libraries:
    wxWidgets 3.0.4
    libcurl/7.68.0 OpenSSL/1.1.1f zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.2.0) libssh/0.9.3/openssl/zlib nghttp2/1.40.0 librtmp/2.3
Platform: Linux 5.4.0-70-generic x86_64, 64 bit, Little endian, wxGTK
Build Info:
    wxWidgets: 3.0.4 (wchar_t,wx containers,compatible with 2.8) GTK+ 3.24
    Boost: 1.71.0
    OpenCASCADE Community Edition: 6.9.1
    Curl: 7.68.0
    Compiler: GCC 9.3.0 with C++ ABI 1013

Build settings:
    USE_WX_GRAPHICS_CONTEXT=OFF
    USE_WX_OVERLAY=ON
    KICAD_SCRIPTING=ON
    KICAD_SCRIPTING_MODULES=ON
    KICAD_SCRIPTING_PYTHON3=ON
    KICAD_SCRIPTING_WXPYTHON=ON
    KICAD_SCRIPTING_WXPYTHON_PHOENIX=ON
    KICAD_SCRIPTING_ACTION_MENU=ON
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=ON
    KICAD_USE_OCC=OFF
    KICAD_SPICE=ON

[tai]

Seems 2 CPUs are being used for my case.
Here`s the screenshot of Resource Monitor when I tried to place a component:

CPU1 and CPU2 are being used, so it’s not single-threaded. But since there are 8cores (4C8T) to spare, these are not used effectively as well.

[paulvdh]

It apparently uses two cores, but each at about 50 percent, while in both My and Seth_h’s screenshot CPU utilization goes to 100% for the CPU core involved.

It could be that it switches between those two CPU’s at a higher frequency then the resolution of the measurement.

I am also still in doubt whether to make a bug report for this.
(It does not affect me on V5.99 and I hope KiCad V6 will be released “soon”).
Are more people reading this affected?

[tai]

Just tried V5.99 and the speed is much faster
I wasn’t aware KiCad6 is coming soon, but I guess I just need to migrate to KiCad6, instead of 5!

[paulvdh]

KiCad V6 is not coming that soon. It can still be about half a year, but that was what I thought 7 months ago too.

KiCad-nightly V5.99 is not a finished product.
Nightly versions are a common practice for open source projects. It is a way in which normal users can help with finding and reporting bugs.

Is it a good idea to use a nightly build version?

Short answer: If you have to ask, you should probably stick to the stable release. What are nightly builds? As the name implies, nightly builds are generated every night from the current master branch of the source repository. They are meant to allow a wider audience to more easily test new features. What are the dangers when using nightly builds? Nightly builds like all software have bugs. Because they have new not yet widely tested features you must expect a higher number of bugs. This inclu…

Once a project is saved in KiCad-nightly V5.99 you’re committed. File formats have changed significantly and there is no (easy) way back to V5.1.x for that project.
If you are a company and your livelihood depends on a properly working KiCad, then it’s probably not a good idea to migrate now.

All that said. KiCad-nightly V5.99 is getting quite usable, and (most) bugs are getting smaller, though it still happens occasionally that a bug is introduced that makes KiCad unusable (which usually gets squished in one or two days).

[davidsrsb]

I used to complain about parts loading years ago, but solved the problem with a SSD drive. First load takes around 10 seconds. Following parts chooser actions are almost instant in the same session.
When I was using a magnetic drive, fragmentation could take first load over a minute.

If this happens every time for you, something is invalidating the cache

[craftyjon]

@davidsrsb

I used to complain about parts loading years ago, but solved the problem with a SSD drive. First load takes around 10 seconds. Following parts chooser actions are almost instant in the same session.

First load on 5.99 should be a lot faster these days, too

[davidsrsb]

I don’t want to force people to 5.99 yet.
There is something wrong with the PC, if the cache is getting invalidated.
What does Task manager show for memory use? If there is something using all the RAM, the libraries will get bumped out of cache.

[marekr]

I wonder if the component chooser is banging on wxFileName which never ends well

@tai

This is actually a long-standing issue for me since earlier version of KiCad5. I’ve been staying with KiCad4 due to this issue, but now checking to see if I can migrate.

It helps to report on gitlab so issues like this get addressed…

[tai]

That’s not the case - as I wrote, this PC has 40GB of RAM and has many GBs are unused.

As Resource Monitor shows, I have 15GB free (not even used for caching I presume) in typical usage, when I’m not running memory-intensive workload.

Regarding the “cache”, does KiCad expect OS to do the filesystem-level caching or does KiCad holds its own app-level caching? From my ProcMon trace, KiCad is always (re-)opening all *.lib files whenever I try to place a new component.

These *.lib files needs to be loadded at least once, but not sure why it’s loading everytime as load result usually never change unless you add new component or library.

[tai]

Does it worth reporting at this point, when we are expecting to see KiCad6? If so, then I’ll submit a ticket to gitlab.

[craftyjon]

There is an internal cache in KiCad that makes it so that the symbols should be loaded from disk only once. This cache was present in 5.x and is still present in 5.99. It’s possible that some issue is causing the cache to be invalidated for your machine all the time, so you see the load every time you go to place a symbol – this is not expected behavior even in 5.1. I would recommend you report it, because we did not intentionally (as far as I remember, at least) fix an issue that would have caused the cache to not work in 5.1 but start working in 5.99.

[davidsrsb]

I agree, raise this as an issue. I am using 5.1.9 on one machine and pre 5.1.10 “Testing” on another and both of them load slowly once at the start of the session and then run from cache.