Database symbol library, reloading

Schematic Library Symbols
1

Currently it seems that KiCAD caches the symbol libraries.
As such one seems to need to restart KiCAD in order for it to register changes in the database library.
(Which is slightly tedious when populating the library)

Is this correct or is there a “hidden” feature to invalidate the cache somewhere?

I have also tries to search the bug tracker for feature requests (both open and closed) along the line of either implementing a manual way of invalidating the cache or even better for it to happen automatically.

Does anyone know if this is a planned feature or if I should write a feature request for this?

1 Like
2

Yes, this is a bit painful, but fortunately KiCad starts so fast, it is not too bad. I suspect KiCad loads the database into RAM on startup, so it may be a tradeoff to keep things blazing fast rather than continually going out to the database for stuff.

3

if it works like the HTTPLIBs then just open the Symbol Libraries Manager and press OK, then the HTTPLIB is re-scanned

4

Doesn’t appear to update sqlite based databases, I even tried to touch both the sqlite database and the .dbl file.
It does however make kicad pause for a second when pressing ok

5

Some of it seems to stems from the way git-plm updates the database, I will try doing a slight rewrite of that

6

You might want to give
parts_db_update.sh

#!/usr/bin/bash

GPLMLIBS="ana cap con cpd dio ics ind mpu mcu pwr rfm res reg xtr osc opt art"

for lib in ${GPLMLIBS}; do
    sqlite3 ./parts.sqlite "DROP TABLE IF EXISTS ${lib}" || return 1
    sqlite3 --csv ./parts.sqlite ".import ${lib}.csv ${lib}" || return 1
done

a try
for me I don’t have to restart KiCAD any more, when I create new parts they just pop into existence

I have until now found the following caveats:

  1. Deleted parts hangs around with empty contents in the symbol chooser (then again one probably shouldn’t delete entries to begin with, at least not after they entered “production”)
  2. Description isn’t updated in the chooser list, but it is displayed correctly in the “details” fields in the chooser and inserts just fine.

I can also recommend visidata as a lightweight alternative to libreoffice calc for editing csv files.

Also thanks for publishing git-plm, and you are of course welcome to include the changes in the official repo if you want to.

2 Likes
7

Done, thanks:

Your fix is obvious in hindsight – thanks for figuring this out!

One thing I’d like to do long-term when I have time is develop a HTTP server in Go that watches all the CSV files, and then generates a HTTP interface with the part data. I think this would be an improvement over the database workflow.

1 Like
8

It would probably also be quite simple to generate a while true, sleep a while, check if the source files have a modification time newer than the sqlite database, if true update it.

It would be in perl… and I’m sure also in bash if one can remember the correct white space usage :slight_smile:

1 Like
9

Wow, visidata is nice!

1 Like
10

Indeed, it was recommend to me by some friendly folks over on masterdon when I asked for a good/simple CVS editor.

11

Here you go:

parts_db_watch() {
	echo "watching csv files for changes ..."
	while true; do
		FILE=$(inotifywait -q -e modify -e close_write --format '%w%f' ./*.csv)
		# debounce a bit as things might be moving around when libreofice is saving a file
		echo "csv file changed: $FILE"
		sleep 1
		LIB=$(basename "${FILE%.*}")
		echo "updating db: $LIB ..."
		sqlite3 ./parts.sqlite "DROP TABLE IF EXISTS ${LIB}" || return 1
		sqlite3 --csv ./parts.sqlite ".import ${LIB}.csv ${LIB}" || return 1
	done
}

Gah, that was painful! :smiling_face_with_tear:

12

Well it seems to work fine, and is more energy efficient than what I would have written.
(polling the mtime every N seconds would consume a little power)

You might want to include “inotify-tools” to the list of packages in the README.md

1 Like
13

Interesting. I’ve found that simply adding entries to EXISTING symbol libraries I don’t have to restart KiCad nor do I have to run the parts_db_watch() command. If a new symbol library is added then things have to be tweaked and KiCad restarted. But compared to just adding parts, one typically doesn’t add a new symbol library all that often.