I’m working on a index/database for KiCad footprints/symbols on the internet, so searching to see if someone has already created a footprint or library for a part can be easier than blindly clicking through Google.
I’ve got most of the primitives in a working state (footprint parser, javascript renderer, git crawler etc), just a few big things to go (token stream generators, index) and a long tail of little improvements.
Is this something anyone would be interested in collaborating on?
Major tasks that could be useful:
Support rendering fp_arc, fp_circle, fp_poly in the javascript preview.
Writing analysers/token-stream-generators to analyse the footprints/symbols and feed the search index
Writing a parser (backend is written in Go) for symbol libraries (havent gotten around to anything to do with symbols yet)
Javascript - UX for searching (autocomplete / suggestions, etc)
If anyone has a server lying around (I have servers, but they will probably feel slow as they run other stuff as well)
Theres plenty of little stuff to do as well, from CSS to fixing minor inaccuracies in the renderer.
The technology stack is:
BE - Go + sqLite. Probably going to end up with CDB or something for the index cuz sqlite aint fast for that.
FE - Angular 1 + Materialize.
If youre interested, shoot me your github username and I’ll grant you access (I intend to open source as well when MVP is functional & I’ve cleaned it up a bit).
Looks neat! I have been collecting all the footprints I can find in a GitHub repo:
(Reminds me that I need to run an update on that when I am on better internet)
I also have a little script that searches GitHub through the API which works surprisingly well and you could easily do for your site (if you don’t already).
These are set to change in v5 v6 so it’s probably worth waiting till the new format is implemented.
No! v5 is already in feature freeze and will come out soon. It still uses basically the same symbol lib format and schematic file format (with some minor additions to add some features) as v4 did.
The new file format is currently planned for version 6. (so it is a few years away.)
Awesome script and repo! I’ll add it to the ingest sources.
Sqlite has fast generalized performance but is slower than other index systems when doing queries that return heaps of results (ie - search indexes).
I can parse the entire KiCad/kicad-footprints repository without exploding (IIRC this is v5 format). The differences are very minor (attr expression, different drill expression, pads are different, and additional kinds of graphic).
I was talking about the symbol libs. The new format coming in v6 will be closer to the footprint s-expression format. I much prefer the s-expression formats and could never make much sense of the current format (not that I tried too hard).
Not enough features.
Not flexible enough.
(If the developers would see a way to use the current format they would keep it. Sometimes it is simply better to start fresh with the lessons learned.)
A few short cummings of the current format:
The current aliasing system is just not powerful enough. (need the same symbol twice but with different footprints. Guess what you gonna make two symbols. Have fun keeping them in sync for the rest of the lifetime of that part.)
The unit management is non existent (either all units are interchangeable or none are)
No way of adding alternative pin names (Good symbols for micro controllers would require that.)
Only one pad per pin (yes you can stack them but don’t use power input as these are global labels)
Also why does an arc include redundant information? (What happens if they contradict each other)
I am sure the developers have even more reasons. These are only the once i noticed while managing the official libs for a year.
I had some issues ingesting your repo, a number of the submodules are broken (auth required etc) and my library does not cope with that very well. I’ve attempted to pull out the interesting repos directly.
As the volume of parts increases, the search is showing heaps of results (and getting harder to use). Any ideas for how to better rank the information? I’m thinking ranking results based on the number of stars on the Repo.
I would assume that part of why people make personal libs is because they do not agree with the requirements of the KLC. This is totally ok.
Much more important is a correctness check.
Do the pin name/ number pairs agree with the datasheet.
Does the pin type make sense.
If a footprint is assigned, does it agree with the datasheet?
Maybe the datasheet supplied suggested footprint is not quite right and does not agree with the dimensioned drawing of the package. How should that be discovered?
…
In the end you will need to check your library parts by hand. Do not simply trust any library. Not even the official lib. (The official lib has a high quality but we librarians can still miss stuff. There are also some quite old things in there that would no longer be accepted.)
Would only work for symbols as footprints typically do not include order information. (Unless you have a fully atomic lib)
And again, this does not tell you anything about the quality of the symbol/footprint.
There is a reason why we librarians value quality over quantity. A lib of millions of untrusted symbols or footprints is not really worth much.
This ^^^^^ is a big deal. What if you have several different parts, that are supposed to be physically interchangeable, and the dimensions of the parts are very similar, yet they all have different Land Pattern Data for their individual Data Sheets?
Practically speaking, I don’t think theres any way I can match the parts in the database against the respective datasheets. Most parts dont even have datasheets or complete part numbers, not to mention the footprints are buried in some PDF as an image file.
I think those issues should be fixed since I updated everything. It’s mainly when people remove their repos: if you try and git clone deleted repos over https then it asks for authentication, even though it would fail even if you gave it.
