KiCad: The case for Database driven design

Software
1

I recently enquired on the KiCad discord channel about the ability to link a parts database to KiCad, and received the response that Database linked Libraries are planned for the V7 Kicad Release. I made a few further posts about it, but before I pressed the issue further, I wanted to better inform myself on KiCad’s capabilities.

Upfront, this capability is planned for the version 7 release, so if you bear with me and read through this and think it is useful, go vote for and comment on, and provide further input on it here:

https://gitlab.com/kicad/code/kicad/-/issues/7436

A little about my background. I’m not a heavy user of PCB schematic capture or layout tools, but over the years I have need to learn and use them for various work and personal projects. In order of tools learned, I came from Orcad/Orcad Layout, then migrated to Altium Designer, and today use (occasionally) Mentor Graphics Xpedition Designer/Layout in a commercial/professional environment. My day job is mostly related to RF/microwave design (which all of the above tools fail at to varying degrees, forcing us back to caveman use of autocad, etc), but for more normal digital/analog design, work well.

I have need to use a PCB design system again for a personal project, and while I have most of the options available to me, I would really like to migrate to and become a supporter of KiCad. Unfortunately, each time I have evaluated its suitability for a project, I have stopped investing time and effort as soon as I learned that it has not yet adopted the ability to link a parts database. Having used (and built) a database linked design ’environment’ in each of the above named systems (Orcad, Altium, Mentor), I find the benefits so core to a successful design that not having that capability in KiCad a non-starter for consideration. That said, I see KiCad being so close to rivaling the above commercial offerings, that I’m here to plead the case for database driven part libraries, with the hope that others will fan the flame and make this happen, pushing KiCad into the realm of professional PCB layout tools.

I refrained from doing this for a month, and to make sure I wasn’t speaking/pleading out of touch, with KiCad’s capabilities, have been spending time better learning this tool via Peter Dalmaris’ ‘KiCad Like a Pro Course’. I haven’t completed the whole course, but have started a simple design of my own that is do-able (because its so simple) without the subject database libraries, but this can’t wait. KiCad needs this capability.

So what is it? Rather than storing part information in the symbol and linking to a footprint, a parts database centralizes all part information parametrically in a database with links to symbols, footprints, 3d models, datasheets, images, etc.

The Benefits/Opportunities/Pro’s of this approach are multi-faceted.

  1. A library of commonly used partsis developed
  2. All pertinent information is stored and managed from one central place (a database)
  3. All part parameters are stored and filterable parametrically (This is the most important from a design perspective!!)
  4. Each part is standalone and once built, need to only be maintained as needed (i.e. pricing, or current inventory if those are important to the user)
  5. A small database can be developed quickly by downloading and populating whole families of parts
    1. A part is built only one time. You learn it once, gather all the information once, and reuse that information many times going forward. But the work in building the part is done only once!
  7. Once setup, you’ll never go back!!

The Con’s of this approach are minimal in my opinion

  1. Takes time to build.
  2. Can get out of date (pricing most likely, but parts obsolescence also)

What do I mean by parametric? Basically, you populate the fields of a part with the following filterable information

User/Company Part Number
Description
Part Family
Part Type
Manufacturer
Manufacturer Part Number
Value
Voltage Rating
Current Rating
Power Rating
Temperature Type
Tolerance
Temp Min
Temp Max
Height
Package
ROHS Compliant
Status
Notes
Distributor
Distributor Part Number
Price
Characteristic (for something that doesn’t really fit any of the other categories but doesn’t warrant its own category)
Sub-Characteristic (for something that doesn’t really fit any of the other categories but doesn’t warrant its own category)
Preferred
Substitutable
Alternate Part Information
Datasheet
Library Path
Library Ref
Footprint Path
Footprint Ref
Temp Coef
Library Ref was the symbol name (reused as needed)
Footprint Ref was the footprint name (reused as needed)

Yes, it takes time to build this, but not a tremendous amount of time for the benefit gained. It took me a few days to populate a data base of several thousand parts a few years ago by downloading parametric part data from DigiKey and populating an access database with the above fields (not every part will have every field populated). But once built, you have developed an “ecosystem” that you design within and build upon as you move forward. And that was just one person’s effort. A community build parts database could be phenomenal!

So once the database is built, if I needed a resistor, I’d filter on type (resistor), value (100 ohm for example) and footprint (0805). That would get me down to a handful of parts to choose from. A preferred part is something I might keep in stock

Not all fields would be populated, only those pertinent to a part. I did this in the past using Altium, and 3D models weren’t available for hardly anything then, but a parts height, combined with the footprint, could give a very good representation…

Again, when you work like this, you are much more working in a design ecosystem that on just a schematic.

The primary benefit for me, as a small company, was the ability to parametrically search the database for a part, and have all the info on that part in one place. If I needed a capacitor, I filtered on caps, then the cap value, then maybe the tolerance or temp rating, etc. I had the data sheet for that cap linked so I could easily get more info on it. And all that info was made part of the electronic schematic (although not all was shown).

When I created a BOM from the finished schematic, the manufacturer part number, digikey part number, and price, along with the total quantity and the ref designator for each instance of its use was pulled into an excell sheet. All that parametric data was also available any time I touched the part. I found it indispensable, and I can’t imagine building a board with more than about 30 parts on it and not doing it that way now. Also, all the footprints and schematic symbols were controlled by me - no update could wipe them out, etc… My database was built in Access, although it could have easily been migrated to something else.

So that’s the ask, and the pitch for the ask. As awesome as KiCad has become, I don’t think it will seriously compete with commercial offerings until this capability is implemented and implemented well.

If you think the above sounds like the cat’s ass (and it is, once you work this way), then I again ask that you give it some momentum for the next release, and development ASAP.

https://gitlab.com/kicad/code/kicad/-/issues/7436

Looking forward to other’s thought’s and comments.

10 Likes
2

This is something I’ve done a little of locally as well. I’d like to see a selection of parts databases myself. As a example, I don’t use SMD components so have little need of them. Having a database that is broken into smaller pieces would be ideal. Examples, SMDs, audio, RF, power, computer/logic etc are some examples. I’d skip RF and SMDs since I don’t use those but others may want to skip old style hand soldering components that I use. By having them broken into pieces, one can install only what they need and avoid cluttering up the listing of components within Kicad or other programs as well. I’m not limiting this just to Kicad.

I do wish there was a single place where I could download all the files I would need regardless of who made them and pick what I need by types. Someone else may come up with a better way or dividing them up but still, as it is; google, end up who knows where to get a symbol, footprint or 3D model if available, download, install, add to Kicad or other software, and repeat this sometimes for each single part.

Does this add ideas to this topic?

3

Sounds like the propsed v7 enhancements amount to providing a link field to a database so the bulk of the work is cleaved off elsewhere and presumably can be a separate project rivalling KiCad in complexity, mirroring the real world, also with its supply chain problems at the moment.

4

The idea is that you build the database you need. Sure a large community database would be fantastic (I’d certainly be willing to donate my own to the community as a starting point/option), but there is no need to download the entire thing. You either build what you need or download the parts of a community database that you need. I could see some sort of “database manager” that would allow one to downselect parts, and also fields as necessary.

1 Like
5

That’s one way of looking at it. Nothing more than a link is really needed, but at the same time, you want to be able to pull in the tremendous work that has already been done on symbols and foot prints.

Re: rivaling KiCad in complexity - I doubt it. KiCad is quite obviously complex!! its really pretty dumb, just a lot of data, appropriately managed.

Re: mirroring real world and current supply chain problems. The dream would be that a public database would gain enough momentum that vendors would start to update it with their current pricing and inventory, but that is a Looooooong way away. KiCad just needs the basics now.

6

I think the left corner approach is the right one. Basic functionality would be a part ID field as a key into a database (and the interface would have to be DB agnostic to cater for all the available ones out there, even if you only count the open source ones). Then functionality to pull in symbols, footprints, 3D models and so forth. So it can be as big an undertaking as participants want it to be.

7

We have been using an open source software called PartKeepr. It is specifically designed for inventory management in electronics. the link is: https://partkeepr.org

I found out today that there is an add on tool to KiCad to help the integration with PartKeepr, KC2PK
at https://github.com/Gasman2014/KC2PK

This tool aims to integrate electronic component management using electronic designs produced in KiCad and inventory management using PartKeepr.

I hope this helps.

1 Like
8

Another open source project is InvenTree - InvenTree Documentation. It also has a KiCad plugin: GitHub - sparkmicro/Ki-nTree: Fast part creation in KiCad and InvenTree.

9

I agree , kicad simply needs a link to a database and a way to search it to select parts.

10

Somewhat depressing that a plea for core functionality that prevents KiCad from competing on the professional level more or less gets no traction. I see functionality pursued that seems ‘neat’ but more or less unnecessary (rendering, etc) get folded in that makes me wonder if KiCad will ever get to a point where it is a serious competitor to Altium, Mentor, etc… Perhaps just wishful thinking?

12

I mean, our official haunt is the issue tracker not these forums

And you already linked this:

Also if you take note
image

So I’m not sure what you are complaining about.

We are all volunteer developers, we work on what we enjoy because it is our free time we are wasting not yours. There is a commercial option for $$$$ for features to be developed on demand. Otherwise as you see, database library support is planned for 7.0, it’ll get worked on when the developer that planned to work on it has free time to do so.

5 Likes
13

While I don’t agree with some of the characterizations made in this thread, I do agree with the general idea that database libraries that integrate all component data would be a significant step forward.

I would suggest SQLite as a target for this. It’s an easy starting point for individual usage. From there, migration to other database systems for more advance use cases should not be a problem. It plays nice with Python and even opens-up the option to create front ends with Django and Flask.

I also understand that this might just be too complex for hobby/educational users. KiCad, as it exists today, is easy to get into. Component creation isn’t complex. Users don’t have to endure the kind of thing one must live through in order to create and support large libraries of vetted components for commercial use.

In that context, not sure what the right answer might be. I know professional users are rooting for KiCad and want to get away from the pain some of the high-dollar closed-source EDA tools have become. I certainly do. Not sure that means KiCad will bend to satisfy pro users at the potential expense of everyone else. Don’t know.

14

It should and can be DB agnostic, think of people who have to work with employer supplied DBs. SQL interface required, probably.

1 Like
15

I guess my point is, it has to ship with something that works out of the box. SQLite is easy. This is the same pattern as used in Django. You can develop a full application with Django using SQLite. From there you can migrate to PostgreSQL or any other DB for deployment.

The other place I’ve seen SQLite used successfully is Solidworks. Solidworks CAM or the upgraded CAMworks package has SQLite as and out of the box option for the database that drives toolpaths, machine specifications, tooling, etc. It works very well.

16

It’s easier to develop an agnostic interface from the start and then supply a reference implementation which could be used OOtB by simple installations.

3 Likes
17

SQLite is strictly single user.
PostgreSQL would be a better target, but like it or not, a lot of commercial KiCad users have SAP lurking

18

Yes. of course. My guess is that the vast majority of users would be well served by SQLite. Migrating to something else should not be a problem for professional users who might need true multiuser access.

Strictly speaking SQLite supports multiuser access for reading. Only one user can write at a time.

1 Like
19

For this I neither like the idea of baking in a single user (writer) interface from the start, nor committing to a particular database format.

This database driven design is mostly for bigger companies, and it’s quite likely they already have some other PCB design suite and an elaborate database with their preferred parts in place.

So I agree wit retiredfeline to define an agnostic interface combined with some reference implementation.

Also, as an Open Source project, there are already a lot of side projects around KiCad. It’s quite likely that if there is a definition for a good interface and a reference implementation is made, that interfaces to other databases will also be released over time.

1 Like
20

l do not disagree with this at all. My suggestion amounts to the idea that, if DB-based libraries become the new standard (as opposed to an alternative) shipping with something like SQLite might be a good idea for the vast majority of users. However, we all understand that, at a professional level, the software would have to be able to easily adapt to the many component management methods present in industry.

When I move away from SQLite on other projects my first option is PostgreSQL, followed by MySQL. I rarely go past those two unless the software I’m using requires it. For example, CAMworks (CNC programming tool that works with Solidworks) used MS Access for quite some time. I hated it. Not inherently because of Access, it’s another tool. No issues there. However, Access is like taking a sledge-hammer to the problem. Too heavy. Later on they offered the option to use SQLite. It loads faster, it’s very lightweight and actually feels better to use.

21

The database library feature will not ship with a database. It will be up to the user to select and configure a database of their choosing, because as was mentioned above, a number of users will be integrating with existing databases rather than creating one from scratch for KiCad usage.

This feature will not become the new standard. It is an optional way to manage libraries that is more suited to larger teams, organizations, and users with special requirements. It will not replace the existing KiCad library project which will still remain the default library mechanism for users.

5 Likes