It should and can be DB agnostic, think of people who have to work with employer supplied DBs. SQL interface required, probably.
1 Like
I guess my point is, it has to ship with something that works out of the box. SQLite is easy. This is the same pattern as used in Django. You can develop a full application with Django using SQLite. From there you can migrate to PostgreSQL or any other DB for deployment.
The other place I’ve seen SQLite used successfully is Solidworks. Solidworks CAM or the upgraded CAMworks package has SQLite as and out of the box option for the database that drives toolpaths, machine specifications, tooling, etc. It works very well.
It’s easier to develop an agnostic interface from the start and then supply a reference implementation which could be used OOtB by simple installations.
3 Likes
SQLite is strictly single user.
PostgreSQL would be a better target, but like it or not, a lot of commercial KiCad users have SAP lurking
Yes. of course. My guess is that the vast majority of users would be well served by SQLite. Migrating to something else should not be a problem for professional users who might need true multiuser access.
Strictly speaking SQLite supports multiuser access for reading. Only one user can write at a time.
1 Like
For this I neither like the idea of baking in a single user (writer) interface from the start, nor committing to a particular database format.
This database driven design is mostly for bigger companies, and it’s quite likely they already have some other PCB design suite and an elaborate database with their preferred parts in place.
So I agree wit retiredfeline to define an agnostic interface combined with some reference implementation.
Also, as an Open Source project, there are already a lot of side projects around KiCad. It’s quite likely that if there is a definition for a good interface and a reference implementation is made, that interfaces to other databases will also be released over time.
1 Like
l do not disagree with this at all. My suggestion amounts to the idea that, if DB-based libraries become the new standard (as opposed to an alternative) shipping with something like SQLite might be a good idea for the vast majority of users. However, we all understand that, at a professional level, the software would have to be able to easily adapt to the many component management methods present in industry.
When I move away from SQLite on other projects my first option is PostgreSQL, followed by MySQL. I rarely go past those two unless the software I’m using requires it. For example, CAMworks (CNC programming tool that works with Solidworks) used MS Access for quite some time. I hated it. Not inherently because of Access, it’s another tool. No issues there. However, Access is like taking a sledge-hammer to the problem. Too heavy. Later on they offered the option to use SQLite. It loads faster, it’s very lightweight and actually feels better to use.
The database library feature will not ship with a database. It will be up to the user to select and configure a database of their choosing, because as was mentioned above, a number of users will be integrating with existing databases rather than creating one from scratch for KiCad usage.
This feature will not become the new standard. It is an optional way to manage libraries that is more suited to larger teams, organizations, and users with special requirements. It will not replace the existing KiCad library project which will still remain the default library mechanism for users.
5 Likes
What craftyjon said -
A database library link does not need to replace the existing way libraries are developed or managed, although I can imagine that its implementation may affect the current way parts libraries are handled.
Also, only a database link is needed. Actually packaging a database with KiCad would be gross bloat for a feature that some may not use. And yes - the link should be agnostic of what database is out there (or at least configurable to several different types of databases. I do believe that this feature, once people start to understand it, adds value far beyond just businesses or large teams. I found it invaluable as a one-man band.
Last, my post was not intended to berate developers or the development process, but to keep this topic alive. Not just with developers, but with the community. The more dialog on it, the better the implementation needs will be known and shaped (although it seems like craftyjon knows exactly what the need is, with respect to it implementation in other layout packages) , and the better understood it will be by the community if/when it is implemented.
That’s fine. I would suggest releasing documentation on this as soon as it might be practical in order to allow people/organizations to start some of the work that needs to be done at their end.
I have worked at companies with a team dedicated to part creation and library management. In other words, EE’s don’t make parts, they ask the library team to create, qualify and add to the library. In my current business every EE creates parts for the library (I use the term interchangeably with database), we don’t have a dedicated library team. In both cases, the transition to a DB-based library for KiCad will require a bunch of work. This could take months to complete. The earlier they can get started the better.
I do realize this is FOSS. The work being done is nothing less than amazing. It will take however long it takes. And that’s the way it is.
The documentation won’t be released until the code is done at a minimum 
There will probably be outlying cases, but I don’t actually anticipate that in the average case it will be too much work. I will be doing some tests to see that this is the case, though.
Drifting a bit off topic, MS Access is in many ways less capable than SQLite and is definitely single user with the very non standard Visual Basic functions
For the record, we were just going to implement ODBC and everyone can then use whatever flavor of database they want. Even DB2 on a mainframe if you want.
4 Likes
Fantastic - I think this (ODBC) is how other platforms do it. Looking forward to seeing this in the future!!
2 Likes
Nice. Can’t wait.
Is there any kind of discussion on what the approach will be at al high level perhaps?
For example, direct placement from the DB into schematic/PCB or will we export from DB to conventional libraries and then place from there? In other words, the “supermarket” model. The DB is the supermarket. The library is your shopping basket.
We are likely to implement this supermarket model ourselves in the next few months. The idea being to use a single symbol and schematic library as bridges from the database into the design.
One advantage of this approach (given how KiCad works today) is that these automatically created libraries are stored with the project. Yes, I know that schematic stores components. However, I like the idea of having design-specific libraries available as well and I don’t think there’s currently a way to export/import from schematic/pcb to stand-alone libraries.
The other interesting thing is that the selection set from the database will be kept in an Excel file. Which means that this will be 99% of the way towards a clean BOM. The only thing missing might be the quantity per component. It also makes it so that recreating the stand alone libraries from scratch is as easy as re-running the DB-to-Libraries utility using the Excel file as the source.
Not a perfect set of ideas but something I have used in the a past with EDA tools that, say, 20 years ago, where at about the same data management level as KiCad is today (in rough strokes).
I am not sure I follow this, but there is not going to be an “export” – a database library will be another type of library you can configure and enable in your system library configuration.
It does not really make much sense to support project-local database libraries, as actually talking to a database will in general require additional software besides just KiCad. But, I think we are talking about different things: the database libraries feature is not about automatic generation of KiCad library files – it is about pulling part metadata from a database in conjunction with symbol/footprint data from existing KiCad library files, at the time when you place a part onto a schematic.
3 Likes
craftyjon -
I only had time to quickly scan it but your spec doc looks fantastic!
My confidence in seeing this implemented is way up seeing that there’s actually quite a bit of thought and understanding put into it!
Thank you for the link!
2 Likes
ODBC is definitely the way to do it, no DB-specific implementations, only standard interfaces.
Just had a chance to go through the document you provided. I need some time to think about it. On the surface it looks like a good path forward.
The one part that caught my eye was:
The actual symbol and footprint data comes from existing libraries that must be present in the library tables just like today.
I understand why, of course, I just haven’t thought through what this might mean.
You are linking to a symbol, which currently contains four mandatory fields. Does this mean KiCad will actually use these fields or will it only grab the symbol graphics? I would suggest all you want is the graphics.
What comes to mind first is that this would establish a situation where cloning the environment would require full copies of the relevant symbol and footprint libraries as well as access to the database and perhaps even some version control.
What happens if someone edits a symbol or footprint in a linked library? Now the database pulls graphics that are different from when the part was created.
It might be best to actually import the graphics definition into the database record and make the database the sole repository of truth in the design. Changes to the symbol or footprint libraries would not break the database definitions.
If the symbol and footprint editor are able to work with a definition passed to them without having to access a library, one could actually consider editing and maintaining the graphics directly on the database without having to touch conventional libraries. Being that everything is coded in the form of s-expressions --text-- inclusion in the database as a field as well as passing back and forth to the symbol and footprint editors should be doable.
Just my initial thought. I’ll come back to this if something else pops up.
Correct
Correct, just like the existing library system. But just like the existing library system, the points at which a schematic or board is updated to incorporate changes from the library are chosen by the user as manual operations. It is up to the user or organization to develop any kind of review process / workflow they want to use.
This is not the approach we’re taking at the moment. We could consider this as an option in the future, though (it would always have to be an option though, because there are some workflows where the separation of the metadata from the graphics is important)