KiCad: The case for Database driven design

i’ve been testing the database libraries with great satisfaction, expecially the
new search capability in the symbol chooser, it works so much better than in the
system i’m currently using.

i have a couple of questions slightly related to the topic of DB driven design:

  • afaik, up to now KiCAD could not could not propose different schematic symbols
    for the same component. With the intriduction of kicad_dbl has something changed
    in that reguard?
    Could be possible to have a mechanism similar to the selection of footprints,
    with multiple names separated by ‘;’?
    This is not a fundamental capability but it happens that me and my colleagues
    sometimes have a different preference on graphical rapresentation on schematic.
    I usually like a more compact and crammed schematic, some collegues likes more
    ‘sparse’ rapresentation.

  • it happens that under ubuntu (20 and 22), the kicad-nightly package is incompatible
    with the ‘unixodbc’ package that contains the ‘isql’ command, useful to check the correct
    configuration of the datasources. can something be done about that?
    Some suggestion of alternative methods to check the datasource configuration?

thanks, C.

1 Like

Not yet. If you make a feature request for it, we can discuss it

What error do you get? I run Ubuntu 20.04 and have both packages installed.

under 22.04 when i try to install kicad-nightly, apt propose to remove unixodbc and viceversa:

> sudo apt-get install unixodbc
...
The following packages will be REMOVED:
  kicad-nightly
The following NEW packages will be installed:
  unixodbc
...

OK, I found the problem there and will fix it in the Ubuntu package.

2 Likes

I know this feature is new and highly undocumented, but I could use a little nudge in the right direction as I’ve been struggling for the past hour to get a mimimal working example library up and running.

I have to admit I am not very familiar with the nomenclature and tools for database work. I’m using a pretty convoluted setup of a local WAMP server running MySQL, and I’ve created my test database with HeidiSQL. I’m sure there are easier ways, but I managed to export my test database as a .sql file.

My main problem (besides a general lack of knowledge and experience in this area :upside_down_face:) is that I fail to see where/how the path to the database file is specified in the configuration file. I have tried simply putting my “SymbolsDB.sql” database in the same folder as the “SymbolsDB.kicad_dbl” configuration file, but this was a wrong assumption.

I guess what I am looking for has to do with the DSN and/or connection string, but after consulting Google and reading a bit at e.g. SQL Server connection strings - ConnectionStrings.com , I want to cry under my desk :smiling_face_with_tear:

Well, for the KiCAD side of the game, now there is enough documentation to let you start.

for the DB part, that’s up to you to configure, KiCAD should be able to interface with any DB as
long as it has a ODBC driver to use (ODBC is a kind of standardized protocol that permits you
to interface in the same way regardless of which DB you use, MySQL, SQLite, MariaDB, … )

the misunderstanding i think that is that you are extracting a file from the contents of your DB,
well… DBs usually works in a different way. On your machine (or on a remote server)
you have a MySQL server, the client (kicad) does queries to the server, doesn’t read/write files.
Think of it like what happens with a browser and a web server, they exchange information via
certain protocols (http), doesn’t exchange ‘files’.

to try to get you started we need some info:
Which OS you are running KiCAD? do you have the MySQL server on the same machine or on a remote server? do you have a management tool like MySQL workbench installed?

p.s. don’t cry, i can show you my connection string:
“connection_string”:“dsn=EDLDB”
pretty simple, isn’t it? :slight_smile:

1 Like

i can confirm.
thanks.

1 Like

Thanks for your reply Claudio.

the misunderstanding i think that is that you are extracting a file from the contents of your DB

Yes, I’m sure I am mixing up terminology (e.g. database and table).

I’m working on a Win10 machine, and the MySQL server is running locally on a WAMP server.
Then I use HeidiSQL to connect to the MySQL server and create databases and tables. The relevant connection settings are hostname (127.0.0.1:3306) and username/password, but I am at a loss on how these parameters are specified in this part of the config file. Username and password are trivial, but that’s about it.

    "source": {
        "type": "odbc",
        "dsn": "",
        "username": "",
        "password": "",
        "timeout_seconds": 2,
        "connection_string": ""
    },

p.s. don’t cry, i can show you my connection string:
“connection_string”:“dsn=EDLDB”
pretty simple, isn’t it? :slight_smile:

My desire to cry hasn’t decreased - my confidence has though :wink: I fail to see how a string such as “EDLDB” translates to (what I assume is what I need in my case) 127.0.0.1:3306.

An ODBC connection is driver-dependent: each different database driver supports different parameters, etc. So, the ODBC part is very generic and does not include things like hostname and port. Those parameters need to be passed to the driver to do the right thing with.

There are two ways you can set up an ODBC connection:

  1. Set up a DSN (data source name) on your system, which has all the right parameters for the driver. Then you can tell KiCad to just connect to this DSN. This is what Claudio has done.

  2. Specify a connection string, which lets you include a choice of driver and all required parameters in the KiCad config file.

See MySQL Connector/ODBC 3.51 Connection Strings - ConnectionStrings.com - if you have this MySQL ODBC Connector version installed, your connection string might look something like the first example:

Driver={MySQL ODBC 3.51 Driver};Server=localhost;Database=myDataBase;User=myUsername;Password=myPassword;Option=3;

4 Likes

Ok, win. Just a bit outside of my ‘comfort zone’ (running linux here).

First of all, have you installed the ODBC driver to connect to your MySQL database?
if i’m not mistaken for MySQL it should be called:
mysql-connector-odbc- something something - win something .msi
should be available from the oracle mysql site.

install it.
(Reboot? don’t remember).
Search from the search tool ‘ODBC’, it should bring you around here:
Control Panel → Administrative tools → data sources (ODBC)
Look around there there should be something like a ‘user DSN’ tab.
add a new DSN of MySQL odbc type. this (or something vaguely similar) should appear:

fill the relevant data of your server.
cross you fingers, click test.

If the test is OK than insert a line on the kicad_dbl just like mine
but instead of EDLDB insert the name you have chosen for your datasource.

    "source": {
        "connection_string": "dsn=<your data source name here> "
    },

that should be it. (hopefully)

5 Likes

Thank you both for your extensive assistance and patience. Between the info contained in both your posts I have managed to get it up and running.

6 Likes

I understand your description however I don’t see how it is “core” to a successful design.

While I understand and support a company based “standard” parts I don’t see how it would deal with new designs for parts that are not “common” i.e. resistors, capacitors, maybe connectors and some common semi’s.

Isn’t there a risk of “using what is there” instead of what is best?

And, unless I misunderstand, a “completed” design would still be connected to the main database. This I feel would be significant risk. Someone could unknowingly add a suffix to a part causing a change in a critical requirement of that component.

Again you didn’t mention the size and type of projects you feel this would be a requirement for.

Considering what i’ve seen in the last couple of years, i’d say the opposite.
If i use stuff i or colleagues have used in the past, ii’s most likely that we have some
in the warehouse, some spare in a drawer or some non working board to scavenge.
Yes, you can not imagine how many times i’ve saved the day scavenging parts
from a broken board or EVB lately. not ideal but you have to do what you have to do.

In my case the successful design is the working design i can deliver in time.

3 Likes

I guess it depends on the product you are providing. I worked in both Aerospace and Automotive. In neither case would parts sitting in a drawer be acceptable candidates to be shipped to a customer. Perhaps they could be used in preliminary breadboards, but nothing beyond that.

IMHO it’s part of the design engineer’s job to insure any parts picked can actual be purchased**. I have in the past ordered a particularly long lead time part during the breadboard phase. Perhaps it might get designed out but usually the engineer knows if the chosen part is likely to remain to the final design.

** They can of course get the purchasing dept / person involved early to insure a smooth transition to production.

this is exactly the reason why you should stick to parts already in the normalization database.

1 Like

As I mentioned, this works for common “jelly bean” parts like resistors, capacitors (some basic types) etc.

In you proposed system who is assigned to keep the database up to date? How will the design engineer know if the database part is destined to stay in production? What criteria is used to add a new component and who vets it’s addition?
If cost is included in the database, who and how is this updated? Last buy, average of the last n buys, if the last buy was only 100 pcs and your design will need 1000 pcs/month how are the costs reconciled?

In a case where the old “standard” part is still available however a newer, better and cheaper part has been introduced in the market place. What part do you choose?

There is a dedicated employee (or department) whose job is to do exactly this.

1 Like

Been there, done that :slight_smile:
I’ve seen these folks/dept being eliminated due to economic reasons. This is not a trivial job. One way of looking at it is the core component database is being updated by someone who is completely separate from the product(s) being designed.

In one company we had a “standards” group (of two). At that time is was still a design engineer but I knew the basic product types that were common in the company, kind of a trivial ability. Normal signal resistors were of type xxxxx and normal ceramic capacitors were of type yyyyy etc.

I have a friend who used to work for Black and Decker. Their design goal was minimal ¢. Every component was scrutinized in painstaking detail for minimum cost. A “database” system as you describe would not work in such an environment.

1 Like

All I can say is that in my experience some form of this database system is quite common in corporate settings (including aerospace) as a way to maintain a list of parts that are available and qualified/approved(/stocked?) for use in new designs.

I agree with you that it is not an easy or trivial job, and that it’s important to have a good process for adding new parts to the database to avoid the “just use what’s approved rather than using the right part for the job” issue. The important part is often the process of qualifying and otherwise vetting the part, rather than just adding it to the database, of course.

Edit: I should also say that a system like this doesn’t in any way replace design, cost, or manufacturing reviews.

1 Like

Oh my, how i’d love to work there as a designer! :neutral_face:
Yes, it will work just the same, but sadly, it will be under direct and complete control of the purchasing dept.
The poor sod that will have to design stuff there will have little voice on the matter.