Database Library unable to find Symbol and Footprint

aberger · June 6, 2025, 7:46pm

I am migrating from Altium’s SVNDBLib component database and trying to connect KiCad to the same component database. I have a .kicad_dbl file that successfully defines a connection to a PostgreSQL database that contains my components. However, the Symbol and Footprint are not properly resolved.

The database has columns “Library Ref” and “Footprint Ref” which contain the filenames of a .SchLib and .PcbLib file, respectively. (e.g. “MK64FN1M0VMD12” and “144-MAPBGA”, respectively).
I’ve added the path to the folders containing these files as “Environment Variables” in the Configure Paths Window.
I’ve added the Symbol Library to the List of Symbol Libraries in “Manage Symbol Libraries.” I first add it with the Library Format = Altium, and then choose Migrate Libraries, which I can see converts the file to the .kicad_sym file format.
I’ve added the Footprint Library to the List of Footprint Libraries in “Manage Footprint Libraries.” As above, I add the .PcbLib using the Altium Library Format, and then click Migrate Libraries, converting it to a .pretty file format.

The converted (migrated) files can be opened successfully in the Symbol Editor and Footprint Editor respectively. However, the database item that points to those files fails to resolve the library files. The symbol is not included, and the Footprint preview shows “Invalid footprint specified” (even though the dropdown list shows “[Default] 144-MAPBGA”.

Do I have something configured incorrectly?
Is the Migrate Libraries step necessary? It seems that, unless I want to be able to edit the files, this shouldn’t be necessary and KiCad can make direct use of the SchLib and PcbLib files. (This would be convenient, as then I wouldn’t have drift between the Altium and KiCad libraries).

{
    "meta": {
        "version": 1.0
    },
    "name": "CCD (Company Component Database)",
    "description": "A KiCad database library connected to CCD Server",
    "source": {
        "type": "odbc",
        "dsn": "",
        "username": "",
        "password": "",
        "timeout_seconds": 2,
        "connection_string": "DRIVER={PostgreSQL ANSI(x64)};Server=<server_name>;PORT=5432;DATABASE=ccd;UserName=ccd;Password=<password>"
    },
    "libraries": [
        {
            "name": "Resistors",
            "table": "Resistors",
            "key": "Part Number",
            "symbols": "Library Ref",
            "footprints": "Footprint Ref",
            "fields": [
                {
                    "column": "Manf P/N",
                    "name": "MPN",
                    "visible_on_add": false,
                    "visible_in_chooser": true,
                    "show_name": true
                },
                {
                    "column": "Value",
                    "name": "Value",
                    "visible_on_add": true,
                    "visible_in_chooser": true
                },
                {
                    "column": "Tolerance",
                    "name": "Tolerance",
                    "visible_on_add": false,
                    "visible_in_chooser": false
                },
                {
                    "column": "Wattage",
                    "name": "Power Rating",
                    "visible_on_add": true,
                    "visible_in_chooser": true,
                    "show_name": true
                },
                {
                    "column": "ComponentLink1URL",
                    "name": "Datasheet",
                    "visible_on_add": false,
                    "visible_in_chooser": false
                },
                {
                    "column": "ComponentLink2URL",
                    "name": "Shiny Part",
                    "visible_on_add": false,
                    "visible_in_chooser": false
                }
            ],
            "properties": {
                "description": "Description"
            }
        },
        {
            "name": "Capacitors",
            "table": "Capacitors",
            "key": "Part Number",
            "symbols": "Library Ref",
            "footprints": "Footprint Ref",
            "fields": [
                {
                    "column": "Manf P/N",
                    "name": "MPN",
                    "visible_on_add": false,
                    "visible_in_chooser": true
                },
                {
                    "column": "Value",
                    "name": "Value",
                    "visible_on_add": true,
                    "visible_in_chooser": true
                },
                {
                    "column": "Voltage",
                    "name": "Voltage Rating",
                    "visible_on_add": true,
                    "visible_in_chooser": true
                },
                {
                    "column": "ComponentLink1URL",
                    "name": "Datasheet",
                    "visible_on_add": false,
                    "visible_in_chooser": false
                },
                {
                    "column": "ComponentLink2URL",
                    "name": "Shiny Part",
                    "visible_on_add": false,
                    "visible_in_chooser": false
                }
            ],
            "properties": {
                "description": "Description"
            }
        },
        {
            "name": "ICs",
            "table": "ICs",
            "key": "Part Number",
            "symbols": "Library Ref",
            "footprints": "Footprint Ref",
            "fields": [
                {
                    "column": "Manf P/N",
                    "name": "MPN",
                    "visible_on_add": false,
                    "visible_in_chooser": true,
                    "show_name": true
                },
                {
                    "column": "Value",
                    "name": "Value",
                    "visible_on_add": true,
                    "visible_in_chooser": true
                },
                {
                    "column": "Tolerance",
                    "name": "Tolerance",
                    "visible_on_add": false,
                    "visible_in_chooser": false
                },
                {
                    "column": "Wattage",
                    "name": "Power Rating",
                    "visible_on_add": true,
                    "visible_in_chooser": true,
                    "show_name": true
                },
                {
                    "column": "ComponentLink1URL",
                    "name": "Datasheet",
                    "visible_on_add": false,
                    "visible_in_chooser": false
                },
                {
                    "column": "ComponentLink2URL",
                    "name": "Shiny Part",
                    "visible_on_add": false,
                    "visible_in_chooser": false
                }
            ],
            "properties": {
                "description": "Description"
            }
        }
    ]
}

craftyjon · June 10, 2025, 4:04am

While KiCad can directly use Altium library files, you can’t directly use the Altium database columns for symbol and footprint ref. Altium uses two fields each to specify the footprint and symbol, but KiCad uses one each. A KiCad library ID (reference) is specified as nickname:entry where nickname is the name of a symbol or footprint library in the KiCad library tables.

So, if you want to use Altium lib files directly, you’ll need to add them to your KiCad library tables, and then add a column to the database that provides the full library identifier (and update your kicad_dbl file to use that column)

You don’t need to migrate the libraries unless you want to edit them in KiCad.

aberger · June 10, 2025, 8:36pm

Thanks. I got that working with the Library_Nickname:Symbol_Name formatting that you suggested. A couple questions/suggestions:

It appears that spaces in symbol names (and footprint names) are not allowed. Is that correct?
It is currently difficult to debug issues with components fetched from a database. The Symbol and Footprint do not appear as available columns in the Symbol Chooser table. (The Footprint does appear as a listed parameter down below once a component is highlighted from the Symbol Chooser table). Alternatively, there is no way to view the database as a KiCad sees it (e.g. Altium offers the ability to open their .SVNDBLib file, and then use “Table Browser” to directly view the database as a table. This makes it easy to see the database parameters for each component).

It’s actually a little awkward that the database needs to know about the Library Path Nicknames, since the Nicknames exist only within the context of the local instance of KiCad. Imagine a scenario where a new component is added to the database, and hasn’t yet had its symbol and footprint(s) defined yet. The database tools may could assign these by selecting from the available symbols found within a .kicad_sym file, or from the available .kicad_mods from within a .pretty directory (it’s possible for scripting and programming tools to peer inside these files/directories and parse the text information from them). However, it’s not immediately obvious how to assign the Nickname: part of the symbol or footprint field.

Do you have any suggestions for how to handle database parameter assignment as new parts are added to the database?

craftyjon · June 10, 2025, 8:45pm

Correct. If you create these on the KiCad side they will be converted to underscores.

There are many free tools for connecting to a database and viewing the data. We elected to not build such a tool into KiCad and instead focus on features that are more specific to EDA. If you say what database server you’re using and what platform you’re on, you might get more specific recommendations.

This is intentional, because the nickname is the way in KiCad to be certain you are referring to one and only one specific library. Filesystem paths can be ambiguous (because relative paths are supported as well as environment variables). But, there can’t be two libraries with the same nickname. KiCad has both global and project-local library tables. Project library tables can be shared along with the other project design files between multiple KiCad instances, and global library settings can be synchronized in other ways if desired.

If you are building some tool to automatically add database entries, you should provide it as one input the library tables you use, which contain the mapping between nicknames and library files on disk. I will note that I just manually paste in the symbol and footprint IDs from the KiCad symbol and footprint editors into my database tool, and I am not bothered by this workflow.

I am not sure what you mean by this, can you say more?

aberger · June 10, 2025, 10:59pm

Can you say more about this? This would also help clarify your later point about providing to the database tool “the library tables you use, which contain the mapping between nicknames and library files on disk.”

We have multiple users who need access to the parts database (as well as some shared collection of the libraries used by those parts, whether on a network drive or version controlled with GitHub, etc). When a new part is added to the database, it comes in with undefined symbol and footprint fields. I’m not sure that it makes sense to give all users full write access to the database. Instead, they should be able to assign the symbol and footprint for the new part by selecting from either the available libraries in the shared library repo, or contributing a new symbol and/or footprint library to the repo and then pointing to that.

Somewhat unrelated:
Why can the symbols not be sorted by any of the Columns in the Symbol Chooser? (For example, sort by Value). All I see is “Sort Alphabetically”, which is ambiguous… using which column?

craftyjon · June 11, 2025, 2:53am

If you are already writing custom tools, you could write a tool that merges a set of pre-defined libraries into the standard KiCad global library table files. Or you could use a standard template library table that you use to set up new machines instead of the default KiCad one.

I still don’t really follow this. It’s not a workflow that makes sense to me. Normally, someone who has the ability to add a new symbol and footprint should also have the ability to add content to the database. Sometimes in organizations these people are “librarians” with special privileges; sometimes it’s any engineer.

KiCad doesn’t currently support sorting by arbitrary columns in the chooser. There is a somewhat related feature request open here: Filter option for specific columns in the symbol chooser (#17938) · Issues · KiCad / KiCad Source Code / kicad · GitLab but this covers filtering, not sorting. You could open a feature request for it.

aberger · June 11, 2025, 4:31pm

I can’t find how the library table files are stored, or how they could be exported/imported to new machines.

Regarding access to the database: it’s really just a safeguard. Full write access to the entire database could be potentially quite destructive. The ability to simply assign or change Symbol and Footprint IDs to existing items in the table is much less invasive. This also simplifies for the engineers: they don’t nee d the database tools, just the ECAD tool.

OK, noted regarding the feature request. Thanks.

Claudio.Lorini · June 11, 2025, 4:43pm

if you are under Linux they are in /home/yourhomename/.config/kicad/9.0/sym-lib-table
under windows, search for the same filename, i don’t remember where are placed.
The file can be copied to other machines.

craftyjon · June 11, 2025, 4:45pm

KiCad is intentionally not a database editor, just a consumer of data from the database. It sounds like you want a custom database editing tool that gives people privileges to only update certain columns.