How to integrate a database library?

Forgot to link the library: https://github.com/issus/altium-library :slight_smile:

2 Likes

Moderator comment:
I cannot see any hidden or withdrawn messages in this thread

The initial post from @issus disappeared for a while with a moderator flag - not sure why, anyhow it is now back OK.

Sounds good, so it’s just going to be a matter of decoding the alitum pcblib and schlib files, then I can generate the footprints for KiCad. I’m thinking I would setup a trigger on an INSERT for the parts table, to flag that new components have been added - which after an hour or so cooldown (to make sure i’m doing adding stuff) would get picked up and cause the new library parts to be generated and pushed to github automatically. This way, the KiCad side of things would be fully automated.

There’s already code in place to keep the webserver’s repo synced to github (since I use a basic decoding of the altium files to give me dropdown selections of which symbols and footprints to map). So the web server has access to the library files for that side of things.

It sounds as though generating the KiCad parts wont be too hard, but I’d need to figure out how to manage them - both qty wise, and naming convention wise. Does KiCad have any sort of filtering capability in it for searching for parts?

1 Like

more mod comment:
I had to allow the one-line post from @issus with the forgotten link, as the system had flagged it.
But didn’t check if the system also had the whole thread unlisted. I could see it on a main page, but no idea what it’s status was then (~12 hours ago).

This forum is the official unofficial KiCad forum for users (yes, it makes sense).

@Rene_Poschl does quite a bit of official KiCad library work.

@issus it appears that a user/system flagged your OP for some reason (glad it wasn’t me). You should get much quicker responses to your queries now that your thread seems to be back to normal.

An open source database library for KiCad does not have to be 100% KiCad KLC compliant. KiCad is still growing and changing at an even faster pace. You may even have ideas to help give direction as it continues to improve. The official place to share your ideas is the Developers Mailing List; I’m certain you can find the link on your own.

Thanks for joining the forum!

I do not official work here on the forum. (Other maintainers need to have the option to give input. This is only the case if the issue is handled over on github.)

Naming convention is always hard to decide (i know because i wrote huge parts of the official library convention)
You could use the KLC or industry standards as a starting point (To avoid starting from scratch and making the same mistakes as others did before you)

One thing you will need to decide early on is if you want the namings to be easily human readable or if you want to have short names (KLC goes towards human readable, IPC towards short names)

Only on the name and tag fields and only in text form. (Text form means it can use wildcards similar to how the linux filename substitution works. * for any number of chars, ? for exactly one char.)
So you will need to come up with a well documented naming convention for users to be able to easily search for parts.

And even more comments regarding mod stuff:

@ChrisGammell took care of unhiding the initial post. He answered to the message that i sent to the moderators group with details. I would have assumed that you guys also get such a message. I think in the future i will manually create a message with all moderators as the group messaging seems to be kind of screwed up.

Hmm, I suspect that perhaps KiCad might not be ready for such a library then (and I say this in the nicest way possible!). The huge advantage in Altium with the dblib (other than placing real parts tied to a supplier) is the ability to filter by parametric values. Like my capacitor example - go to the Capacitors MLCC view, filter by 0603, filter by 0.1uf, filter by 16v capacitors, sort by price and choose the cheapest.

Without filtering, and having over 5000 resistors in the library alone (01005 to 1206, with every value in 1%, 0.5%, 0.25%, 0.1% and 0.05% from several suppliers) it would make it very very difficult to find the part you need.

So, what can we do to get this functionality into KiCad?

  • Does KiCad support plugins, so we could develop a KiCad plugin for the database library?
  • If built in functionality was developed by the community (assuming someone is willing to help) would this be integerated into the main release, despite not being on the roadmap?
  • Would there be enough interest in the community in the library to get help with integrating it? I’ve had a lot of messages from people saying they would love to see the functionality in KiCad, but that does equate to help in implementing it.

I’m happy to help out as I can, but my software dev experience is all in .net on Windows, and C on microcontrollers. I don’t feel I have the experience to implement this myself. I feel I should be able to tackle the footprint and symbol porting, just not the KiCad modifications.

I’d be happy to port the library over to MySql (basically have a MySql mirror running on our server along side the main MSSQL instance) - OR - there is a JSON web api for the database, which could be extended and used rather than a direct database connection.

to be honest i am not really convinced that it is a good idea to have one tool for everything. In my mind part management is best kept separate to the circuit design tool. So this would mean you have a program that has your part management that you use to select your component and then either push the information about the selected component into kicad directly or provide a primary key that is then used for interfacing (To get a BOM out of the design tool.)

This will allow you to have the part management tool specialized for your company internal ordering system (in a lot of cases this will mean interfacing with SAP) and any changes to that system do not require any changes to your CAD program.

Maybe this is just a difference in thinking as i would say the design should drive BOM not the other way round.

You’re right in your thinking - and thats how the DBLib works with Altium. There’s no ‘local stock on hand’ capacity in the library - it’s just a huge database of footprints and symbols related to parts. The only reason it would need changes to the CAD package is because it works so differently from an integrated library (to use Altiums terms) which has a symbol associated with a footprint to make a specific part - as this leads to generic components for passives and such.

So in Altium without the DBLib I’d place an 0603 sized resistor and set it’s value to 10k. Then in my BOM i’d just have a line for RES 0603 10k. With the DBLib I go and find a Panasonic 10k 1% 0603 resistor and place that on the board. When my BOM is generated, it has RES 0603 10k 1%, with the manufactuer Panasonic, Part Number pn10k1pc (made up lol), Digikey as the supplier, the DK SKU being P10K-1-ND (again, made up) and a link to the part for ordering.

This doesn’t sound like a huge advantage until you try to place an 0402 Cap that’s 47uF 50V… and find it doesnt exist. So when placing parts I can just rely on that part to exist, not have to stop my ECAD work and go over to a supplier website to look up a potential part, then place it, edit it to set the part number or value or whatever. Furthermore, when you start looking at say a MOSFET which has a generic footprint, you have to go back through during BOM editing and figure out which one is going to be best - OR - do that at the time of placement, again taking you from ECAD over to your browser and you need to edit the value of the part so it relates to the part you just chose from the supplier website.

The DBLib significantly cuts down on mistakes and significantly speeds up design time, for me at least. It means I know I’m building the board I’ve just spent weeks on as it was designed in a schematic, and if I’m not sure of a part when I am assembling the board I can click on the symbol in the PDF of the schematic and it takes me straight to the datasheet for that part. So debugging/diagnostics are faster too. I like that it lets me get on with the busines of designing a schematic or board, rather than faffing around figuring out parts or editing a BOM just so I can order it.

Setting up a BOM for ordering/manufacture went from being a whole days work on a 200-300line BOM to being a 15 minute job (because Altium sucks at accessing remote databases efficiently haha) which doesn’t require any interaction from me.

Attached is a PDF print of an Excel BOM out of Altium for an open source board I made. No editing, no modification required. Without the DBLib, the description column would just be the Value field of the component (ie: 0.1uf) and the manufacturer/pn/supplier/sku/price columns would all be missing and need to be filled out manually.

A very large company might be using SAP, but your average student, hobbyist, small and medium business will not and has no ERP system. So for me, this is a huge time saving on the design side of things because I can place real parts into my schematic, and I can order my BOM really quickly (ie: copy and paste from the spreadsheet into digikey and click order). I would like to eventually build in the functionality to filter to just parts you have in your parts bin/use as standard (so you don’t end up with 6 different reels of 10k resistors haha), but I’m not there yet.

Feeder Board.pdf (105.4 KB)

1 Like

This will work the same even if the tool is external. there the workflow would then be to look in your external tool what you want to place, generate the symbol/footprint pairing (a good tool will try to use as many generic assets here as possible to cut down on library maintainance)
and insert that generated pairing into the schematic. Yes there is one more step but it will mean that this whole workflow is a lot more flexible when compared to integrating everything into one single tool.

Well, I guess if the interest doesn’t exist I’ll just leave it there then. I’m not a KiCad user myself, so there isn’t much point in me putting a ton of work into it if the solution isn’t going to be easy/smooth for the end user’s experience.

Just a heads up: I am not a dev. Ask over on the mailing list or the bugtracker if you need new kicad features to implement your thing. I can only tell you my opinion that might be biased as i am the main library maintainer.

Hi @issus, having a db in kicad would be nice…
you may find this thread useful:

As someone who has used KiCad to design a few PCBs, the hardest part is finding the right component footprints. I feel lucky when the footprint comes with a decent 3D model. If KiCad had a good integration with the issus (Celestial) database, it would boost my productivity substantially.

It appears that Digi-Key is a big sponsor of KiCad already. Perhaps they would donate more money to prioritize this kind of database integration.

Footprint management is hard and these changes shouldn’t be taken lightly. At a minimum, the Library Browser will need a new filtering system and will need to start making async network requests.

@issus, can you post some screenshots of the search-filter-select workflow in Altium?

2 Likes

I forget who, but a knowledgeable Forum Member made the suggestion that Footprint information on a DataSheet is just a “Serving Suggestion”.

Digging a little bit deeper into KiCad, it is not that difficult to create your own Atomic parts with your own custom Footprints.

Making custom footprints is rather time-consuming. And not many circuit designers are skilled 3D modelers. I think most KiCad users will benefit from improved component search.

KiCad’s built-in component library has grown a lot lately, and now has 10,189 footprints and 11,300 3D models. Maybe it would be sufficient to add search on top of KiCad’s library.

Still, it would be better to have the 29,765 parts in Celestial Library. They come with datasheets and Digi-Key SKUs. An added benefit of the external database architecture is that users on old versions of KiCad can get updated components without manually installing a backport. But because of trust and reliability concerns, it will be important that users can configure their database connections.

I’m no PCB design pro, but the comment about the ease of creating “atomic footprints” makes me think that maybe the “leverage” is in creating a script that will go through the mondo “issus” database, find a part (maybe a short-list) matching descriptors, and create the KiCAD footprint from the data found for the user to manage and use as they would any other. Let the DB be “what it is”, and enable the KiCAD user to get what they want from it with minimum time & motion?

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.