Creating comprehensive component libraries from supplier data

Schematic Library Symbols
1

Continuing the discussion from Default manufacturer’s part number field in KiCad libraries:[quote=“Joan_Sparky, post:14, topic:4387”]

Many parts in the libs would multiply by 10-100 or many more parts… that’s just insane.
[/quote]

Let’s find out… I decided to scrape data form Digikey, for this exercise I picked through hole resistors. Digikey list 284,958 parts in that category. We can cut it down a bit by selecting bulk packaging and Active parts only, that leaves 134,833.

So I created a library with those parts, and loaded into KiCad. KiCad becomes unusable with that many parts, the library browser gets stuck in an update loop, and I have to kill it. Ok, so 130,000 is too many, how about splitting by manufacturer? Well, Vishay have 100,000 parts, so that is still too many.

However, most of the Digikey parts are non-stock, i.e. ordered on request. I filtered those out, and am left with 4734, which is much more usable.

A second part of the exercise is to see how much can be generated. The part data is fairly easy, Digikey data is quite good. Working out the footprint takes certain amount of guesswork, but the majority of resistors were matched to one of the existing Resistors_THT footprints. The actual footprint used is a matter of choice of course. There are a few resistors that don’t match any existing footprint.

An example part looks like:

2097×1457 78.8 KB

I don’t know if nightly builds have addressed any scalability issues, but dumping tens of thousands of components into KiCad is probably not going to be practical. Digikey list over 400,000 SMD resistors, over 116,000 if only stock items are counted.

So I think even if a complete inventory of supplier parts is created, it would need some pre-filtering before being used in KiCad.

DigiKey, through hole resistors, Status=Active, Stocked

Digikey_res_tht.dcm (355.4 KB)
Digikey_res_tht.lib (2.9 MB)

2 Likes
2

Good experiment.
One possible way of cutting down the number of components would be to have the components defined on a “family” level (resistors, capacitors, inductors).
By that i mean Value + Tolerance + Power Rating + Footprint combination = component.
The component would then cover multiple manufacturers. It’s not truly atomic but maybe would be manageable from a library perspective.

/s

3

Even if you were joking. I think your idea might be a good one in the short/middle term.

Currently one could give each of these families a separate house part number that is used as a key in a database.
An external database then connects them to the ordernumber of the chosen distributor.

There might be more than one order number per part depending on the needs for the project. Example: fastest delivery, cheapest option, low unit count, … (These would need to interface the distributor databases to get the daily prices and delivery times.)

The only thing i’m certain about is that it is probably a better idea to have the symbol used in the schematic abstracted away from the real part.
Why?
What if i want to build that one pcb years later? The order numbers might have changed. (or the resistor i chose back when i first made the board is out of stock, …)
Maybe i don’t want to build thousands of units during development but later on i want to make lots of products cheaply. …

I also highly dislike the idea that one program should fulfill all possible tasks of a project.
I really like the early open source (linux/unix) doctrine of having multiple specialized programs with a clear interface to each other.
In short: one program per task (In this case: one program for designing the schematic+pcb, one for creating the mcad drawings, one for ordering stuff, …)

1 Like
4

@Rene_Poschl: Not sure why you were thinking i was joking :slight_smile:
I’m more in your camp with this, i also abstract parts from symbols.
It just looks like there will be problems for people that do want to go the atomic way (as @bobc points out) that creating 100s thousands of atomic parts is going to cause issues with KiCad at the moment.

/s

5

Doesn’t /s normally stand for: end of sarcasm? (At least on reddit it does.)

6

Didn’t know that, i guess i’m showing my age :grinning:
.
/steve

7

The duplication of effort is a problem. I use the Malaysian branches of RS and Element14, so locally unique part numbers and these frequently change as the SE Asian structures get shuffled.
Bringing in supplier part numbers causes extra work for multinationals when they move production between countries

8

Then you adjust your house-partnumber-to-manufacturer-partnumber-conversion-list that is used by your script/tool/etc. which works on the BOM and call it a a day?!

I already stated that if I rework my personal libs again (4th time?) - and I see it coming already - I’ll go with the house-number approach that @Andy_P uses, I just didn’t think of it on the 3rd time I redid my libs (had started with KiCAD 2 months before) :stuck_out_tongue:

The ridiculousness of the approach itself only applies (IMHO) if one would try to apply atomic parts (house-numbers or not) to the public KiCAD library repo(s).
That’s were the trouble starts, as the sheer numbers of parts that would be needed to be kept up to date to fulfill everys nerds dream is insane :wink:

The more fruitful approach would be if tools like KiField, house-partnumber-scripts, some nice GUI with some database backend would be developed/thought about, which could be used by everyone for his parts (that then could pull in/connect parts from the KiCAD libs).
That standardization approach some time back for the partnumber field naming convention was a start to that, but I think it got lost in details.

Without wanting to derail the thread - tell me if I should just repost this in a separate thread (probably already developers working on something like this privately) - the topic should be ‘what does this program for ordering stuff need to look like for hobbyists?’.

PS: cool work @bobc

1 Like
9

I’ll have a look at doing that. A disadvantage with family or grouping into house part is the question, what exactly is an equivalent part? Some properties like temp coefficient might be generally unimportant but important for some users. It might need some intelligent decisions which a dumb script could not do. There are also some exasperating differences in notation, e.g. 1, 1.0, 1e-3 k, 1R 1R0, 1 ohm, 1 Ω are all the same value. Simply matching values is not trivial.

There are also issues of different order codes for loose, reel, part-reel, tray etc. The advantage of the strict atomic approach is that it’s completely WYSIWG, the part you pick is the one that is ordered.

I also tried using an external tool to match schematic symbols, which was ok until I got into alternatives. I use the symbol e.g. ®, footprint, value to query Octopart via their API. That returns some nice results, but of course relies on Octopart.

https://cdn.hackaday.io/images/1075251455574916535.png[quote=“Joan_Sparky, post:8, topic:6734”]
That standardization approach some time back for the partnumber field naming convention was a start to that, but I think it got lost in details.
[/quote]

Wayne had a good idea on that, the user could specify some default fields that are automatically added to the component fields. He is quite against the idea of KiCad creating an “official standard” for field names, or embedding that in any way into KiCad (as I expected).

Good question, but it seems that there will be a different answer for every person :slight_smile:

1 Like