Ughh, the current libraries are a mess

[joost]

@hermit: I hate doing my own models as well as doing my own footprints. Somehow I make one mistake after the next with all that detail. But, I will do it because the alternative is far more frustrating. The start was/is difficult when my library was empty and just about every component would need to be done. Now its just one here and there.

Another side effect of doing your own, and not liking that, is that you more often reuse existing parts. I question myself a lot if I need yet another device/resistor/cap that is slightly different from what I already proved. It also means that I have less inventory which is easier to handle and can get (slightly) better prices on larger quantities. Additionally, reusing parts and schematic parts across many designs makes one very familiar with what is normal for the part i.e., troubleshooting and design limits go much faster.

I am totally with you on the “helping out with open source” part. I simply question the mountain of work that seems needed to happen to move the KiCAD libraries along. The discussion how bad the libraries are comes up again and again (they are not that bad imho). I have also seen the various discussions on naming libraries and - another question use of anyones time to me. Having a good search function is far more to be preferred over a library naming system that few will completely remember. Lots of brain power is wasted on those kinds of discussions. I rather do work on the essentials and forget the rest - it is noise.

But I hear ya, helping out is a good thing since I too am plucking the fruits of your (collective) labor and I will think about where I can best help and put in some spit and polish.

[dchisholm]

I’m with @Rene_Poschl: Make the libraries as good as you can justify. The fact that users will use them as a starting point for their own symbols or footprints is a compliment, not an insult. Even when I was using expensive layout programs (before I started with KiCAD), i had developed the mind-set that anywhere from 20% to 80% of the effort for ANY layout project would be “library work”.

And keep in mind that the idea of “libraries are a mess” is really TWO separate problems. One mess is content, such as inaccurate pad placement, silkscreen on pads, symbols with pin designations that don’t agree with footprints, etc. The second mess is structure and organization of the library subsystem, difficulty of locating symbols and footprints, etc. Both problems seem to be receiving active, and effective, development effort. Give the librarians and developers credit for fighting a war on two fronts.

Dale

[1.21Gigawatts]

@mikecrash

Until the library will be sorted like this, it is useless in production environment. Current state is only for students and hobbyists.

This is simply nonsense!

@mikecrash

Also what is needed is part numbers to create order list. There is no such thing in KiCAD.

But you maintain your own libraries so surely you’ve added this field.

[1.21Gigawatts]

@Andy_P

Yeah, enough with the 4000-series CMOS and all sorts of other crap you’d never use in a design.

If there is a part missing from the libraries that you think should be included then create it and submit it. Propagating some misguided notion that the reason it is missing is because the 4000-series CMOS parts are included is just idiotic.

[1.21Gigawatts]

@davidsrsb

… a quick check shows that RS alone lists 125 4000 series parts. Somebody is buying them,

More importantly, they are still listed as ‘active’ or ‘in production’ with several manufacturers, far from obsolete.

[joost]

@dchisholm Your two points were also mine - i just come to a different conclusion. A. Any component i accept in a design i will check and modify if necessry, B. The library structure must fit my way of thinking.

That simply means that i take the parts i am using, test and modify these if i can or create new parts in a library structure per my own design. I dont see how anyone with commercial needs can accept untested parts. Therefore, i do not see much need for a highly structured library (some obvious is needed but no intricate layout) and only those checks an balances to throw out obvious badly designed parts. I would rather focus on a good search function and a fantastic part creator (symbol, footprint and 3d).

I think the communicty effort to create a standard library is noble but not effective - there are too many parts, too many ways to make mistakes, too many folks that need to be trained to do it right. I just want to be realistic about this.

Just my 2ct.

Btw i had my first go at pcb design on the commodore amiga. The software came with a small parts library and i found out how awful it was during first article production. Learned my lesson then and there.

[NMNC]

Coming from a predominantly Altium background(with some Eagle mixed in sadly) I am a fan of using a reduced number of files to manage the libraries and rely upon the search tool.

In my workflow, I have used an Electronic and Mechanical file for my parts library. This means that all atomic parts are kept in two places based on their function. I will also use manufacturer supplied libraries, but if it is used, and then checked I move that part(with any changes needed) into my official library. I’d like to see all non generic parts named by their manufacturer name, with a sensible default footprint where linked to the manufacturers device name.

Kicad could do a similar thing, and use a minimally structured system like this for the schematic libraries:

Kicad - Electronic - Official
Kicad - Mechanical - Official
Kicad - Electronic - Pending
Kicad - Mechanical - Pending

Any official libraries supplied by manufacturers(ie TI, AD, NXP etc) or suppliers(Digi, Octopart, Ultralibrarian, etc) could live in their own libraries. This would also go for peoples personal libraries that they will develop over time.

But what this means is that Kicad could have the stable parts that have been verified by use and by vouching by the librarians in the Official version, and then the ones for review sitting in the Pending. This would allow more parts to be put in to the Pending, and help get more people using parts so that they can be vetted and tested. Once things are in the Official file they will be part of a growing library that the community is building in the back.

I’d also posit that some sort of voting/upsorting be applied by key members for parts would be useful and decrease the load on the librarians but I don’t know how that would work. I’m sure there are many people on this forum who wouldn’t mind sharing a few of their parts that they have designed for their own uses either.

[ouabacheDesignworks]

95% of the internet is crap and that has remained true since it’s earliest days. 95% of Kicad libraries are crap or at least are parts that you will probably never use. The trick is to figure out how to give access to everything but only show each user the parts that they really use.

We need to develop search tools that can screen through everything and only show the user what they want to see.

We need to manage our IP the same way that the National Bureau of Standards manages their standards. They keep the golden standard under a bell jar somewhere and then use transfer standards to pass it on to every supermarket scale and gas pump in the country. We call them “mirrors” and will mirror a rcs repository onto our servers to give us access to far away IP. Mirrors need to be maintained and updated whenever the master changes.

So set up a website to collect the URLs for all the Kicad libraries that are out there. Have a search tool to customize your view into this collection. Mirror the useful parts onto your workstation. If you are on a design team then you mirror into a company server and the entire team will mirror from there.

Kicad should not look at the libs on your machine but rather at the index for every lib on the internet. If you select a part then it will download the repo at that time.

Don’t try to solve this with incremental improvements, You need a solution that gives you a quantum leap in performance

John Eaton

[bobc]

@ouabacheDesignworks

We need to manage our IP the same way that the National Bureau of Standards manages their standards. They keep the golden standard under a bell jar somewhere and then use transfer standards to pass it on to every supermarket scale and gas pump in the country

Um, technically that might be the ideal - but NIST have a budget of $964 million dollars (2016) !!

Budget available to KiCad: $0.

I’m afraid to say these library discussions will be endless, but the underlying issue is that everyone wants good quality data, but very few are willing to spend money (or time creating it), unless for their own need. 1% rule. There are several companies that will take your money if people really want a part data service. Otherwise… “beggars can’t be choosers”.

I think probably the library maintainers are on the right track, focusing on a relatively small number of high quality components to a consistent standard is the most effective use of their limited time. The reality is that KiCad will never have a vast library of good quality parts which meet everyone’s requirements. Unfortunately we will always have to put up with “observations” (aka complaints) that “KiCad doesn’t have the parts I expect from a professional software package even though I know it’s created by volunteers and I didn’t pay for it or contribute my time supporting it (and never will)”.

[joost]

@ouabacheDesignworks and @NMNC: yes Yes and YES.

Librarians; this is not a slam on you or the efforts that went into the libraries thus far - it is a request to (re)focus.

[davidsrsb]

@ouabacheDesignworks

95% of Kicad libraries are crap

Err no, in my two years using KiCad I have seen the library quality improve in leaps. Two years ago it was common to find bad parts and models. Now most of the gross errors are gone and IPC footprint compliance is the norm. Built in 3d models are now at the point that I don’t have to go to 3rd party libraries first.
There still are plenty of symbols not KLC compliant in the less popular libraries, but many of these are being fixed. The struggle is that different users want different “standards” - I personally like to see multi unit 74xx with separate power unit, others don’t

[Joan_Sparky]

@ouabacheDesignworks

Kicad should not look at the libs on your machine but rather at the index for every lib on the internet. If you select a part then it will download the repo at that time.

Don’t try to solve this with incremental improvements, You need a solution that gives you a quantum leap in performance

All dandy, but IMHO this doesn’t solve the core problem - how do you feed back high quality parts into this system, which are vetted and adhere to a standard? And which standard is this?

If a search engine helps you navigate a sea of crap, at the end of the day you will still wind up with lot’s of crap in your net. (*)

Who is going to do the vetting?
Who is going to be the authority?
How do you keep those persons sane?
How do you organize their workload?

And then comes the part that concerns KiCAD - how does one hook this all up with KiCADs code?
And that’s were the librarians (for a while now) are trying to push the envelope and try to get rid of some older infrastructure. Infrastructure that is in the way of everybody. And the developers doing the same. EEschema and the symbol library system underneath currently is undergoing transformation… without that there is no sane library concept at all.
Anyone who has dabbled into this territory knows this.

So before anything or anyone can do ‘big leaps’ here, KiCAD needs to go through some ‘babysteps’ to make that leap possible.

*) this is a problem no one has solved afaik. For anything. As usually the buck ends with a person, and the quality depends on a person at the end of the day. The more persons you have to organize, the worse your outcome will be, unless you enforce strict quality standards by a quality control system.
Now that is something the KLC + automated scripts are trying to do.

But without people that are WILLING to come back and correct their work to adhere to that standard, there will be no work approved and nothing to share that is worth sharing.
And that’s what the problem is - motivation.
The buck then ends with the person doing the vetting and it’s ability to do the work needed itself and it’s motivation.

The new 3D models are a very good example of this. @Shack did most of it and hold HIMSELF up to a very high standard - voila, you get lots of models that are reliable.
Try the same with crowd sourced 3D models and you wind up with a mixed bag.
And now try to get all the contributions up to a similar level of quality, as this is what you actually want.
So again, the problem we face here is how do you implement a quality control system that motivates contributors to GO BACK and refine their work for OTHERS, while for themselves it already is good enough.

The official KiCAD libs do the only sane thing here - being a beacon of vetted parts that try to be on a common level of quality, vetted to the KLC with as much work the librarians and contributors are willing to do for free.

[Joan_Sparky]

@bobc

I’m afraid to say these library discussions will be endless, but the underlying issue is that everyone wants good quality data, but very few are willing to spend money (or time creating it), unless for their own need. 1% rule. There are several companies that will take your money if people really want a part data service. Otherwise… “beggars can’t be choosers”.

It’s even worse than that. The shared data you get by crowdsourcing is always good enough for the person who created it, but mostly not good enough for a common standard.

How do you motivate all those persons (that to their own standard already are finished) to go the extra mile and revisit and resubmit their work?

I think that’s what the 1% rule is about - not many persons are willing (and have that state of mind) to do work for somebody else with pretty much nothing in return. Might even be some sort of masochistic disorder

[NMNC]

If the bottleneck is people checking parts, could a voting system be used within the application? Ie have the rules for part development next to the part that someone needs to click to say Y/N for the part before submitting to a next level of inspection? This would be like a bug/feedback form with prefilled out rules that need to be completed before uploading the part.

Is there a possible script or addition to Kicad that could allow people to check parts and submit from within Kicad? Removing barriers to entry for inspection and submission of parts will see more people participate. If the challenge is how to best use the resources available for the librarians, then we should discuss what options there are to improve their workload and share it more with the community.

[Joan_Sparky]

How do you motivate people to ‘play’ quality control check, that adhere to the KLC?
How do you get the contributors to come back and apply the fixes needed to make contributions KLC compliant (or anyone else)?

Lowering the bar, by implementing human QC interfaces into KiCAD is a path I guess, but I’m pretty certain not worth it - as being a QC system is pretty boring work actually. Humans don’t like boring stuff (or is that just me?).

[NMNC]

People are motivated by their intrinsic motivation to help - like most open source projects people will come and go with their enthusiasm and time constraints, but if each person using Kicad does at least 1 part of prechecking before the librarians have to look, then that may be a lot less person-hours needed for the librarians.

The QC done by the librarians is greatly appreciated and if there is a way to reduce their workload by outsourcing some tasks to the greater community, then it may be worth investigating.

Kicad is an open source project that has core developers and librarians who do amazing work. It doesn’t have the paid resources of the Orcads/Altiums/etc, but it does have openness and extensibility. Another important resource available is in the form of users. Taking advantage of their willingness to help can be done by lowering the bar to assisting.

Making it easier for users to upload their part candidates to the librarians would be a great example of how to leverage volunteers en masse in a project like this.

[Joan_Sparky]

Librarians are QC checkers (and maintainers).

After you get more people to do the checking, how do you get more people to apply fixes to get the quality to KLC level?
As we have heard:

@Rene_Poschl

If you look at the open pull requests you will notice that most of the open requests are awaiting response from the contributor.

I’m not saying this is impossible - far from it - I’m just trying to point out what I think you need to overcome to make this a successful exercise.
Just saying ‘build it and they will come’ doesn’t work IMHO, unless you can explain why it will work.

[NMNC]

That is a tough problem. If people are submitting a part, and then it is kicked back down for a fix, but the part works well enough for the person to use it but it doesn’t meet the standards it lives in limbo. It sounds like a “good enough” problem where there is little incentive to revisit and push it past the 90% level. One way to fix that is to simply drop the QC - but I don’t think that is a valid solution.

I think a more scalable solution without sacrificing quality is to open up the part to everyone to fix and submit in a method more in line with the workflow of the software. I think this is where a “Pending” library could be useful when coupled with in-application editing, rule checking and submitting of schematics or footprints.

I understand there will be issues with quality anyways with a pending library, but it would provide useful information about which parts are actually being used and need a greater focus of attention. Checking the origin of parts will also be important to stop people poisoning the system with bad parts, but ultimately the librarians will be responsible for the final word on part inclusion. (EDIT: Perhaps this can be fixed with part wizards that more rigidly enforce Kicad standards as a final step?)

Moving forward into the future, where people will be moving from software like Eagle (due to licencing, ownership, costs, quality etc), the parts library of Kicad will be a huge strength as the leading EDA in the free/open source world. I think that being able to use a scalable, community oriented parts development, maintenance and adminstration system will only add to the existing strengths of this system.

[Joan_Sparky]

@NMNC

I think a more scalable solution without sacrificing quality is to open up the part to everyone to fix and submit in a method more in line with the workflow of the software. I think this is where a “Pending” library could be useful when coupled with in-application editing, rule checking and submitting of schematics or footprints.

I thought along the same lines. Personally I have a fp library called ‘test’ where I put all fps in I download and need to check or work on that I don’t need myself… it’s a simple file operation underneath were one just copies files into that test folder.

For the symbols this isn’t possible yet, as the underlying structure doesn’t work like that. The nightly versions already have a similar sym-lib-table like there is for fps, but the backend isn’t ready yet. That’s what needs to happen before anything of this has a chance (v5 was supposed to be it, but it’s be going to be v6 afaik).

In the end it would be nice, if there was some KiCAD interface on the main launcher, that would point out to anyone interested that there are symbols/footprints to review and upon clicking that the stuff get’s downloaded to your machine and you can have a look and fix/comment on it…
Would need some sort of central software akin to github, so you can get a unique ID and rise in rank for quality reviews.

Needless to say, the review interface needs a direct hook to the last KLC version.

[John_Pateman]

I think that a lot of people would be very happy to submit their libraries/footprints and models but there is a significant barrier in the form of Git. The git workflow is quite involved and a bit alianating if you don’t have a software / programming background. Even though I use a versioning system (fossil) for my repos I find Git a tricky beast especially keeping my cloned fork synchronised upstream. Submitting a complete component involves forking, pushes to three separate repos and then making pull requests on the main repos. Looking through the GitHub issues the ‘something went wrong’ problems are quite common. An easier mechanism to post components to might encourage more submissions. (And might mean the librarians have more time to look at the components rather than having to explain the vagaries of Git to the bemused).

I think the librarians are doing a fantastic job and really value the fact that they check the components carefully against an agreed standard. I still check myself but the quality is excellent and the quality models an extra bonus. I would rather continue to have components of this quality and make the odd extra that I need (and then be able to simply submit it for review).