Ughh, the current libraries are a mess

I envision a tag and voting system where tags can be organized in a taxonomy, which could include manufacturers, ipc info, wave/hand/oven, and other searchable variables. And a submission system that can be done from a board in KiCad. After manufacture and assembly, a user could submit the board with tag info from a dialog that asks/adds info on the soldering method and success/QC. A submission script would extract all footprints, symbols, 3D models and add the indicated tags to the used symbols after a simple spreadsheet interface. Counts on the “votes” from these submissions, maintained by the online system, constitute QC. Unchanged use and manufacture by multiple people seems like a decent QC system for use by the wider public.

Thoughts?

1 Like

@Joan_Sparky Thanks for the kind words :slight_smile:

Im kinda confused when people complain about a bad quality of the libraries and meanwhile want to remove the qualitycheck and “make it more open”.
IMO the low quality library parts are leftovers from when the libraries were more open.
personally I always check a part before using it. If its wrong or low quality, I fix it and add it back to the libs. currently Im reworking the Directfet footprints and 3D models because I thought i needed some for a project. Turned out I didnt, oh well hopefully somebody can use it :slight_smile:

8 Likes

I don’t think there is anyone here arguing for a decrease in the quality of the libraries. But there does seem to be some suggestions on the table for how to get more public community generated content into the libraries with faster feedback cycles between librarians and users.

I believe that the libraries will be a real turning point for how open source EDA software is evaluated. Community contributions in the vein of “many eyes make all bugs shallow” should only make Kicad a better system to use. Especially if we can leverage the libraries that everyone is generating in their own private collections - tapping this resource properly will make a huge difference to development speed for new and old users alike.

1 Like

It’s great to see such well mannered discussion :slight_smile: and it is clear that many people value the community contributed libraries.

I will reiterate that the KiCad libraries are only moving towards more strict standards (such as IPC compliance) and hopefully the checker scripts can be further improved to alleviate more work.

It used to be that checking a single submission took at least half an hour. It can be only a few seconds now. Now that most of the grunt work is done via Python and Travis, the majority of time is spent dictating required changes back to users.

I would like to see a system where the Python scripts can suggest fixes for simple issues and actually fix problems without further discussion with the contributor. However in many cases such changes are non trivial and hard to program.

I do think that using git is a barrier to entry. But I have not found a better alternative. Suggestions that we “make a new system” will have more success if they include a reference to a toolset that already exists and will be superior to the current solution.

Bear in mind that there are currently around 4 people who manage the libraries. We don’t get paid, we have jobs and lives and other commitments. Managing PRs is easier now, but there have been countless hours of work to get it to this point.

7 Likes

Would a plugin that made use of something like GitPython work? A user would probably still need a github login, but if it could download an “unstable” part library with the github comments, display the part and allow easy editing and commits this may be less of a barrier than currently exists. Thoughts/suggestions anyone?

1 Like

I do not agree with the boldness of this statement, @hermit, but perhaps I am not disillusioned enough yet

“Unfortunately, AFAIK, no software provides large, usable, good-quality library. You need to built every freaking component from scratch, a fact that often slowly sinks in as every beginner hopes to get libraries but get slowly disillusioned.”

I am using the kicad libraries from git. They are moving. A lot has changed in the past few months. From time to time that is a burden, but it is a consequence of my choice to use the latest stuff.

I guess ones impression of lib quality depends on what it is one is doing and which kind of parts one is using. Yes, there are parts that are not in the libs, but in my experience it is not the majority. I have made some of the more complicated parts that were missing and donated them to the libraries. It is a community effort, after all, and I am part of the community.

1 Like

That is the reason the strict quality control should remain in place.
I like the idea @NMNC introduced for a ‘pending library’, but I fear it will grow very fast to the point that no-one has the guts to finish other people’s mess.
Then some people will start using the ‘good enough’ parts from the pending.lib and in a years time we will be back to square one: A whole bunch of non-complient parts and four poor maintainers that are finishing the work that others couldn’t be bothered about. That must be very frustrating.

7 Likes

+1
The progress could be slower in increasing the library parts, but surely more robust and reliable.
The same has been done for the new 3D parts library, and the result is a fantastic (IMO) ready to go 3D mechanical and rendering library for ECAD that I consider the best available around!

4 Likes

My personal take on all of this is that building and maintaining good libraries is a huge asset to KiCAD but it is also a lot of work. The parts I created were hard work to make, get them to comply and into the libraries.
I have accepted that. It will not become any easier if we want to create parts that anyone can use.

We all appreciate using parts that others have made and this is largely due to the fact that there is some level of standardization in naming, orientation, numbering etc.etc., despite the efforts manufacturers seem to be making to obscure things (for instance by using their company specific designations for housings).

As a consequence, creating a part for your own use is different from creating a part that you want to share. If you are creating something to share, it must conform to guidelines we have learned to appreciate so much when we use someone else’s efforts.
Let’s accept that and get on with it.

I do not want to degrade this conversation in any way, because it is useful to discuss how we want to endorse which guideline. I merely want to point out that, whatever standard or guideline is chosen, it will always be hard work that requires a lot of attention to details.

8 Likes

If you want to talk about the “good old days” (formerly known as “these difficult times”) . . . . take a poll to locate Forum members who laid out boards with tape on mylar. Often at 2:1 scale factor, then photographed and reduced to true size. Libraries? Well, you had stick-on patterns for 8-pin, 14-pin, and 16-pin DIP packages; TO-3, TO-5, TO-220 and TO-92 transistors. And the layout tape was black - using red and blue tapes for the two layers wasn’t yet common.

But nostalgia just ain’t what it used to be!

Dale

Thank God for Bishop Graphics. You did the design with colored pencils and then taped over the lines.

John Eaton

Haha, yes I remember those days. I did the tape and mylar also until I “upgraded” to a plotter with etching ink. What a disaster that was, never got it quite right. I made a simple sound sampler for the Amiga which turned out to be pretty popular during social commodore gatherings (i.e. copy parties). So I sold a bunch of these samplers almost every meeting. Although drilling holes in a PCB has some therapeutic character, doing 40 a week or so was a bit too therapeutic for me so I made a rig allowing me to drill 5 boards at the time. I sniffed up so much PCB dust, leaded smoke and board acid, its a wonder I am still here.

Now to send off gerbers and get boards back in the mail is magic, I enjoy it every time I get them back in.

I was even thinking that a simple Wordpress form or a Google Forms submission page where you could upload a library entry ± footprint ± 3dmodel step/wrl (along with the datasheet and screenshots) and a script to process them might work
 But I accept that that is not a complete ‘toolset that already exists’. And as far as superior to the current system that is also, I guess, depending on who’s perspective. For those comfortable and happy with Git it will not be a significant improvement, but anything to ease the submission mechanism might encourage those who aspire to help to put their shoulder to the collective wheel would, in my mind be a help.

I have been thinking about a tool that removes the need to attach screenshots to each submission. The python scripts could render the symbols or footprints and then perhaps upload to Imgur or similar.

But the following problems would need to be solved:

  1. Create a set of rendering scripts for symbols and footprints
  2. Work out where to store the images (AFAIK you can’t attach an image to a GitHub discussion via API)
  3. Comment on the discussion with a link to the screenshots.

It’d be awesome if the rendering scripts could also highlight any KLC errors too


If anyone has any knowledge on how to achieve these points, fire away :slight_smile:

1 Like

KiCad can render to SVG, and SVG can be displayed in a browser.

imgur, however, cannot host SVG files (I just tried it).

There are a few services that might, but I haven’t used any of them.

Here’s an attached svg file:

Yes but KiCad cannot be installed and run on the Travis server. We would be limited to simpler scripting tools such as Python etc.

1 Like

Oh, KiCad other than pcbnew, isn’t scriptable. So this would be outside of KiCad. With that, we could use external rendering, but it would duplicate much of the rendering within kicad.

Ideally, we could extract images from KiCad itself.

How about something like this:
https://pypi.python.org/pypi/svgwrite/

SVG seems like a very good rendering model for this use since it’s vector based instead of raster.

Edit: But it’s also a “simple” text based format. So would be possible to create from scratch, no external library needed.

To be clear:

Our scripts run on the travis-ci servers, which spin up a new virtual environment each build. KiCad software is not installed and so.the scripts would need to download and install KiCad each time. This is something I think we do not want to pursue.

Duplicating rendering effort in Python shouldn’t be too hard* and allows us to do cool stuff like circling pins that have errors


* I have almost got rendering working for symbols. Footprints will be similar amount of work but probably more as I want to be able to highlight certain layers to show errors etc.

I’m using python and pycairo for rendering

1 Like

Apparently GitHub Gists can be used to share SVGs
 (Not sure how circular this argument might become if at least one of the the aims is to get away from Git!)

1 Like