Can I propose something “radical” for the library stuff folks? I too like the idea of a relatively easily accessible holding pen for new parts but as reported earlier in the discussion, (too) many contributors often submit ‘good-enough-for-me’ parts. I’d suggest to add an “Obsolete” repo in addition to the “Pending” repo, and use it in the following way:
After a user submits a part, it is stored in the proposed “Pending” repo. Normal submission rules apply and our intrepid Librarians do their chosen job. If said part is not fully compliant to be included outright, and the aforementioned Librarian comments on the part to make it so, the user submitting the part has a 3 month window to fix it, following the date of the comment. If no or insufficient action has been taken during this period, the part is moved to the “Obsolete” repo. In this repo the part is only visible for the user and Librarians, other users have no access to it. This means it cannot be used in new designs. The “Obsolete” repo also has a time limit: after 9 months the part is permanently removed. During this period (in total one year from day of submission) the user can work on the part to improve/fix compliance and request a revaluation by a Librarian before the allowed time runs out and the part disappears into the digital shredder. (note: one could read “obsolete” as “I-don’t-give-a-fsck” But that might be a bit harsh)
Having said that, an external tool to assist contributors to submit their new parts (libraries, footprints, 3D models optional) could be used to ease the burden on using Git for non-coders (like me) and at the same time offer functionality to check parts prior to submission for compliance should reduce the number of ‘good-enough’ submissions. The tool could (would/should*) give the PuT (part-under-test) a grade, reflecting the measure of compliance. This grade can then be used by the Librarians to determine which part to check/comment on/include on the basis of a higher compliance number meaning less work for them to allow the part into the stable/production repo. For obvious reasons said grade cannot be tampered with by the user submitting the part.
Everything you describe here does already exist. The pending repo is the pull request itself.
There are scripts for locally checking contributions. (the same scripts that are run automatically for each pull request.)
Closed pull requests could be seen as the obsolete repo.
The pending lib as suggested would allow other users to step in and help; the pull request does not. There is a difference @Rene_Poschl, or am I overlooking something?
Another thought for @Dutch_Master: For my parts, we had a small but useful discussion about in which lib the part should be stored when done. For many parts it is obvious, but for some parts that could fall into different categories.
This can be done in the pull request, as usual. I would not like to loose the information or discussion on the definitive lib when a part is in the pending lib.
And, finally, a thought about git. I appreciate some users have difficulties when trying to use git. However, it does exactly what it is supposed to do. I believe it is the right tool for the job. The job of maintaining information and allowing many people to change it in a controlled manner, is not easy. We can discuss tools at length, but in the end I believe that if we were to change tools, the job at hand would not make it any easier. It is a complicated subject and will remain complicated.
By the way, the first time I added a part to the libs, I followed the instructions (almost) to the letter. They were really helpful.
Fair enough Gents, I’m not overly familiar with the procedure. A GUI would simplify things for most contributors though, I’m not aware of an already existing one? (but that’s probably me missing something!)
I imagine that the role of the “Pending”(“Unstable?”) lib would be where parts are parked during their review by the librarians. It could be a place where parts are first put for community perusal and testing if they meet a level that is not quite good enough for the official, but not bad enough to reject outright.
I’m not sure I understand why an created part would be removed for lack of communication, when there should be a better way for other people to fix the part. This way information is not lost, and can be corrected by the community eventually. Of course there may be other reasons for removal from the “Pending/Unstable/Unfrozen/Slushy” lib which may include being superseded by a better version of the part, poor/incorrect design information, EOL, or simply no one uses it.
How would you want to oganize work cooperation on these “pending” symbols/footprints? SHould they reside in a GIT repo? Then can anyone edit the symbol at the same time? How do you organize that? I think the PR-system of GIT is actually a quite good option … If you want to help someone, you can simply download the PR, edit and then post a patch that the initial poster can apply and use … If you have a repo for “pending” that anyone can edit,I fear that will end up in chaos … or should we develop a completely new versioning system?
As one of the librarians, I see the shortcoings of the current approach, but I also don’t really see how to improve on that … an integrated GUI in KiCAD which does the necessary GIT-procedures in the background would be nice though …
I agree with what Jan has said above. Further to being unregulated it also increases the work of the librarians. Now there are two systems to maintain, and the responsibility for making good quality contributions is shifted from the contributor to the librarian team! People will simply dump their submissions into the pending bin and not give it a second thought.
An overarching response to the key questions posed in this thread should be:
“If you don’t find the symbols you want in the libraries, it’s because you haven’t submitted them yet.”
The system we have in place isn’t perfect but with the automation work we’ve been doing, it’s heaps better now.
A GUI tool to help the process along could be good but it still needs GitHub authentication and would be huge amounts of work to get it usable, let alone helpful.
Most good quality submissions are merged very quickly - if users are willing to play their part, the library grows.
Maintaining two systems would indeed appear to be more work, and I can understand the concern of it raising the workload of the librarians. But I also think of a pending repo system to be more of an extension of the current system that allows faster community assistance.
I envision an ideal system would consist of some of the following points:
All editing of the part and submission of changes would happen within a Kicad plugin. The use of wizards could probably help generate KLC type libraries with reduced mistakes. Technical challenge: wizard development, plugin system, schematic/footprint/3D editing systems.
It would have all automated checks to submit parts to the pending library(ie with a 90% pass of a weighted test) that is a repo on git. I’m not completely aware of the automated testing systems used by the librarians at present. The pending library or plugin would not come with a standard install and would need to be added after. It would still require a valid github login. Technical challenge: Automated submission through a git client accessible within Kicad. Must track and allow edits/comments from github.
From there users could edit parts based on the comments of the part and the reasons it had been previously rejected. If the user has made the required changes the part could then be submitted/flagged for librarian review. If rejected, it stays in pending with the comments or reasons for rejection until the next time one of the community chooses to use/modify the part.
In open source terms, I believe a system like this would allow parts to more easily churned through the bazaar before being sent to the cathedral.
Ideally this would mean that the librarians unless making a part from scratch themselves would only be looking at the parts that have been submitted by the community through the as yet to be developed plugin (Kicad Integrated Pending Public Engagement Repository or KIPPER for short )
This is of course theoretical, and without the requisite technical solutions to the above challenges hard to move forward on.
No (the only option would be a field “Comment” for symbols, but nothing comparable in footprints that I’m aware of … it does not. What did you have in mind?
Storing review comments or usage notes in the file. Even if there are “generic” comments within the file format not necessarily within a symbol or footprint definition, there could be a standard format defined for associating a comment to a specific symbol/footprint.
There are the description and keywords fields that store a description of the symbol and for footprints usually a description, usage comments, a datasheet/sketch link … Why should there be review comments in there?
The best part of the github system is that I can use Screenshots with some highlights to communicate what is wrong. Text alone is not enough in most cases. (especially for footprints and 3d models.)
“I’m afraid to say these library discussions will be endless”
It would help with some standards in place and reference material provided: IPC-7351/IEC 61188-5-1 Standard PDF
Next is to make sure there is somewhere to list what ‘Standard’ was used and any deviation to that standard Inside the part file.
And then to make sure everyone realizes that the board house that builds the PCB has final say.
If you want to submit symbols or footprints to the official Kicad libraries then the “standard” is the KLC, Kicad Library Convention. The reference material is provided here:
Don’t despair Librarians! No matter how much thought and effort you put into getting a system that works most of the time there will always be those who, after a minimal thought and effort, try to tell you how to do it better.
While this topic is now almost 100 posts long, judging by the level of participation I think it’s safe to say that the majority of us are content with the current system.
Although most of us, myself included, would like if the symbols and footprints in the libraries were in our preferred style, such as zigzag resistors , we should be grateful for any libraries at all. At the very least they serve as a starting point for our own libraries.
A discussion like this serves two purposes. One, it helps those who aren’t aware of the reasons why things are done a certain way better understand the social, functional and technical basis for design and implementation decisions. And two, it allows fresh eyes and new ideas have a look at a challenge where they may be improvement and possibly look at a problem in a new light with new information.
No one is saying that the librarians are doing anything wrong in any way. But if there is a possibility of improving the process and helping them with their fantastic efforts, why not talk about it? This discussion costs nothing to have, and this is where old problems might be looked at in a different way.
All progress comes with some change, and I am aware that there will be more beneficial changes with v5 and v6 etc, so it doesn’t hurt anyone to talk about possible changes along the way.
I am tempted to make this thread a FAQ item for the new user who pops up every three months asking why their parts are not all included as they want them
But that might be a bit harsh)
)