Workflow for better KiCad library contribution (Git + KiCad)

In my current job we are starting to use KiCad. We want to contribute to the KiCad libraries as much as possible but we are having some trouble finding a suitable workflow.

The current setup is the following:

  • We have a team shared fork of each of the four repos of the libraries (only two or three people is going to work in this repos so we don’t want individual forks).
  • Currently, the master branch can only be written by me in order to avoid conflicts with the upstream repo and to be able to mirror the original repo.

We want a way to push changes to the libraries in the most individual way possible so that the PRs are kept small but on the other hand we want to be able to have our changes together so that we can keep working on the current board projects.

At the moment we are creating a branch for each PCB project. We create symbols and footprints in that branch. After that we have to create a branch from master and manually transfer the changes for each footprint/symbol to be reviewed. It is too much hastle to be efficient.

Any suggestions on how to organise this in a better way?

if you keep your commits clean you can use git cherry-pick to create your branches for the pull requests.

That is usually the case. History is usually clean. It may be a little more difficult sometimes with symbol libs due to multiple symbols in the same file.

With a little bit of effort this will surely be an improvement over the current workflow.

Yup i know the feeling.

First, in your working copy of the fork add the KiCad repo as another remote:

git remote add -f upstream

(Your team members don’t need to do this, only the maintainer of the fork)

This incantation will take care of keeping all branches up-to-date between your fork (origin) and the kicad repo (upstream)

git push origin refs/remotes/upstream/*:refs/heads/*

Here push origin sets the target repo, and then <src>:<dest> where src is absolute refs, and dest is relative to the target. This keeps your fork up-to-date with KiCad.

I would suggest you create a new branch in your fork, say ourstuff and make it the default for your team. This keeps your fork master clear and consistent with the KiCad master. If there are conflicting changes between Kicad master and your ourstuff branch, just merge it into your branch and resolve the conflicts. Depending on the nature of the conflict, it might be easy to do with file merge tools.

This isn’t directly related to answering your question, but for the digikey-kicad-library we have a source folder with one part per schematic lib and there’s a python scripted build step that changes the PN from Digikey to MFG, organizes into logical groupings and adds metadata from the digikey api to the final usable libraries. Our primary motivation for this was management and workflow, but it’s a bit different use case than yours. We have something like 8 people contributing so it’s nice to have each symbol as a single lib for merging purposes. It’s not so great when people want to edit the finial library in-situ and contribute back.

For fork sync I do something similar. As the mantainer, I am the only one allowed to write to master thus no conflict will take place. I am only interested in syncing the master branch and this is done with a script automatically in my machine. That is working correctly at the moment and it is not the problem we are facing.

This is the real source of conflict. When we need to develop a new board with multiple new elements in it we create a branch to do so. We are forced to do this as we cannot create individual branches for each new element because KiCad is not compatible with that workflow.

Once we have finished the initial board definition and we want to push the library elements to the KiCad repo, reviewing many changes in one PR is far from ideal, becoming very tedious and long process. For this reason is desirable to split the changes in multiple branches, which also becomes far from desirable.

I have contributed script generated elements to the current KiCad library. In some ways this is less tedious as usually one script generates one full lib thus allowing to do a copy paste once all changes have been reviewed. It is still far from the ideal integration that one may desire for such a tool.

For the previous reasons I am trying to think in some way to improve the current contribution process. I believe that I can get to a point that if steps are correctly defined most of the task can be scripted.

Will the new library format split the symbols into different files such as in the footprints? That may simplify my problem.

Yes the new format will be one file per symbol.

Any estimation date for this to be implemented/used?

I am not part of the development team. I hope it will make it into kicad 6. But still kicad 6 will be at least 2 years after kicad 5

Two years :sob:

Very very slow release cycle. I’ve became familiar with KiCad during the 4.0.7 to 5 transition which is still happening so this has given me the impression that things moved quite fast.

Large changes like a massive change in the underlying file formats take a lot of time to implement and even longer to test.

Yes, I know. That is a major change and proper testing needs to be done. Also, changing the library formats is a pain for users, developers and librarians. Given that, I know that such a major change would not happen sudently but thought that someone had been working on it for a bit now. Also I would have expected a release cycle a little bit closer to one year independently of what major changes are included in the release.

What would a yearly release be worth if it does not really contain new features. It only gets in the way of making large changes. If you want cutting edge use the nightlies.

I was thinking about the smaller features that may improve the usage of the program and that do not require such testing. I try to keep up to date with the nightlies as much as I can for this reason. But I guess that at some point or another a big change like this would need to freeze other with less relevance :smiley:.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.