Team PCB projects, best library/workflow practices?

Community
1

Greetings, everyone.

A few people at our institution have used KiCAD to design small PCB’s. We have gone through a few software revision cycles. We started working in KiCAD 4.x, and we’re now mostly using 5.0.2.

Up to now, our division of labor has been one person, one project. We’re now at the stage where we want multiple people to work on larger, shared PCBs which incorporate custom symbols and footprints from previous projects. We would also like to grab parts of old schematics.

We have been making some mistakes in our early efforts. We don’t know the best way to share symbol and footprint libraries. It’s really easy to “rescue” symbols when loading an old schematic, but that also isolates a project from updated symbols. Opening multiple schematics, and copying from one schematic to another is also eluding us.

Most of us are using Ubuntu Linux, and I could probably persuade the one Windows user to switch.

Thanks for your advice.

2

I suggest updating to 5.1 as 5.0 is no longer maintained. The 5.1 version is fully compatible with 5.0 but has a lot less bugs and a much cleaner graphical interface.

And the switch might be even more important for linux users as 5.0 had a reduced feature set because of dependency problems that got solved with version 5.1 (You do not have python support with 5.0 on linux)

I think this is not yet implemented. If you work with highly hierarchical design philosophy then you can copy sheets from project to project (as long as you ensure both projects have access to the same libraries. Meaning a global library setup might be best suited for your needs. Project local libs would also need to be copied if you want to go that route)

There is no single “best” way it all depends on your available infrastructure, your skill level, …

In my opinion the best option is to place your library (and projects) into some form of version control. Make clear rules how any library asset is supposed to be made and if possible have a review process. (I have made the experience that nearly every library contribution has at least one mistake in it.)
One option would be using gitlab, bitbucket or github as your git provider. This would allow you to use merge requests (or whatever the equivalent is called on each platform) to manage your asset reviews. All these providers allow you to have private projects (will cost a bit) and at least gitlab and bitbucket can be installed on a personal server.

SVN might also be a good option as it is easier to learn and allows locking assets to ensure only one person works on it right now (avoid merge conflicts which require skills to be resolved)

Most importantly have clear rules about how the library structure should look like. How should libraries be named? Which asset should go into which library? How should assets be named such that one can find them later? … (Example for such a ruleset: https://www.kicad.org/libraries/klc/)
It might be best if your organization name is used as the prefix for library names to reduce the danger of name conflicts if you ever need to work with other organizations or if you want to continue to use the officially supplied libraries (for example as a basis for personal assets).

Adding new libraries also needs some thought. There is not really an option for KiCad to automatically add libraries. This means if one of your coworkers adds a lib then everyone needs to add it to their lib table. (Will get less and less of a problem as your library grows.)
In theory you could share the library tables (they are just files) but i think it might be more work than doing this manually when required (you would need to have a script that takes the shared lib table and copies it over the users lib table or you would need to point KiCads config directory to the location of the shared lib table.)

A full on lecture about all your options is found here: Library management in KiCad version 5

1 Like
3

Rene gave good advice - I mostly repeat what he said in different words. I have gone partly the same route myself alone, but certain problems come even without collaboration. I have edited projects in two different locations through git, so it’s partly similar to collaboration.

I, too, recommend a version control system and a centralized (in practice 3rd party) server. You have to be careful to always remember that what you are doing is local and you must always share the changes to the version control server.

It’s a well known fact that 2D (or 3D for that matter) data isn’t a good fit for collaboration through a version control system. Conflicts arise and they can’t be merged in any practical way. Therefore you can’t do parallel work on one file. You must isolate your changes into one file at a time and take care that nobody changes that file meantime. Schematics are better in this respect because, as Rene said, you can use hierarchical shematic files. Something similar may come for layout, but not in version 5.

Even without collaboration I have noticed that

Needs grow all the time, assets grow, maybe you have to have a clean start at some point, but try to anticipate. Don’t be afraid to create a new named library when you have only one or two symbols or footprints in it. Moving components from one library to another is also a problem.

One problem which I haven’t solved is what to do with updated symbols/footprints. In KiCad 5 symbols and footprints work differently so they don’t have the same solution. But in general you may make even critical changes to footprints and want to propagate them to new future versions of old designs, but it’s not possible to do that automatically or even semi-automatically. There’s no way to track down which footprint is used in which project. You should invent a good way to communicate critical changes (which affect for example manufacturability or assembly) with the team or even to yourself so that when you make changes to an old project you can choose what updates you take in from the libraries.

Another related problem is that you may have old and new versions of a footprint in a project if you’re not careful. Possibly it’s a problem, possibly not.

Copying from one schematic to another really isn’t possible in v5 through the UI, except through reusing hierarchical sheets. It’s possible in the development version. I’m not sure if this works, but you could try adding even a main sheet of another project as a hierarchical sheet for the new project, then copypaste inside the new project and abandon that sheet.

Reusing both schematic and layout in a controlled manner isn’t possible, either, in v5. There are tricks which use manually copying parts of text files, but it’s difficult.

I don’t think having different OS’s is a problem if you share through version control system, or even through a shared drive. The projects or libraries don’t depend on OS or hardware.

4

Hi Ladsky,
I am using KiCAD to work collaboratively between 3 users within my company. I also work from 2 PCs one in the office and one at home. This works nicely using google drive file stream or dropbox as a common storage location for projects & libraries. Key to making this work are the following:
(1) Set up environment variables, in the configure paths dialogue, for the directories where you are storing libraries etc. In my set up these are:

OB3D G:\Shared drives\OB Library Drive\Electronics EDA\Kicad\3D Shapes\OB 3D Shapes
OBMOD G:\Shared drives\OB Library Drive\Electronics EDA\Kicad\Footprints\OB Footprints
OBSYM G:\Shared drives\OB Library Drive\Electronics EDA\Kicad\Symbol libs\OB symbols

This is really important; Google recently changed there drive naming from team drive to shared drives, which played havoc with paths that are not set as environment variables. I spent many hours updating paths to 3d models thanks to this “useful” Google update!

(2) Configure Google Drive File stream so that an off line copy is kept in your google drive. This is important, because if only the link to the file in the cloud is there, it is too slow and falls over. When using Linux I use a programme called isync to replace Google FS. See https://www.insynchq.com/
(3) Use only the default symbols & footprints as global libraries, add all imported and newly created libraries as project specific libraries. If you do this, the required libraries are available when a new user opens the project for the first time. This avoids the risk of libraries being recovered, which can be a real pain.

This solution works in my organisation, as all of our IT infrastructure is cloud based, this may not be an option for many organisations with local networks. The use of environment variables & restricting library creation to project specific libraries, should work here.
This approach may well fall over if multiple users are editing the same file simultaneously; so far I have not tested this!
Good luck with finding a solution to your needs, there are a few ways of doing this as detailed in the other replies.
Kind regards
Jim

5

Not a good fit for @Ladasky as they want to be able to reuse schematics (and pcbs). For that usecase global libs are the way to go.

6

Not being able to define several library tables (within global scope) is a problem. I have “global” company libraries but I add them separately to each project’s local library table. I’m not sure if this is the best way to go, but I presume updating one global library table is a pain if you mix the KiCad libraries and your own.

It must be remembered that local library is a different thing than a local library table. One should always make clear which one is meant. As far as I can see, Jim_RXP uses mix of local and global company libraries, but I can’t say if the global company libraries are in the local library table.

I agree that project specific components should be local to the project, but I have noticed that if I create a new component, it won’t be local very long. Rather it’s better to move all new components to global libraries, even if you think you will use it only once. Only custom edits to components (for example footprints tweaked for some project-specific purpose) should be completely local.

Again, keeping components up to date is a problem. If you make enhancements or fixes to local components, you don’t get them into other future projects unless you propagate the changes to global libraries. And even then you won’t even get notified if something important has been changed in the global libraries. It’s a complicated task and I think KiCad needs a way to take care of that. Maybe not a built-in solution, but some component management system which has knowledge of all the libraries and projects, and which could compare and propagate changes to either way, upstream or downstream.

For example, I make critical changes to a footprint which affect assembly phase to avoid assembly problems. There’s one projects in which I notice the problem and that’s the moment when I edit the footprint. No matter if it’s a local copy in a local library or just in a global library, I want to make sure that all project which have used that footprint will get that change in their possible future versions. Currently there’s no solution for this other than some TODO list.

Here I come to version control again. It’s too easy to edit several things and commit after a day. But one benefit of version control would be that when you edit components one by one and commit after each change, you can read the commit messages later and remember what you did and why. This could even be a possibility for a semi-automated change alert system which utilizes formal commit messages.

Dont’ be lazy to commit and and be careful when you write commit messages. I have suffered from this lazyness too many times.

7

What I think might work in you case is:

  1. Have your symbol and footprints library in a git (or other) repository to which everyone can contribute and pull from
  2. Add these libraries as global libraries in each workstation.
  3. Create a repository for each of your KiCad projects (or related projects)
  4. When working on a new project, don’t forget to pull both the latest libraries and projects (you can have a bash script that does this setup for you)

I do a lot of boards for the same client and they share a common footprint/symbol library. In this particular case, I have a single repository with all my boards and projects and this works pretty well when switching between workstations.

8

Why would it be a problem to have company and your own libs mixed in the same table? While it might be easier to have your own global table (i hope we will get multiple global tables sometime in the future) i really see no reason to have it strictly separated. (Remeber kicad updates to not touche the library table so there is no danger there)

There kind of is. When continuing to work on a project the first step for me would be to run “update footprints from library”. Check with DRC what changed and if necessary fix stuff up that comes up because of the change. Store this updated version as the first commit of the new and improved version. Only then do i start introducing the changes i planned to make.

If there was a kicad update in between then maybe even do one extra step of saving both the schematic and pcb in a separate commit such that the noise generated by this update is in its own commit. (the same holds true if you edit a symbol library. separate the updated kicad stuff in its own commit.)

9

Oh yeah, I have already forgotten that because I have left KiCad’s own stuff untouched.

But only “kind of” - there’s not much control over what you get this way. And it doesn’t work if you want to have local copies of footprints in a local library to make the project standalone while still using centralized global libraries for sharing. But that’s really more high level stuff. Simple global libraries + update all footprints may be a very good solution for some workflows.

10

There are probably as many workflows being used as there are people. So keep this in mind that no one single workflow is the best.

I personally prefer git to share libraries and project over dropbox/gdrive.

As for the KiCad setup I also prefer using only and only global libraries. As others have said it is common that symbol/footprint used for the first time is likely to be used again. So for collaboration this requires that everybody has the same set of libraries which are shared through git. Obviously any new symbols/footprints/3Dmodels have to be added to the libraries before using them in the project. We use similar procedure that KiCad uses with official libraries. This slows the development of the project a bit but on the other hand every symbol/footprint/3Dmodel is properly reviewed before being used for the first time. So if everybody is using the same set of libraries then the collaboration is much much easier.

As for propagating the changes done in one project to other project, again git with CI pipeline. It would be possible (I haven’t done this) that when a project is pushed to remote repository a CI pipeline is activated which checks if all the symbols and footprints used within the project are the same as in the libraries. Also a CI pipeline could be triggered upon global libraries change, which would check all the projects and report the differences.

Regarding collaboration on a project I know that collaboration on schematics is doable if each person has a dedicated hierarchical subpage(s) to work on. Then you can merge contributions pretty seamlessly without any external tools. Obviously this is not straight forward with layout. But (and this is a shameless plug) it is doable with Save/restore action plugin. If the design permits, you could layout one (or more) hierarchical sheet and export the layout via plugin. Then one has to import all the layouts done, and place them together.

1 Like
11

I just watched this 16-min. video that shows how git works to allow multiple people to work on the same project (set of files) concurrently without losing anybody’s changes.

Apparently git allows each user to work on a “branch” of the project, test his changes, and validate that his changes work with the existing code and data before committing his branch back into the “master” branch.

I am playing with git 2.20.1 as a project version control system for backups of my personal kicad project files.

I think that you should do this AFTER–not BEFORE–you figure out how to integrate Rene Poschl’s “highly hierarchical design philosophy” and his philosophy on global libs and how to integrate data from “global library setup” and “project local libs”.

I have yet to learn the implications of this distinction for a single developer, let alone for a multi-person project. I just know that if different people are developing different functions designed to work together flawlessly, then global variables are one way to make this happen. But doing so still requires careful coordination of changes done by every possible sequence of changes to data in order to make it work. And still there will be an unforeseen sequence of changes that results in kicad files becoming corrupted.

1 Like
12

Unfortunately most of the benefits of git which apply to programming language source code projects don’t apply to schematic or PCB design files. Despite the fact that the files are human readable, data which describes items in 2D or 3D space just isn’t suitable for distributed workflow. Merging isn’t practically possible if there are conflicts. Therefore branching isn’t useful, either. Only as much as data can be split into different files a distributed workflow works.

1 Like
13

Step:1 Create a library. Get a file server that is backed up daily to two separate locations and designate that as the storage location for all of the teams IP.

Step:2 Appoint a librarian (s) to organize and assign storage space and names. They create your Dewey decimal system. Document everything

Step:3 Pick a revision control system. If it is not git then repeat step 3 until it is.

Step:4 Gather,collect, sort,categorize all of your legacy IP and store it in the library. This includes all versions of all tools and tool libraries.

Step:5 Designers will start their projects by checking out the projects repo from the library and committing all changes back in. When a project is finished the designer will delete the repo from their workstation. If they find that they need that repo in the future then they must check it out again. Stale repos and not considered safe.

The designer does not send the release package to the board vendor, they send it to their manager. The
manager will check out the repo and recreate the package. If the two packages are not identical then we find out why.

14

“Learn GIT in 15 min” is a 16 min video? @@ marketing at play again! :smile:

15

Well you need at least a minute to get the mandatory “don’t forget to like and subscribe” and other overhead across.

16

Thanks, everyone for your thorough advice!

To clarify what capabilities we have at our disposal: we have an enterprise Google Drive. Placing shared Google Drive folders in our KiCAD path lists sounds like a good idea.

In my own KiCAD workspace, I have stored all symbols in a “Custom_Symbols” folder and likewise with “Custom_Footprints”. Some of these symbols and footprints might be updated in newer versions of KiCAD, I’m not sure. If they are, I should be able to switch because I haven’t mixed my symbols / footprints into the official KiCAD hierarchy.

Regarding version control: we looked into GitHub in the past for another project. We might or might not have to pay for what we need (we’re a non-profit university, and there are apparently academic accounts). If we had to spend money for GitHub, it’s not too intimidating for me to consider setting up a git server on my work PC. Alternately, we could ask our IT manager to run a git server on another machine in the building. We would lose GitHub’s web interface, but people who develop PCB’s can probably live without that.

17

Gitlab and bitbucket can be setup on your own systems and have a near identical interface compared to github

18

At least Gitlab.com offers free private projects with unlimited number of collaborators, so that could be a good option.

(EDIT: we have to understand the difference between a hosting software platform and a hosting web service - GitHub and Gitlab have developed their software systems which you can install locally and host your own projects, but they offer also web hosting. The software is Free, at least partly; their web hosting services have free and paid options. Choosing between free and paid affects the features and services the user gets. Doing your own hosting is out of question for many small companies and especially individuals.)

If it wasn’t clear already, using git is overkill for PCB development. (Actually it’s overkill for almost any development, software or not, Open Source or not – it was made for one unique project with needs unlike 99.9% of all projects.) But it’s a de facto standard and useful to learn anyways. The web service interfaces are handy and offer much more than just git. I haven’t used for example issue tracker for PCB projects, but actually this discussion made me think it could bring huge benefit for our projects compared to ad-hoc notes I currently use. Someone hinted about possibility of some kind of automated checks. Etc.

19

Git is certainly not the only game in town and may not necessarily be the best fit. It does have lots of support and lots of learning resources and some fancy front ends. However, there can be some advantages to other versioning systems - for instance, using a centralised, non-distributed system such as SVN for common shared assets can be useful as items can be locked when checked out for editing.

Using a distributed system, you need to take some care to avoid merges as dealing with a merge conflict in a pcb layout description is very different to merging source code. I think KiCad has got a bit better in this regard - in the past great chunks of the design text got moved around rather than altered in place making viewing diffs a nightmare.

Adding features onto Git - like documentation and bug tracking seems to require extra modules. Git ≠ Github. I have found that Fossil www.fossil-scm.com used for managing the source code for Sqlite is a really simple, compact and self-contained distributed version control system that is well worth exploring. The whole website linked to is itself a fossil repository. Email, ticketing, tech notes, forum, wiki and web server included. Active development, free, cross-platform and with a really helpful support forum - you often get posts from the lead developer in answer to your queries - I doubt that Linus will reply directly to your Git query!

1 Like
20

It looks like it has advantages, especially if you want to do your own hosting. Everything in one package, easy to install and set up. And as they advertise, it may be a better fit for a centralized project which PCB design necessarily must be and even most of the software projects in the world are anyways. I have never tried it, though.

1 Like