As for the schematics, it would be great if someone would build binaries from plotkicadsch. Then the rest of the work flow would be practically same as above. Another option would be if developers implement command line interface for eeschema, which has been proposed before https://lists.launchpad.net/kicad-developers/msg06052.html
This coveres only diff functionality, which is based upon svg files. Merging is completely different beast as @bobc already pointed out so I am not even thinking about it.
As for development, concurrent development is doable in eeschema if you limit yourself to separate hierarchical sheets. I don’t know how the pcbnew workflow looks like, but I’d like to hear some experience.
I have done a ‘Visual Diff’ for PCBs. (https://github.com/Gasman2014/KiCad-Diff) It takes commit references and performs a visual diff between them (or between current version and commit reference). The output is presented as a webpage, split by layer with the differences highlighted in red (removed) and green (new). Happy for anyone to come up with a less garish .css!. To make a meaningful text diff to back annotate which actual component has changed would require a lot more work. I think you could do it by parsing changes in the s-expression back up to the source component in the tree.
If there was a python interface to eeschema, it would be easy to do the same for schematics. I have tried to do this using janvila plotkicadsch but with limited success (plotkicadsch is still in development). I have only got neo900 to run on Linux.
I was about to suggest something similar with KiCad. It should be possible without big issue to work in separate schematic pages. I believe only that schematic page file will change.
One thing is basic version control. There is a lot of internet sources available to get you up to speed.
What is the VCS workflow with KiCad is another thing. As already mentioned in this thread take a look at Massaging your git for kicad.
And even when using version control with KiCad things are quite different if we are talking about one man band project or a team project.
In one man band projects, you don’t really need any special workflow. Pick files which should be under VCS and commit them (project-cache.lib should be under VCS). Until there are good diff tools available for your platform this is about it. If you feel brave enough you can integrate any of already mentioned diff solutions so you easily can compare revisions.
With team projects the main issue is eeschema libraries. There are a couple of setups that work:
each team member has exactly the same libraries, in the exactly the absolute path on his/hers system and in exactly the same order. Obviously this works only if everybody has the same platform (Win, OsX, Linux distro). Even if this is the case I would not recommend this setup.
each team member has exactly the same libraries, somewhere on his system and you use environment variables to resolve relative path (Environment user-defined variables: how to use them?). This is a better solution, but it requires some manual editing of .pro and .sch files.
each team member has his/hers own libraries. If helps if these libraries overlap. An example of this would be official KiCad library with user appended symbols. Also using enviroment libraries like in 2. is preferred. Additionally for this setup to work you have to manually edit .pro and .sch to place the project-cache.lib on top of the library list. WARRNING: this is somewhat unsafe procedure, but if everybody is aware of this it works quite nice. How this works: when team member A puts symbol X in the schematic, the symbol is also copied to the project-cache.lib. Once team member B edits the new version of schematics, Kicad grabs the symbol from the project-cache.lib, sot team member B does note need to have the same schematic libraries. Caveats:
nobody ever edits the project-cache.lib
if you want to change symbol for one part, but there are two or more parts in schematics you have to delete the part. Create a new symbol for the part in the user libraries under a different name and place it in the schematics.
With 1. and 2. you will quickly figure out that having the libraries under VCS is a must. With 3. this is not strictly required but it helps a lot.
Disclaimer: I’ve tried 1. and 2. but I’ve quickly abondoned the setup as it caused a lot of headaches. Since then I’ve done three projects using setup 3 and they did not have any major issues.
Create your project as normal, or use existing project
cd to project folder and type git init
Add files with git add <file> or git add *
Commit to repository with git commit -m "message"
That’s all the basics, for 3 & 4 and most other regular tasks I normally use the GUI, type git gui. In Windows git commands are available in Windows explorer, so you don’t need to use command line.
Whilst Git is an excellent tool, it does have a steep learning curve to use fully (https://xkcd.com/1597/) If you do any software development then learning to manage Git is probably a good investment as it is one of the most widely adopted SCM tools. There are some good graphical git tools out there too - but git on the command line is often necessary and a tricky beast. One of the advantages of Git is lightweight branching and merging. Branches are obviously useful in code development and useful for development of alternative versions of a design. They make it easy to try new variations with the ability to back out your putative design to where you were before. The ability to merge though is more of a problem - whilst it is great for software, merging is less useful for hardware projects - reviewing a text merge of your schematic or pcb layout is problematic at best, meaningless at worst.
Git is certainly not the only show in town - @Andy_P mentioned SVN which is another excellent choice and one which is generally considered a bit easier to learn. It is a centralised VC system (i.e there is one true master and a client - server setup ) and is not a distributed, peer to peer system like Git - this may or not be an issue depending on your use case and how you plan to deploy it. There are many others - CVS, Mercurial, Perforce etc
I would also highly recommend having a look at Fossil - www.fossil-scm.org - which is another but less well known distributed VCS. This is a single, small binary (< 3MB) - available on all platforms and is very easy to deploy and use. It is free and open source. It is very well supported by the author DR Hipp who is also the author of SQLite. SQLite developers use Fossil-SCM as their VCS of choice so, although it is less well known, it has a good pedigree. It is also under active development and has a supportive user group (which, as you know, count for a great deal!!).
For electronic design it is an excellent fit as it does all the SCM version control stuff but additionally has a built in bug tracker, wiki and tech notes sections which make it very simple to keep all your project documentation in one place, cross referenced and easily presented. The bug tracker can be used to link commits to specific ECOs. Add all your datasheets, design notes, project libraries etc to the checkout and commit away. It is very easy to use & has an inbuilt web server which provides an easily accessible GUI.
Only because people are not prepared to read documentatios/ learn the ■■■■■■■ command line. (I hate guis. Well i am using unix for 12 years now. Never going back to a operating system with a weak command line.)
Whether you find Git straightforward or not rather depends on your background, approach and needs. If you are a programmer, especially dealing with anything open source you have pretty much got to know Git and in some depth; you are probably using for other things all the time and it is or becomes almost second nature. Git is not too difficult when used for your own projects but the complexity increases when you start to clone, push stuff back and then put in pull requests. If you do that all the time, it becomes much easier and familiar but for a lot of people wading through it for the first few times, this is where it starts to get a bit confusing.
If you have used other versioning systems - esp non-distributed systems like SVN there is definately a different approach needed and you do need to think about Git in a different way. This might add to the confusion for those who think they already understand VCS and come unstuck when they first try Git.
The point is that there is an investment in time needed to learn anything - Git is no exception - if that is central to your work then it is obviously well worthwhile. I don’t do any programming or electronics at all in my day job so that investment is entirely from my spare time. If you are trying to learn Git to manage something at the periphery of your main interests and then don’t use it very frequently, you might be forgiven for taking the view that wading through a big manual to do something fairly simple is too big an ask. Its a bit like learning vi - if you use it all the time, then the shortcuts available can save an enormous amount to time - but nano/pico does a useful job for the those where the payback in terms time saved /time spent learning is not worthwhile. It’s that shallow learning curve again…
Anyhow - I really quite like the command line for all sorts of things and usually have a terminal open somewhere - but I wouldn’t want tot try to layout a PCB with it alone - after all, Gui’s do have their place …
git promising been faster, only right way, only for “real” coder which motivated me to dump all my company 20GBytes SVN repository away into git. After hard work for that, I found – oh shit – the “term” of fast isn’t the same as my dictionary. Lucky, some one come up with LFS that kind of help my repository usable again (after a huge amount of scripting and self learning) for correctly convert and “life” sync commits from SVN to git or via versa. But so many oh shit like (I need another work around script before continue my project)! So far I like the git idea, great flexibility, great motivation for parallel development, but I still suffer to fit in. I had not see the maturity level as and old guy tools offer yet. So work with git like chasing mine 10 year old boys
Just like is - how long you can keep your LCD TV work without repair vs the stupid CRT TV?
Is there a way to have the libs stored in a SQL database ?
A fairly common setup for large teams, where libs are “connected” to other manufacturing and management databases.
Where I work, we use Altium and have a centralized libs repository. Libs are stored under SVN (not a database).
To use the libs, we checkout local versions.
Now, ONE person is in charge of maintaining these libs (the PCB routing technician). This simplifies the process (no concurrent editing, or names clash).
Every time we need a new component, we append our request to a “prioritized request list” to the person in charge, who updates the lib, and draws the symbols and footprints as needed.
Bottom line : we have to checkout the libs quite often.
This is a bit like working with a database, except that we have to checkout to stay up to date with the changes (a central database is always up to date).
The libs them selves not. (Extremely simplified: SQL is a specialized toolset for managing information. Kicad is text file based not based on a database file system.)
If you are worried about ordering information then you have two options:
Include everything in the symbol and keep that up to date, or include a so called house part number that indexes to your database.
In both cases you would heavily use scripts to manage that.
Works the same in kicad. Kicad libs are simple text files so any tool that works well with managing text can be used to handle kicad files.
When you use svn you don’t really need to have one person in charge. One can lock files (and i think directories). This takes care of avoiding conflicts.
The second approach is to use the git philosophy and instead of avoiding collisions provide a toolset that makes it easy to fix them. (This gives away control from a central authority and moves part of the responsibility to the people contributing.)
Both philosophies work very well with the kicad library as it is based on human readable text files.
For databases you also need to “checkout” the information. (Somehow you need to get the information from the database server to you)
See the svn server as your database server and the local repository as a cache. Everytime you need to work on the information you need to make sure your local cache is up do date. The difference is that in most database applications all of this happens somehow hidden from the user.
What you really wish for is a better integration of kicad in version control systems. (Some way to do at least the most basic svn or git operations directly from within kicad)
With the quote from my previous message, I have to add now, that it was written for KiCad V4. As V5 is handling symbols in the same way as footprints, the collaboration is a lot easier and using git for library management and distribution is practically the same as you have written for Altium. With git branches, you do get an option to have multiple branches available locally at any time and you can merge between them quite easily.
We are using the following setup:
As the foundation library we are using Kicad official library. This means even though we add element to our own library we can still pull from the official library.
We have an organization remote repository (three of them for symbols, footprints and 3D models) reachable to anybody in organization. Here we have basically just one branch named “org_master”. Anybody can pull this branch and a few people have write privileges and they are the ones handling pull request from everybody else. For pull requests, authors can pull their branches to this repo.
locally we encourage people to maintain their own branch and merge from official KiCad library and organization library master branches, but those who do not make new symbols and/or footprints are staying on organization master library.
With this setup you can even change entries which come from KiCad official library and still merge any changes (e.g. in footprints we set the footprint name on Fab layer to invisible, we add certain fields to the symbols, …)
In fact, database integration goes far behind the libs (and templates) storage, at least if there is any form of manufacturing management process. In this case, components and PCB are only a part of the story, and components references needs to be “linked” to other data (witch are almost universally stored in database tables).
Indeed, hobbyists and small scale operations have different need.
Now, thinking again about it, it is not necessary to have the libs themselves in the database.
The only requirement is a way to uniquely link a “component” lib entry reference (not the symbols, footprints etc…) to an unique database entry.
Nice ! Merging branches (projects or libs) is a dead end with Altium! (the Altium’s integrated VCS interface don’t offer the option - for a good reason !).
Have you experienced any problem when merging libs ? The alterations you describe (flags or new fields) are mostly isolated “one liners”, that are probably easily dealt with.
But any problems with symbol shape or footprint mods ?
As this is a “text line” oriented merge, there may be cases where the context is inadequately assessed by the merge tool, and you get errors and conflicts.
Happens with heavily modified source code.
Not really. And i regularly merge them as i am part of the official kicad library maintenance team. (Only symbol libs make problems sometimes but if you do it for a few times you will be able to read the file format well enough to make manual merges possible.)
Sometimes we get the same part submitted by different people. This can be avoided by having some internal processes that force your users to check if their part is already in the process of being merged. (Avoiding double work is always hard no matter what backend you use)
A specific merge tool (in the VCS backend) that would atomically replace top level entities (instead of text lines) would eliminate inconsistencies, but, based on your experience, it doesn’t seem really necessary for day to day use.
If it would be a problem then we librarians would have invested time into creating such a tool. Just be aware that git is much more powerful with regards to merging than some other version control tools.
