Does Kicad need version control integration, or is it just me?

Schematic
10

I’m not sure what integration with version control would mean but it seems to me that the difficult piece with making KiCad work with version control is there is nothing that’s the equivalent of diff. How do you merge two files together that have both diverged from a common ancestor?

If KiCad could somehow read two schematic files (or two circuit board files) and display where they differ and offer you a way to select one or the other for the output, I think that’d go a long ways to helping this problem. Sounds like a big job though.

11

You mean something like this:

Disclaimer: I’ve just heard about this. I have not attempted to use this tool yet so I don’t know how robust it is. As always, YMMV.

12

Some idea: It would be nice to implement something in / with KiCad that renders all schematic files to PNGs files and keeps a current and previous version. People could keep it in the version control (ok I know they are binary) but it could help to visualize the changes between versions.

13

I’m sure there are people who don’t like git submodule, but you make it sound universal? In the 21st century it’s practically impossible to write useful Software which does not rely on other people’s code. Even the C Programming Hello World relies on the C Standard library. You can implement your own printf() but why?

Thank you all for your responses. I get the feeling that electronic design is not a collaborative or cooperative process. At present silicone companies create a device and in a datasheet they draw a picture of a bog standard schematic for the device. I then have to read that datasheet and manually copy that schematic design into a CAD package, in this case Kicad, and possibly tweak it to my requirements. Apart from the tweaking all the rest of this ‘work’ is being repeated numerous times. I’m surprised the boilerplate stuff ain’t automated.

This all started with my switching a SMPS to the LM5088. I’m not expecting the company to create a schematic file for every CAD package out there. People are already doing that but it’s work that currently doesn’t appear be shared easily. To me it’s a logical extension of creating a symbol, or footprint, in the kicad libraries. I could add the LM5088 to my own private library but that’s not helping anybody. Adding the symbol to the official kicad symbol library and perhaps it helps somebody out, and they don’t have to repeat boilerplate tasks. (Hippy :wink: )

I should point out that I never suggested which version control system should be used, I just said that I tried this with git. And my subject / title was really bad in using the word integration. That’s the wrong word and I should not have used it, sorry. I should have said that Kicad could perhaps “facilitate” version control. At present I can’t open a design and close it without “saving changes”.

Somebody suggested that a schematic file is useless without a library file. I’d imagine that if I could share a kicad schematic it’d be best if all symbols were from the standard kicad symbol library.

Thanks for pointing out git smudge, that might be the way forward. I must look at python in Kicad as that might be the way forward as well.

14

Like this?

2 Likes
15

This is also a promising approach:

3 Likes
16

Yeah, that’s the sort of thing I was talking about. Well cool.

17

I may have looked like a bit aggressive in my previous reply. Let me clarify my standpoint.

git submodules are useful and useable for an extremely small set of use cases (mostly, independent external libraries). For most others, it can be considered harmful (messing with the included submodules can be tricky). I would recommend using git subtree for most of your needs, this brings saner repo setups.

Anyway, being part software, part hardware designer, I can tell you that hardware design can be as collaborative and cooperative as software. The main difference lies in the artifacts generated from these activities. The development cycle of software is more iterative and can be split in cleaner subparts, without taking care of the cost of deadcode, dangling features, mainly because you don’t have economical incentive to do otherwise.

In hardware, you usually want your final product, the electronic board, to be as tightly integrated, focused and price aggressive as possible. That’s more a team work on a sprint (tasks can be either functional, design, routing, procurement). What you can share between designs is a working setup, some ideas of hierarchy and a set of proven references but all this stuff is not easily formalized.

Working together on a single schematic is possible and even encouraged. The same rule applies as for software: if several collaborators are working on the same part of the schematic, you can have conflicts when trying to merge. Otherwise, Kicad is quite friendly to the git workflow, when you use the clean/smudge feature. You have the same issues with some code editors that mix up pure build recipes with IDE setup in the same files.

Each Kicad schematic project embeds a cache library file of all the components used, so just adding this cache file to your git repo ensures that your commits are all complete and self-contained.

2 Likes
18

The main difficulty is that storing old versions is the easy part, you can already do that.

What is hard is finding a minimal set of differences between two versions (so changes can be reviewed in isolation) and merging diverged states.

Text files are a one-dimensional stream of characters with newlines in between them, and tools barely keep up with these. If you swap the order of two functions in a file, you get a huge diff, even if there are no functional changes, reviewing that is annoying and you can almost forget about automatic merges. Most projects have adopted a coding style to keep pure formatting changes to a minimum to avoid conflicts.

With schematics or PCBs, we have a two-dimensional layout, where functional and aesthetic choices are interwoven even more than in text files with program code. There is no “layout style” to be followed that can be automatically verified to reduce conflicts.

If we had a good proposal how to solve diff and merge, the implementation of that should be straightforward.

5 Likes
19

This is the minimal way I have found to state the difference between pbc board design and program source code editing.

In programming one change in the editor is one identical change in the file. In pcb design one change in the editor is a random change or many changes in the file. Therefore in the latter merging becomes very difficult or impossible.

1 Like
20

I don’t think we should expect a full VC integration, but there are some principles that could lead to better interaction. Two of them are the principles of minimal change and of locality.

  • minimal changes implies that changing the place, the shape or a property of an object must not change the place of the description of the object in the file. This way, a change to an entity appears as a unique diff hunk in the file, not like a deleted block somewhere and a new one somewhere else. Kicad already follows this rule.
  • locality means that the new lines matching a group of additions to a schematic should be relatively close to the lines of the parts close to the changes in the schematic (and close to each other). This one is more blurry, but it should allow to easily merge files where changes were performed in different locations of the schematic by not making them appear at the same place in the file. This is not bullet proof, but at least, it would be better than today’s append strategy.

I know it’s easy talk and I’m pretty sure it could be done offline by finding new primitives in the schematics and moving them to a “more sensible” location.:thinking:

21

I didn’t take it as at all aggressive, and sorry if my response made you consider that possibility. I was just very surprised that git’s submodules was frowned upon as it’s the only mechanism I’ve ever seen used for managing code dependencies which are in another git repo. I don’t claim to be an expert at all and wondered what the alternatives are, as I’ve not encountered them. I’ll have to look at subtree.

I agree with that statement, but I can’t image that 100% of Kicad’s users are focused on that set of requirements. I’d assume that’s the high end Hardware design shop and I think probably a significant percentage of Kicad users don’t design PCB’s for blue chip HW venders.

I must try to both find and follow an open Hardware development project as I can’t see how the tools are available for a smooth workflow. That’s my perception with very limited knowledge, or imagination. Perhaps my initial message documented a problem which doesn’t exist, but given the effort that people seem to be making to get around the non existent problem, writing bash or Python scripts to do image diffs, or using git smudge there seems to be an issue.

I’ve obviously got little idea of the internals of Kicad, and perhaps I’ll make some time and start looking at the code, but it seems that Kicad contains all the information that it needs to specify a schematic. (I’m going to concentrate on just the schematic for the moment.) So Kicad processes and saves all the data which is needed for the schematic I just wonder could that data be redistributed in files so that some files contain the essence of the design but other files contain implementation details, which change on a regular basis and are project specific. So I hit a snag with component references, perhaps they are project specific not part of the core design. So in my example two project share a LM5088 SMPS, the circuits are exactly the same but component references are specific to the individual projects, not the circuit design.

22

Seeing as I have been working on an open project keeping the kicad files in step with the eagle files, I feel I have a good sense of exactly how you would need version control to work for a veiwer for minimal complexity creating,

  1. Footprint Locations and Rotations, Kicad already can export this Info, but currently unsorted (I have made a report on this) https://bugs.launchpad.net/kicad/+bug/1793764
  2. Netlist differences between the Board, Summerised to different footprints, different nets on pads, and different values,
  3. Different zone polygon outlines and properties, ignore the fill data and just look for the shape and settings,

This would cover most increments giving the most surmised form of changes on a board, yes there is much more that can change, but trace data would make it much harder to merge, keep track of, etc,

2 Likes
23

I second that. I fullty agree that some form of version control would be very nice, but it does not need to be “integrated” into kicad, whatever people think that means.
m

24

I have been dreaming of light integration so that with some key combination KiCad would do git commit with automatic message. It would be nice if the message would be human readable, but it would be difficult to generate meaningful message which would tell what was changed since last commit.

1 Like
25

Maybe this has changed in 5.0, but at least in 4.0.7, KiCad did not always follow this rule:

  • Entire sections of the .pro file would move around when just one setting was changed.

  • If I change a footprint in CvPCB (say I change R2 from R_Axial_DIN0207_L6.3mm_D2.5mm_P7.62mm_Horizontal to R_Axial_DIN0207_L6.3mm_D2.5mm_P2.54mm_Vertical) and then sync the PCB to the new netlist, any changed footprints move to the end of the .kicad_pcb file, rather than R2 maintaining its position within the file.

26

Sorry, I haven’t been enough precise: my experience is restricted to eeschema. It’s unfortunate that CvPCB does not behave similarly.

27

Hi John, can you share that similar script in Fossil with me please? Thank you.

28

@jwhitmore: In the software world how do you resolve a GIT merge conflict? Well with a diff tool and your (IDE) editor of course. What you use to resolve the conflict are the very same tools you use to generate the work (code) i.e., you work within the same confines and context as during normal work (code) generation. Not so with CAD. Though the KiCAD files are text based and therefore easy to put into GIT, I would never in my wildest dreams want to resolve a merge KiCAD schematic/board conflict on a regular basis with my text editor.

Though a GIT submodule is a natural for a code library to pull in and plug into for software, not so for a CAD system. If I pull someones library in through GIT in my code, I just get myself familiar with its API so I can make intelligent calls - the submodule largely stays a black box to me. I would never accept this method for a schematic that goes on a commercial board. I need to know and understand far more about what it is I plunk on the board from that library. I.e., there is an interaction between my schematic and the part I just pulled in - that needs to be quantified and qualified. This is very different from a black box sofware module. So though GIT submodule “sounds like” CAD submodule - they are completely different ideas. Not to say they cannot be combined, but the need/nature is different.

Btw, reference designators can be hugely important in some designs; I have been part of modular designs whereby a significant part of schematic A was the same as schematic B (think of a microcontroller design with different analog interfaces). To make repair and test much easier (manual job), keeping all the components around the microcontroller the same with the same layout and the same reference designators makes a difference.

All that isn’t to say we should/couldn’t have revision control in (Ki)CAD. Just that it isn’t the same as in software. At first face it looks the same/similar but dig a bit deeper and it isn’t.

My 2ct anyway!

1 Like
29

Hi joost, thanks for your response, but a few thoughts…

You mentioned that a Software library is a black box to you, only familiarising yourself with the API calls you need. You’d never work that way with electronic design for a commercial board. That’s great I’m not asking you to change the way you work. I only suggested that information be stored in files in a more flexible manner, so that if people wanted to use a more collaborative process, assisted by version control, they could.

I don’t think that 100% of Kicad users are creating commercial boards, but even if I was, if there was a SMPS hierarchical schematic sheet which I could treat as a black box which exported 3 symbols (Vin, Vout and Gnd) I’d be happy with that black box as well. I have no problem that people might want to spend time rolling their own SMPS. I’ve actually done that, but I now want a more flexible way of including that “module” into a number of other designs. On that note I’m not sure where you draw the line with what a component/library/module is. I’ve used an LM5088 for my SMPS, and it’s got maybe 10 or so components around it. If a company took those components and put them into a package with three pins, (Vin, Vout and Gnd, a bit like a linear 7805) which I could place on my schematic like a 7805, would that be philosophically different?

Maybe it’s my SW background. Every time I create a new project which needs a linked list I don’t write a linked list from scratch. That would be a waste of effort and is not the focus of the new SW project. I’ve written one in the past and I’ll just reuse it. Likewise for my SMPS. I’m happy for people out there who want to rewrite a linked list for every project, as I’m happy for people to redesign a SMPS for every project. Everybody works different. Maybe I’m just lazy, or don’t get the same kick out of designing an SMPS.

And your point about Designators being important. Did I say that they weren’t important? I’m sorry if I gave that impression. I merely suggested that they be moved to a different file. So that rather then contaminating a hierarchical schematic sheet they be stored in a project specific file. I’m not saying that designators have to be different in every design, If you have two related designs and want to harmonise designators I’m not suggesting impacting that possibility.

I think I’m advocating flexibility here not trying to constrain how you work. On that last paragraph on how SW and HW are very different. You don’t have to tell me :wink:

On that note I recently watched an EEVBlog video [1] where Mr. Dave was doing a mailbag video in which he reviewed a USB Oscilloscope which somebody had sent him. Mr. Dave had a bit of a melt down when he saw the Eagle schematics for the product. It’s open hardware but the schematics, according to Mr Dave, were a Dog’s dinner, (bit of a mess) He (Mr. Dave) wondered about that fact and suggested that if this was an Open Source SW project somebody would have cleaned up the schematics in the file and issued a Pull Request back to the project. That does not appear to be the philosophy of Open Hardware. (Well that’s the impression I took from Mr. Dave). Maybe there’s a chicken and egg thingy where the tools don’t lend themselves to that philosophy because Hardware Engineers don’t practice that philosophy, or vice versa. Software tools are designed with collaboration in mind. They don’t have to be used in that manner. I’m sure there are a multitude of SW project which don’t use a collaborative philosophy at all. It’s not compulsory. But even if your doing your own personal SW project in isolation I’d still suggest Version Control be a good thing.

I’m going to try and take time this weekend to look at the code for Eeschema and ponder about how information is stored for a hierarchical schematic sheet. I’ll probably run away again but who knows.

[1] https://youtu.be/2Z3URu9vQBk (It’s at about 42:30)