Using git submodules is already considered a bad idea for plain source code files, so I don’t see how it could be a better idea for schematic files which are more closely integrated together. Moreover, .sch files are not autonomous, you need a lib file for the components. Adding lib files for each sub sheet would really mess up the hierarchy of your project files. Your approach with submodules is not composable: imagine that you would want to use two sheets from two submodules where the identifiers are conflicting, how would you solve this?
As for the identifiers of components, you really want them to be part of your schematics and not be able to modify them in a third party project. The subsheet is not like a library presenting a crisp interface, it is a plain part of the schematic that is required for the downstream manual tasks: BOM, routing…
The closest I could find to the approach you are considering is the use of HDL (hardware description language), but I am not aware of any tool dedicated to PCB design.
One major point is that kicad stores the annotation (references) of a hirarchical subsheet inside the subsheet file not its parrent. This means even without version control you already run into problems when sharing the same file (example via file links) between different projects. The problem happens if you change the annotation of any of the projects where you use the file.
If you look closely at the subsheet file you will notice that for multiple instantiated files you get a line added for every time you instantiate the file. This line will hold your identifiers. If you get kicad to see adding such an external file as instantiating it one more time then you might be able to use it like you want. But this is outside of the feature set envisioned by the devs and therefore untested.
Right now kicad simply is not build to support independent modules. I consider this a useful feature independent of version control. I think such a feature is planned for kicad. But it is one of the long therm goals not something that will be implemented anytime soon.
I’m not sure what integration with version control would mean but it seems to me that the difficult piece with making KiCad work with version control is there is nothing that’s the equivalent of diff. How do you merge two files together that have both diverged from a common ancestor?
If KiCad could somehow read two schematic files (or two circuit board files) and display where they differ and offer you a way to select one or the other for the output, I think that’d go a long ways to helping this problem. Sounds like a big job though.
Some idea: It would be nice to implement something in / with KiCad that renders all schematic files to PNGs files and keeps a current and previous version. People could keep it in the version control (ok I know they are binary) but it could help to visualize the changes between versions.
I’m sure there are people who don’t like git submodule, but you make it sound universal? In the 21st century it’s practically impossible to write useful Software which does not rely on other people’s code. Even the C Programming Hello World relies on the C Standard library. You can implement your own printf() but why?
Thank you all for your responses. I get the feeling that electronic design is not a collaborative or cooperative process. At present silicone companies create a device and in a datasheet they draw a picture of a bog standard schematic for the device. I then have to read that datasheet and manually copy that schematic design into a CAD package, in this case Kicad, and possibly tweak it to my requirements. Apart from the tweaking all the rest of this ‘work’ is being repeated numerous times. I’m surprised the boilerplate stuff ain’t automated.
This all started with my switching a SMPS to the LM5088. I’m not expecting the company to create a schematic file for every CAD package out there. People are already doing that but it’s work that currently doesn’t appear be shared easily. To me it’s a logical extension of creating a symbol, or footprint, in the kicad libraries. I could add the LM5088 to my own private library but that’s not helping anybody. Adding the symbol to the official kicad symbol library and perhaps it helps somebody out, and they don’t have to repeat boilerplate tasks. (Hippy )
I should point out that I never suggested which version control system should be used, I just said that I tried this with git. And my subject / title was really bad in using the word integration. That’s the wrong word and I should not have used it, sorry. I should have said that Kicad could perhaps “facilitate” version control. At present I can’t open a design and close it without “saving changes”.
Somebody suggested that a schematic file is useless without a library file. I’d imagine that if I could share a kicad schematic it’d be best if all symbols were from the standard kicad symbol library.
Thanks for pointing out git smudge, that might be the way forward. I must look at python in Kicad as that might be the way forward as well.
I may have looked like a bit aggressive in my previous reply. Let me clarify my standpoint.
git submodules are useful and useable for an extremely small set of use cases (mostly, independent external libraries). For most others, it can be considered harmful (messing with the included submodules can be tricky). I would recommend using git subtree for most of your needs, this brings saner repo setups.
Anyway, being part software, part hardware designer, I can tell you that hardware design can be as collaborative and cooperative as software. The main difference lies in the artifacts generated from these activities. The development cycle of software is more iterative and can be split in cleaner subparts, without taking care of the cost of deadcode, dangling features, mainly because you don’t have economical incentive to do otherwise.
In hardware, you usually want your final product, the electronic board, to be as tightly integrated, focused and price aggressive as possible. That’s more a team work on a sprint (tasks can be either functional, design, routing, procurement). What you can share between designs is a working setup, some ideas of hierarchy and a set of proven references but all this stuff is not easily formalized.
Working together on a single schematic is possible and even encouraged. The same rule applies as for software: if several collaborators are working on the same part of the schematic, you can have conflicts when trying to merge. Otherwise, Kicad is quite friendly to the git workflow, when you use the clean/smudge feature. You have the same issues with some code editors that mix up pure build recipes with IDE setup in the same files.
Each Kicad schematic project embeds a cache library file of all the components used, so just adding this cache file to your git repo ensures that your commits are all complete and self-contained.
The main difficulty is that storing old versions is the easy part, you can already do that.
What is hard is finding a minimal set of differences between two versions (so changes can be reviewed in isolation) and merging diverged states.
Text files are a one-dimensional stream of characters with newlines in between them, and tools barely keep up with these. If you swap the order of two functions in a file, you get a huge diff, even if there are no functional changes, reviewing that is annoying and you can almost forget about automatic merges. Most projects have adopted a coding style to keep pure formatting changes to a minimum to avoid conflicts.
With schematics or PCBs, we have a two-dimensional layout, where functional and aesthetic choices are interwoven even more than in text files with program code. There is no “layout style” to be followed that can be automatically verified to reduce conflicts.
If we had a good proposal how to solve diff and merge, the implementation of that should be straightforward.
This is the minimal way I have found to state the difference between pbc board design and program source code editing.
In programming one change in the editor is one identical change in the file. In pcb design one change in the editor is a random change or many changes in the file. Therefore in the latter merging becomes very difficult or impossible.
I don’t think we should expect a full VC integration, but there are some principles that could lead to better interaction. Two of them are the principles of minimal change and of locality.
minimal changes implies that changing the place, the shape or a property of an object must not change the place of the description of the object in the file. This way, a change to an entity appears as a unique diff hunk in the file, not like a deleted block somewhere and a new one somewhere else. Kicad already follows this rule.
locality means that the new lines matching a group of additions to a schematic should be relatively close to the lines of the parts close to the changes in the schematic (and close to each other). This one is more blurry, but it should allow to easily merge files where changes were performed in different locations of the schematic by not making them appear at the same place in the file. This is not bullet proof, but at least, it would be better than today’s append strategy.
I know it’s easy talk and I’m pretty sure it could be done offline by finding new primitives in the schematics and moving them to a “more sensible” location.
I didn’t take it as at all aggressive, and sorry if my response made you consider that possibility. I was just very surprised that git’s submodules was frowned upon as it’s the only mechanism I’ve ever seen used for managing code dependencies which are in another git repo. I don’t claim to be an expert at all and wondered what the alternatives are, as I’ve not encountered them. I’ll have to look at subtree.
I agree with that statement, but I can’t image that 100% of Kicad’s users are focused on that set of requirements. I’d assume that’s the high end Hardware design shop and I think probably a significant percentage of Kicad users don’t design PCB’s for blue chip HW venders.
I must try to both find and follow an open Hardware development project as I can’t see how the tools are available for a smooth workflow. That’s my perception with very limited knowledge, or imagination. Perhaps my initial message documented a problem which doesn’t exist, but given the effort that people seem to be making to get around the non existent problem, writing bash or Python scripts to do image diffs, or using git smudge there seems to be an issue.
I’ve obviously got little idea of the internals of Kicad, and perhaps I’ll make some time and start looking at the code, but it seems that Kicad contains all the information that it needs to specify a schematic. (I’m going to concentrate on just the schematic for the moment.) So Kicad processes and saves all the data which is needed for the schematic I just wonder could that data be redistributed in files so that some files contain the essence of the design but other files contain implementation details, which change on a regular basis and are project specific. So I hit a snag with component references, perhaps they are project specific not part of the core design. So in my example two project share a LM5088 SMPS, the circuits are exactly the same but component references are specific to the individual projects, not the circuit design.
Seeing as I have been working on an open project keeping the kicad files in step with the eagle files, I feel I have a good sense of exactly how you would need version control to work for a veiwer for minimal complexity creating,
Netlist differences between the Board, Summerised to different footprints, different nets on pads, and different values,
Different zone polygon outlines and properties, ignore the fill data and just look for the shape and settings,
This would cover most increments giving the most surmised form of changes on a board, yes there is much more that can change, but trace data would make it much harder to merge, keep track of, etc,
I second that. I fullty agree that some form of version control would be very nice, but it does not need to be “integrated” into kicad, whatever people think that means.
m
I have been dreaming of light integration so that with some key combination KiCad would do git commit with automatic message. It would be nice if the message would be human readable, but it would be difficult to generate meaningful message which would tell what was changed since last commit.
Maybe this has changed in 5.0, but at least in 4.0.7, KiCad did not always follow this rule:
Entire sections of the .pro file would move around when just one setting was changed.
If I change a footprint in CvPCB (say I change R2 from R_Axial_DIN0207_L6.3mm_D2.5mm_P7.62mm_Horizontal to R_Axial_DIN0207_L6.3mm_D2.5mm_P2.54mm_Vertical) and then sync the PCB to the new netlist, any changed footprints move to the end of the .kicad_pcb file, rather than R2 maintaining its position within the file.
)