I guess my problem is that I’m coming from a Software background and source text files are much easier to handle where a human is the only thing making changes to the text files. Kicad on the other hand is making changes to the text files, in this case the .sch file.
I should say what I’m trying to do. I’ve designed schematic for a SMPS based on the LM5088 buck converter chip. I wanted to include that 12V (Automotive) to 5V step down converter into a number of designs and keep it under version control. The reason for version control is so that if I update or make changes in one project that is using the SMPS I can quickly pull those changes into any other projects using that “part”. I created a git repo with just the LM5088.sch file and no other files, and used that as a “git submodule” in the project I want to add the SMPS into.
So far so good but as soon as I create a sheet in a project and add the file LM5088/LM5088.sch as the schematic file the file, which is under version control, has numerous changes made to it. Most of the changes are identification numbers of the various components. So all the resistors are stripped back to R? rather then R1 etc.
Those changes aren’t the kind of information which should be under version control. The resistor values should be under version control, so if I find that a value needs tweaking for the design that can quickly be distributed to other projects using the SMPS. If one project calls a resistor R201, whilst another project calls the same component R305 that’s not something for version control. However if the value of the resistor is changed then that’s probably something that should be distributed.
Perhaps I’m doing this wrong, but at present I’m wondering should the .sch file be split into the core design and the project specific information. There’s probably an easier way I’m missing.
KiCad is agnostic to the existence of a version control, same way as a version control system is agnostic ( should be ) to a particular application.
I am not sure if I fully understood the issue.
On my preference, if I got a file that is versioned control ( eg a .sch ) anything that changes on that file I would like to record it.
I see it too important to track a change of component reference numeration on a schematic.
I’d highly recommend that you look into git’s smudge/clean filters. Many folks here would disagree with your assessment of the usefulness of version control over the designators but you can always make your own smudge filter to remove the aspects with which you disagree and only track those that make sense to you.
I don’t think KiCad needs version control integration, but I think it could be “friendlier” towards version control. Specifically, by not introducing unnecessary diffs. Sections in the .pro file seem to move around at random, and the power symbols in the .sch are constantly getting renumbered for no apparent reason, etc.
It just seems like version control wasn’t on the minds of the KiCad developers.
Full integration is not needed, but I certainly agree it would be nice to streamline for VCS.
I’d like to see the python plugin process continue for the project management window, and then it would allow plugin usage rather than trying to shoehorn in any single particular versioning system.
Using git submodules is already considered a bad idea for plain source code files, so I don’t see how it could be a better idea for schematic files which are more closely integrated together. Moreover, .sch files are not autonomous, you need a lib file for the components. Adding lib files for each sub sheet would really mess up the hierarchy of your project files. Your approach with submodules is not composable: imagine that you would want to use two sheets from two submodules where the identifiers are conflicting, how would you solve this?
As for the identifiers of components, you really want them to be part of your schematics and not be able to modify them in a third party project. The subsheet is not like a library presenting a crisp interface, it is a plain part of the schematic that is required for the downstream manual tasks: BOM, routing…
The closest I could find to the approach you are considering is the use of HDL (hardware description language), but I am not aware of any tool dedicated to PCB design.
One major point is that kicad stores the annotation (references) of a hirarchical subsheet inside the subsheet file not its parrent. This means even without version control you already run into problems when sharing the same file (example via file links) between different projects. The problem happens if you change the annotation of any of the projects where you use the file.
If you look closely at the subsheet file you will notice that for multiple instantiated files you get a line added for every time you instantiate the file. This line will hold your identifiers. If you get kicad to see adding such an external file as instantiating it one more time then you might be able to use it like you want. But this is outside of the feature set envisioned by the devs and therefore untested.
Right now kicad simply is not build to support independent modules. I consider this a useful feature independent of version control. I think such a feature is planned for kicad. But it is one of the long therm goals not something that will be implemented anytime soon.
I’m not sure what integration with version control would mean but it seems to me that the difficult piece with making KiCad work with version control is there is nothing that’s the equivalent of diff. How do you merge two files together that have both diverged from a common ancestor?
If KiCad could somehow read two schematic files (or two circuit board files) and display where they differ and offer you a way to select one or the other for the output, I think that’d go a long ways to helping this problem. Sounds like a big job though.
Some idea: It would be nice to implement something in / with KiCad that renders all schematic files to PNGs files and keeps a current and previous version. People could keep it in the version control (ok I know they are binary) but it could help to visualize the changes between versions.
I’m sure there are people who don’t like git submodule, but you make it sound universal? In the 21st century it’s practically impossible to write useful Software which does not rely on other people’s code. Even the C Programming Hello World relies on the C Standard library. You can implement your own printf() but why?
Thank you all for your responses. I get the feeling that electronic design is not a collaborative or cooperative process. At present silicone companies create a device and in a datasheet they draw a picture of a bog standard schematic for the device. I then have to read that datasheet and manually copy that schematic design into a CAD package, in this case Kicad, and possibly tweak it to my requirements. Apart from the tweaking all the rest of this ‘work’ is being repeated numerous times. I’m surprised the boilerplate stuff ain’t automated.
This all started with my switching a SMPS to the LM5088. I’m not expecting the company to create a schematic file for every CAD package out there. People are already doing that but it’s work that currently doesn’t appear be shared easily. To me it’s a logical extension of creating a symbol, or footprint, in the kicad libraries. I could add the LM5088 to my own private library but that’s not helping anybody. Adding the symbol to the official kicad symbol library and perhaps it helps somebody out, and they don’t have to repeat boilerplate tasks. (Hippy )
I should point out that I never suggested which version control system should be used, I just said that I tried this with git. And my subject / title was really bad in using the word integration. That’s the wrong word and I should not have used it, sorry. I should have said that Kicad could perhaps “facilitate” version control. At present I can’t open a design and close it without “saving changes”.
Somebody suggested that a schematic file is useless without a library file. I’d imagine that if I could share a kicad schematic it’d be best if all symbols were from the standard kicad symbol library.
Thanks for pointing out git smudge, that might be the way forward. I must look at python in Kicad as that might be the way forward as well.
I may have looked like a bit aggressive in my previous reply. Let me clarify my standpoint.
git submodules are useful and useable for an extremely small set of use cases (mostly, independent external libraries). For most others, it can be considered harmful (messing with the included submodules can be tricky). I would recommend using git subtree for most of your needs, this brings saner repo setups.
Anyway, being part software, part hardware designer, I can tell you that hardware design can be as collaborative and cooperative as software. The main difference lies in the artifacts generated from these activities. The development cycle of software is more iterative and can be split in cleaner subparts, without taking care of the cost of deadcode, dangling features, mainly because you don’t have economical incentive to do otherwise.
In hardware, you usually want your final product, the electronic board, to be as tightly integrated, focused and price aggressive as possible. That’s more a team work on a sprint (tasks can be either functional, design, routing, procurement). What you can share between designs is a working setup, some ideas of hierarchy and a set of proven references but all this stuff is not easily formalized.
Working together on a single schematic is possible and even encouraged. The same rule applies as for software: if several collaborators are working on the same part of the schematic, you can have conflicts when trying to merge. Otherwise, Kicad is quite friendly to the git workflow, when you use the clean/smudge feature. You have the same issues with some code editors that mix up pure build recipes with IDE setup in the same files.
Each Kicad schematic project embeds a cache library file of all the components used, so just adding this cache file to your git repo ensures that your commits are all complete and self-contained.
The main difficulty is that storing old versions is the easy part, you can already do that.
What is hard is finding a minimal set of differences between two versions (so changes can be reviewed in isolation) and merging diverged states.
Text files are a one-dimensional stream of characters with newlines in between them, and tools barely keep up with these. If you swap the order of two functions in a file, you get a huge diff, even if there are no functional changes, reviewing that is annoying and you can almost forget about automatic merges. Most projects have adopted a coding style to keep pure formatting changes to a minimum to avoid conflicts.
With schematics or PCBs, we have a two-dimensional layout, where functional and aesthetic choices are interwoven even more than in text files with program code. There is no “layout style” to be followed that can be automatically verified to reduce conflicts.
If we had a good proposal how to solve diff and merge, the implementation of that should be straightforward.
This is the minimal way I have found to state the difference between pbc board design and program source code editing.
In programming one change in the editor is one identical change in the file. In pcb design one change in the editor is a random change or many changes in the file. Therefore in the latter merging becomes very difficult or impossible.
I don’t think we should expect a full VC integration, but there are some principles that could lead to better interaction. Two of them are the principles of minimal change and of locality.
minimal changes implies that changing the place, the shape or a property of an object must not change the place of the description of the object in the file. This way, a change to an entity appears as a unique diff hunk in the file, not like a deleted block somewhere and a new one somewhere else. Kicad already follows this rule.
locality means that the new lines matching a group of additions to a schematic should be relatively close to the lines of the parts close to the changes in the schematic (and close to each other). This one is more blurry, but it should allow to easily merge files where changes were performed in different locations of the schematic by not making them appear at the same place in the file. This is not bullet proof, but at least, it would be better than today’s append strategy.
I know it’s easy talk and I’m pretty sure it could be done offline by finding new primitives in the schematics and moving them to a “more sensible” location.
)