How to clone a project

[kicker]

Hi,

I have an existing project ( prototype ) and want to fork a new version. There does not seem to be any means of simply doing a Save_As at project level to dupe everything under a new name.

Why not ?

The only way I can see is to do all this by hand ; copying and renaming any and all files which seem related to a project.

What am I missing? Is there some reason that this functionality is not included? It seems an obvious enough scenario and just about any other kind of software provides this kind of thing.

TIA

[davidsrsb]

Being able to easily make save points and revision copies is a feature many of us would like

[kicker]

I’ve tried the copy/rename trick and most of my components are now showing up with questiion marks. This needs a clean , reliable function provided in the program, not hacking around under table then trying to fix the breakage.

[devbisme]

You probably don’t want to hear this (most people don’t), but git easily does this for you. (Note that I’m talking about git, not Github.)

1. In your kicad project directory, just do git init. This creates a repository that stores all your project versions.

2. Add the project files you want to keep track of. For kicad, there’s really only three you need: git add myproj.pro myproj.sch myproj.kicad_pcb.

3. Store the current state of your project: git commit -a -m "Some message about this version".

That’s all you need to do to setup versioning. Then whenever you have a version you want to save, run the command git commit -a -m "A message about this version". If you also want to give the snapshot an easy-to-remember name, you can use the command git tag my_version_tag. Then if you ever want to return to that version, use the command git checkout my_version_tag. And if you want to see the history of your project, use git log.

I’ve seen plenty of tools that provide some form of “archiving”. They’re pitiful when compared to using any modern versioning tool like git or Mercurial. If the tool builders were smart, they’d just provide a button you could press to execute the commands I showed above.

[kicker]

Thanks, that’s a useful tip. As a quick kludge to ensure backups, I’ve just been tarring the whole directory to be on the safe side. The git solution is a lot better.

However, one needs to know what needs archiving and the list you gave is not sufficient since there are no library files. The problem I pointed out above is due to kicad missing some library info. and not reproducing the supposedly cloned project.

Also I have two versions that I need instant access to, this is not for historical purposes and I don’t want to be saving, popping git commands and coming back, opening files etc.

I have a proto that I need access to ( it’s a working example ) , I’m now doing v2 off that proto and I need it to be as simple as : file open_recent to flip back and forth.

So I still need a complete list of relevant files to ensure that git will produce a working backup and not restore my project in a broken state.

And I still need a means to reliably clone a project rather than just archiving it for posterity.

But thanks, that’s very useful tip. I will start archiving like that as soon as I know what is needed to fully restore a project.

[PCB_Wiz]

@kicker

I have an existing project ( prototype ) and want to fork a new version. There does not seem to be any means of simply doing a Save_As at project level to dupe everything under a new name.

Why not ?

Good question.
Maybe the multiple-files aspect, bumped this into the too-hard basket ?

The Project wrapper already ‘protects users’, and if you start as stand-alone you get more menu choices
… in eeschema you can 'Save Current Sheet as"
… in PcbNew you can 'Save as" and “Append Board”

So a Save Project As , at the Project Level certainly ‘completes the set’, it just needs multiple files.
There is
New Project
New Project from Template

so maybe add a
New Project
New Project from Template
New Project from Existing

[cbernardo]

You probably missed some of the local cache files. I’m not even happy with the pcbnew “save as” function; it doesn’t seem to be of any use at all. Maybe the kicad UI needs a “Copy and Save As” item to make a copy of all the relevant files.

[PCB_Wiz]

@cbernardo

I’m not even happy with the pcbnew “save as” function; it doesn’t seem to be of any use at all. Maybe the kicad UI needs a “Copy and Save As” item to make a copy of all the relevant files.

Can you clarify the issues with PcbNew Save-As ? It seems to work ok ?
Do you mean it is somehow incomplete & needs more files ?

[cbernardo]

The behavior is awful - it will simply make a copy of the pcb file and continue working with that copy. How is that useful to anyone? If you want to fork a design or work on a new design to be merged back into the old design then I agree with devbisme that git is the tool to use. At least with git you know that if you reference 3D models in the project directory then the models will still be there; if you use the pcbnew “Save As” you will need to go into a command shell and copy other files.

[PCB_Wiz]

@cbernardo

The behavior is awful - it will simply make a copy of the pcb file and continue working with that copy. How is that useful to anyone?

Well, yes, I find that useful, as that is exactly what I expect single-file save-as to do.

@cbernardo

If you want to fork a design or work on a new design to be merged back into the old design then I agree with devbisme that git is the tool to use. At least with git you know that if you reference 3D models in the project directory then the models will still be there; if you use the pcbnew “Save As” you will need to go into a command shell and copy other files.

If there could be another variant of save-as, that would include 3D models, then yes, that also would be useful to have.

[kicker]

There is also the question of .pretty files .mod files and probably more I don’t know about.

The program MUST know what files it is referencing so why does the user have to start digging around into the mechanics of how kicad works to second guess what it is using and try clone them?

There does not seem to be anyone here even who has a definitive answer as to what I need to back up to be able to rebuild a project from scratch.

Another aspect of the same questions is : what if I want to transfer a project to another machine. What do I need to copy over?

Either Kicad should be able to dump a list of all related files or , better, have some tarballing facility which wraps stuff in a single file back up or copy.

[devbisme]

I did an experiment to see the minimum set of files I needed to rebuild a design. Here they are:

• myproj.pro : The project file that ties everything together.
• myproj.sch : The Schematic file (or files in the case of a hierarchical design).
• myproj.kicad_pcb : The PCB layout. It also contains the footprint for every component used in the PCB layout.
• myproj-cache.lib : This contains the schematic symbol for every component used in the schematic file. It gets generated whenever you save the schematic or project.

So all you need are those four files. If the shared libraries with the component footprints and/or schematic symbols are missing, then KiCad seems to get the information it needs from these files. These files may also be given priority even if the shared libraries are present; I don’t know.

You also mentioned you needed instant access to two versions of your design. That’s easy to do with git: just clone the repository into another directory and checkout the version you want. This works with two, three or any number of versions.

[davidsrsb]

It gets messy with custom local libraries and shared libraries. Do you want to fork them? Not an easy one for the software to guess

[kicker]

Well if it can’t be guessed because there is not a unque answer, there will be a save_as dlg which allows the user to chose whether the libs get cloned or used as a common resource.

That is not a reason not have such a feature.

[kicker]

@devbisme

I did an experiment to see the minimum set of files I needed to rebuild a design. Here they are:

Thanks, but you’re a testing a trivial, subjective case. What happens if you modify one of the footprints or have to build you own component because it does not exist in the std distributed libs?

I very quickly found I needed to do that and this leads to yet more files and types of file.

What happens if you save to current library vs save to a new library?

As I said above, there is a need for at least display of text file output of everything that kicad is referencing in the whole project.

It seems that kicad is a suite of programs which have evolved fairly independently and the ‘project’ interface is a bit of an after-thought that has not been really integrated.

If kicad is going through a an overhaul, this aspect probably needs redesigning in a top down manner rather than the current rather unstructured bottom up stitching together of various bits.

[cbernardo]

Having a facility to dump all needed files into a directory (for sharing/collaboration etc) has been on the wish list for a long time, but all devs have other priorities and the task is not as simple as we would hope.

[davidsrsb]

But having a simple list of the files used would be very useful

[devbisme]

Sorry, I thought what you wanted was the minimum set of files needed to regenerate a project. The files I listed are those files.

If you need to modify a schematic symbol or footprint, both EESCHEMA and PCBNEW let you load those into the symbol or footprint editors, modify them and save them back into the cache.lib and .kicad_pcb files, respectively. And the same thing applies if you want to generate your own schematic symbols or footprints from scratch. There isn’t a need to keep the shared directories of schematic libraries and footprint libraries.

[kcuzner]

I’ve been using git for version control for my kicad designs for the past 2 years or so. The newer system (not relying on a the cmp files, the “pretty” libraries, etc) works much better for version control in my opinion.

Just wanted to add my 2c, I almost always end up having to use my own symbols and footprints at least once in every project. Many projects end up sharing parts that I had to enter a custom footprint or symbol for. Originally, I tried to create my own shared library that hung around in my projects folder, but this made things complicated when I wanted to post a project on github or move the project to a different machine. Obviously the caches make it work at first, but modifying footprints or symbols starts to get hairy after a while (especially footprints).

What I’ve ended up doing is having every project have its own symbols and footprints library in the repository itself. No project depends on any others around it. This has made it much easier to manage everything with git. Forking footprints is easy: I just copy and paste the required kicad_mod file into the forked project. Schematic symbols aren’t quite as smooth since the import component button will only import the first symbol in the library (it seems…I may just be using it wrong), requiring the corresponding export button to be used in the project the symbol is being forked from.

[cbernardo]

Well, if you want a definitive answer, the files you need are:

a. PROJECT.pro : this may require some modifications to the list of libraries used; in particular any symbols used should be copied to a local library and that library must be specified in the .pro file. All other libraries may be ignored since any relevant symbols from them should be copied to the local lib - naturally if the local lib already exists it shouldn’t overwrite itself. If the PROJECT-cache.lib is up-to-date then the new local lib may simply be a copy of this.

b. PROJECT.sch : this will require modifications to point to the newly created local lib rather than any other libs which may have been used during design. The exception is the PROJECT-cache.lib, which should be updated before copy operations proceed. If PROJECT.sch is hierarchical then the referenced files must also be copied and modified to use the local lib.

c. MY_LOCAL_SCHEMA_LIB.lib : well, whatever name you want really - this should be the schematic lib to be shared. At the moment it is a single file; as current work on eeschema progresses we expect this to become a directory

d. PROJECT.dcm : this only needs to be copied/modified if it’s actually used, otherwise it can be ignored.

e. PROJECT.kicad_pcb : this must be copied with all MODULE instances modified to (1) reference footprint files in a local directory and (2) reference any 3D models in a local directory. This starts to get tricky because footprint files can’t be blindly copied to a single flat directory - for example there may be 2 different components with the same file name but different absolute paths: ${PATH_A}/GENERIC.kicad_mod and ${PATH_B}/GENERIC.kicad_mod. The same goes for any 3D model files which are referenced - they cannot necessarily be simply copied to a flat directory structure.

f. MY_LOCAL_FOOTPRINTS : a directory needs to be created to hold all referenced footprint data

g. MY_LOCAL_3D_MODELS : a directory needs to be created to hold all referenced 3D models

h. fp-lib-table : rather than blindly copy this file, a new file must be created to refer to the local footprint directory

That’s all I can think of for now. There is no point in copying the NET file since that could have any arbitrary name and is easily regenerated anyway. Other ancillary files like manufacturers’ documentation for parts etc. would need to be handled separately.

It’s no small task to do this in software, especially not in KiCad with the way the code is currently structured. It can be done though and is a feature I imagine many people would find useful.