Hello, I’m the author of kicad-kbplacer and I’m hoping to get some feedback on some plugin related improvements ideas.
testing is challenging - API changes unexpectedly (which I actually don’t mind that much) but infrastructure is lacking. There is official kicad docker image but only for nightly builds. This forces me to maintain my own copy with tags for various versions. This works but requires constant monitoring of PPA for new releases (need to update and push images manually). Having official docker images would be a huge help for plugin developers.
kicad’s official image is certainly a good start but it requires improvements in order to be usable, let’s say that I would like to help - where should I start? How we can move forward with this?
there is no good way for managing plugin dependencies (or is it?) - for now kicad-kbplacer is self contained - it uses only pcbnew, wx and standard python libraries but let’s assume that some third-party python package is required as well.
I can ask user to install ‘plugin backend’ with pip and for PCM package only GUI (I think some plugins handles this situation that way) but this introduces another set of issues. On windows it should be fine since kicad uses own python executable, but on linux it uses distribution one. I think that using pip globally is discouraged. For example on arch linux I would get:
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try 'pacman -S
python-xyz',
there can be dependencies conflicts if installing globally
I’m thinking that it would be best if each plugin would run in isolated environment (some equivalent of python -m venv .env && source .env/bin/activate) - I did not investigate this yet and I don’t know if this need to be supported by KiCad or plugin can actually manage this by it’s own (I know that some python tools can create isolated environment, for example python -m build)
@marekr should be able to give you directions on how to help improve official docker images.
Regarding plugin dependency management, this is a known issue. So far the recommendation is to either bundle the dependency with the plugin in a submodule (works if it’s a source only lib without compiled c extensions), ask users to run pip command or just avoid using the dependency.
First method is clean but doesn’t work for every lib. Second method runs into the issue of potential lib conflicts because all plugins live in the same environment, isolating them is hard because they all are in the same kicad.exe process. Not using the dependency is sometimes easiest because reimplementing a bit of a large lib that you are actually using can be done in some cases but obviously not always feasible.
This problem should be solved in future implementation of plugin architecture which will use separate processes.
You can achieve this now to a limited extent by having your plugin be just a launcher of external process. But then you don’t have same API possibilities as plugin running inside pcbnew and also you have to provide your users platform specific instructions on how to setup the external part of your plugin which complicates the install process.
I was stuck between putting out fires and “how-to sneak in docker images into the main code repo when we normally prohibit platform/release configs from the main repo”
Easier release process by just having the tag on the main repo trigger the build of a stable release, maximum laziness because we consistently forget things :3
In a perfect world security is not needed because there are no malicious actors and build systems don’t have to be isolated and passwords are not a thing and it’s perfectly fine to have one pc be able to write to dozens of other completely different systems with no issues.
We don’t live in a perfect world so there are barriers for automating everything and they are there for good reasons. Yes, things should be automated when possible, but checklists are for those other cases.
Meh, there are different kinds of checklists. Unit testing is an quite elaborate way of an automated checklist, but it can’t check everything and some hard manual labor will remain for the foreseeable future to.
first method should work for my case. I wanted to add yaml support and PyYAML fortunately provides pure python version (bindings to compiled libs are optional).
so I could have pcbnew part of the plugin (without external dependencies) separated and it could spawn processes in isolated environments (with dependencies). I could even create these environments automatically with venv. Although possible this would be overcomplicated given my current needs (and there is still the issue of inter-process communication if needed)
btw, I’ve been working on some improvements for my plugin packaging (mostly for my own amusement) and this hatch plugin GitHub - adamws/hatch-kicad: Hatch plugin to build KiCad addon packages was born out of it. It simplifies process of creating zip archives and metadata json.
It does not yet handle things like dependencies packaging but even now it should be usable in most of the cases.
Checklists save lives.