at FOSDEM I discussed with Wayne the idea about an official high level python API for KiCad. From his perspecitive it would be okay as long as it does not requires much attention by the KiCad devs to support it.
As reference, this is the current documentation of the low level api: http://docs.kicad.org/doxygen-python. This is a 1:1 export of the c++ classes and methodes, which means also including all namechanges,…
My proposal would be building a mostly thin abstraction layer over the low level api to provide:
a stable interface for plugins
extensive documentation including example codes (build with sphinx)
workaround/hacks to fix bugs (like you cannot check if two SWIG objects pointing to the same object)
pythonic and intuitive interface (properties instead of getter/setter, self explanatory naming, …)
unittests to detect breaking changes immediatly (mostly smoketests, which can be embedded in the doc itself)
In a first step I would like to discuss the idea itself, as well as how the interface to the internals of KiCad should look like. I made some initial coding to test ideas and get the system up and running. I don’t plan to do a full abstraction of the low level api. It would be already very cool to support the most common usecases.
That’s a good idea, I am wondering how it will work? For example, if I look at your repo, I can’t tell what version of KiCad it is targeted at.
The problem with an extra interface layer is that it becomes another thing to get out of sync, and also is a further level of confusion when it comes to debugging.
At the moment I target the current nigtly. In the future the idea would be that either KiCad distributes the current version of the library, or the library is written in such way to support different API versions.
To handle out of sync, unittesting would be in place. In fact, every code snippet in the documentation is executed and checked if something differs to the expected behaviour. This should in theory catch most of those problems immediately.
KiCommad looks cool, but is its own scripting language. I want a clean api with documentation for normal python scripts. Nevertheless, KiCommand could be used to identify high level functions which should be supported in this API as well-
I would like to say something because I feel this is a good and important thing, but I don’t have anything specific to say. I don’t have enough experience with the C++ API or python API. In general I would like to see first class python support and at the moment the support isn’t in a good condition. There are too many small and big problems to even know where to begin with. A clean stable API would be very welcome, as long as it doesn’t mean restricting the python developer’s possibilities.
Maybe the C++ API wrapper should be as thin as possible WRT board (and later schematic) manipulation. When possible extra higher level functionality - like what’s in KiCommand - is identified, it could be put to another library which is also maintained and distributed with KiCad. Only when something can’t be done with using with lower level API it could be put to lower level. Otherwise it could be added as a higher level function or class with pure python implementation.
@eelik: the library is designed in a way to always have the possibility to enclose a low level API datatype into the abstraction layer, and get the low level datatype out of the high level one.
@maui: thanks, I didn’t checked if some forks of the old repo made progress. My approach does not differ much to the already existing ones in case of the public API, which also means I can copy quite some parts of the code (thanks to the GPL licence). But in comparison to the already existing approaches documentation and testing is the key priority.
I agree that the fluidity of pcbnew python interface is at least a problem. But it is documented and since I don’t have much experience with other APIs I don’t find it really limiting. Though the fluidity of the API forced me to build some basic unit test.
I am sceptical about proposed approach and I have the same hesitations as bobc. Additionally:
If the interface layer is not shipped with Kicad, it will not help much for the plugin developers. Some users have trouble even installing a just a plugin. Requiring that they also install dependencies is a no-go.
This would be a long term commitment preferably by more than one developer. I assume that your intention is to gather interest and I can only encourage people to join as my skills are not really up to the task.
I’d take things slowly. When V5 is released, I assume the python interface will be a bit more stable. Then the interface layer can be developed and included and shipped with V6.
There is also a different approach (regarding plugins at leat) proposed by eelik, which is more to my liking. With a bit of polish a GUI can be built easing the installation of plugins. Moreover the team supervising would be in the position to enforce certain standards for the plugin if it wants to be included in the plugin list. A presence of unit tests would be one. Thus you would have a system which would more or less mirror the library system:
You would have master plugin repo which would contain links to all the plugin repos contributed.
There would be some requirements for plugin to implement (similar to KLC)
There would be a team managing all this
Any way the development proceeds I think the main thing to keep in mind is to go slowly. There is no a lot of developer power available as most of us do this in our free time.
I think it would work if the interface was installed with KiCad, as the current SWIG based one is. Otherwise trying to work out which version a user needs to download and install will make it unusable except by expert users.
I think managing available plugins is a separate issue. Handling dynamic content is something KiCad has just not got to grips with yet. We’ve had various piecemeal methods (currently 3 different methods I think), and the github method for footprints as default is being rolled back for next release.
Hmm, that could be a problem. Curated content is nice, but finding curators is difficult. I would prefer a scheme that can operate with minimal input in the first instance. If a plugin is faulty, contact the author or uninstall it. The Arduino plugin framework works like that, Arduino maintain a list of third party content, but they don’t vet or approve content.
I take it as self-evident fact that the pointhi’s proposal as an “official” api is tied to KiCad so much that it’s always up-to-date (maybe except between some nightly builds by accident) and part of KiCad installation.
And about plugin management - I now have spent couple of nights with a gui for a plugin manager. It’s not ready or even committed yet to the repo, but general level discussion about plugin management (and development) is very welcome. I suggest we continue it in the old thread, One script to rule them all, to keep this one for the API discussion.
I added some additional properties for testing purpose, and improved the string representations of objects.
I also managed to have a working equal operator, which is pretty cool I think (Although it required some really dirty hack). In the original API that’s not possible because every GetModules for example returns a new SWIG object.
For now, this means two additional improvements I prototyped for this high level API:
Useful string representation of python objects (to actually understand what a object represents)
Working equal operator to do actual comparison of objects.
Is this API only for pcbnew? Are there any plans to also support eeschema? Is it only for drawing? I think plot function is in demand (batch generation of board images to be used for textures, printing for PCB production at home, per layer image generation for graphical diff for easier version control). Also would be nice to have access to conversion functions kicad_pcb to Gerber and Excellon, kicad_pcb or sch to SVG, KiCad project to BOM file, kicad_pcb to a 3D model…
This proposed high level API is based on the existing lower level API. At the moment (in 4.0.7 and in upcoming 5.0) there’s no python API for eeschema. In the original post there’s a link to the low level api, there you can see what’s possible. New things can be added based on needs, but it requires some work and interest from the core developers.
@tpimh
for the moment it’s drawing, and only a subset of that stuff. But I’m also interested in supporting plotting, exports, tool support,… But before this can be added in the abstraction layer It’s required to think about how this tools should be used by the users in the and, and how the low level API handles those things.
I also added the first real abstractions into the API:
In KiCad, all drawsegments (lines, arcs,…) are defined using a single class. My API on the other hand exports them in their own classes which allows us to limit the properties to those which are of interest for us, and having nice _repr_ implementations.
I also defined the public interface of (kicad.pcbnew.Circle) in a way that it has a center and a radius/diameter as it’s normally expected from a circle . Not a center and an end point which KiCad uses in it’s internal definition. The abstraction layer handles the conversations between those different representation. I’m still unsure how the public interface of an Arc should look like.
so many thanks for this. i found what i was looking for. but i was wondering, pointhi, may i ask you questions in case i would have some for you? thanks again!
@nickoe, I wanted a clean repo from the beginning. When started the first thing by me was setting up a clean environment including testing, styling, documentation and a running CI. Having an existing repo would mean fixing all those little issues thrown by the different components at the beginning before even starting with actual improvements. And then you have things like pcbnew_easy.py which are simply a duplication of the whole API abstraction in some other way.
It would have been possible to reuse the existing repo. But I simply thought using a new one would be simpler to get all up and running, and to avoid legacy stuff.
Is there a (rough) schedule for Python support in eeschema ?
General Scripting comment: One feature in Mentor’s PADS I liked, was the scripts are very similar, and it was possible to code one script that would work in either SCH or PCB.
There was a ‘which-host’ boolean that allowed support for minor function differences.
That’s useful for things like BOM, and FPGA pinout connection extraction tasks.
. Not a center and an end point which KiCad uses in it’s internal definition. The abstraction layer handles the conversations between those different representation. I’m still unsure how the public interface of an Arc should look like.