Hello guys,
it seems that the relevant KiCAD developers are more active here than they are active on the developers’ mailing list. So I will try my luck here.
What is the status of KiCAD 6 Python API? There are 4 days until RC1 and I am wondering (probably just like other plugin developers) - what will the API look like? Or will there be only the old SWIG-based API? What can we expect and what didn’t make it in v6? Can we read about it somewhere or start testing it in our plugins?
Please, let us know!p you be more open and transparent about your goals, processes, and progress?
Personally, I don’t have any hard feelings about not having a new API in v6 (if there is no regression compared to v5). I know how hard is to maintain a large project and I appreciate your work. However, over the last year, I got under the impression that API is something you refuse to talk about. Why is it so? Not talking about it makes a lot of people furious and I guess the power users’ community of KiCAD would appreciate knowing, e.g., that no API is coming so we can prepare for it. However, living in the limbo of not knowing anything and basically being ignored is really annoying and tiring. I think being transparent is beneficial for both sides!
What can we, as the community of KiCAD users, do to help you be more open and transparent about your goals, processes, and progress?
It is unlikely that the API will be ready for general use by RC1. Part of this was the technical hurdles to build it internally to the codebase but mostly this has been my fault for not getting the technical bits out in time for the rest of the team to help with the actual python coding/interface.
We recognize that this process for the updated API has a lot of room for improvement. We’re planning on reviewing as a team these processes following the release of v6.
Is it then possible to have 6.1 with the new API plus some feature enhancements? The new plan for yearly releases looks promising, but waiting another year for the API is so big disappointment that I would rather have 6.1 soon with nothing else new but that API. You of course know that many plugin developers have even slowed down development while waiting for the API, and to be honest the new PCM is half pointless without more and better plugins. The current API is relatively difficult to use.
I can guess that if the API really is too late for the milestone this is a big disappointment for you and the rest of the development team, too, maybe even more so than for us end and semi-end users. We can live with this, and KiCad is better and stronger than ever anyway.
Thank you for sharing. Does this mean there is not even a draft of the API? If so, do you consider involving the community in specifying the API?
Wishful thinking; could you reconsider the option for having full SWIG export before proper API is released (without any support from the dev team) as I proposed in https://gitlab.com/kicad/code/kicad/-/issues/7134? That would allow the plugin creators to discover what functionality is actually needed and what we would like to expect in a proper stable, officially supported API.
Full SWIG export is not doable. Python doesn’t map 1:1 to C++ and whenever you expose something sometimes it just works sometimes it needs tweaks. The bigger the surface area of the code you expose the more tweaks you need, the more the tweaks break when code underneath changes. Maintenance effort multiplies very quickly.
Sometimes you also have to compromise C++ conventions and style to make something exposable in SWIG. Doing it for whole codebase is a bad idea.
What’s the current status, plan and schedule for python plugin support in Eeschema? I can see that there’s now a console, but is there currently a SWIG API? If not, is there any plan to add one, or is the plan to skip SWIG and implement a “real” API whenever possible?
Thanks, I was always under the impression that SWIG promotes itself as “automated export, which can be messy, but it is automated”. Unlike PyBind11, which is manual, but finely crafted and makes Python and C++ cooperate seamlessly.
This is why I always add in my request “unsupported”. Even make it undocumented, so people that do not know KiCAD internals don’t try to use it. I know, that the full export could be a mess and there surely will be stuff broken, but it should serve as a sandbox. Basically, yes, I could do the bindings on my own, but then I have to deal with distribution across platforms. So that’s why I am asking for KiCAD to bundle it.
SWIG will do everything automatically and give you an interface which is useless in some cases. Pointers, templates, lambdas just don’t work unless you add swig directives and/or change the cpp code itself. It would be not just “unsupported” but “unusable”.
SWIG has caused the dev team no end of headaches. It is anything but automated to keep it working. I cannot overstate the maintenance issues @qu1ck mentioned.
It wouldn’t, really. The issue is that SWIG generates wrappers automatically on top of C++ code, but wrapping internal C++ code does not make an API. An API requires manual design to be a real API: it needs to have conventions, so that doing a similar thing to a different type of object feels the same. It needs to have abstractions, so that the implementation of something internal doesn’t change the way it’s exposed to API callers.
SWIG is a good solution if you have a program that already has a stable API in one language, and you want to expose that API to other languages. The SWIG implementation in KiCad today isn’t that. There is no internal “stable API” layer that is then exported to Python via SWIG.
The big shift in paradigm for KiCad with the new API is not switching from SWIG to pybind11 (although there have certainly been hurdles there also). It is defining a stable API layer that actually acts like an API. At the point that we have that, really the difference between SWIG and pybind11 is down to technical issues that shouldn’t matter to Python API consumers (SWIG also makes compiling KiCad take a lot longer, for example). In the diagram below, you could replace pybind11 with SWIG and basically get the same effect (just a worse developer experience for the C++ devs)
So, unfortunately, there is no easy path to just have SWIG generate you an “all of KiCad API”, because for SWIG to do that, we would have to already have this “all of KiCad API” implemented in C++. Without this, SWIG can’t generate something from nothing, and as @qu1ck says it would be completely broken.
The answer depends on how much work/time is between today and a working, ready-for-release API, which at this point is an unknown. I think we’re open to the idea if the time needed is significantly less than the V7 release schedule.
