Moving from Python to Lisp/Lua

Actually I’d call into question the utility of python vs how hard it is to maintain… I have a few scripts I use to rescale parts and such but it’s almost not worth it since the environment breaks about every time I update it.

What about an integrated javascript parser like duktape etc… something simple that allows programmability but doesn’t break as easily. Having a stable / usable embedded programming language is hard when the language and libraries it’s based on are changing at a much greater rate.

Embeded common lisp might be an option for people that get Emacs (not I but there are alot of those types out there).

Then of course there is the very popular LuaJit… all three of these are designed for embedded unlike python where it’s sort of a secondary feature.

Or perhaps a common api for working with KiCad that different language plugins can be written for… so people can get thier Python/Javascript/Lisp/Lua groove on and get on with thier lives lol. No idea if that is feasible but it would certainly be neat if it were possible.

1 Like

No matter what the scripting language will be, the interface between that scripting language and kicad (the so called API) needs to have an abstraction layer such that it can be stable.
I doubt this will be any easier to provide for any language.

4 Likes

For sure, I was just a bit concerned that Python itself isn’t that stable… there is alot of reliance in it on external package hand-waiving working.

Python is stable, even with the Python 2 vs Python 3 incompatibilities. It is wxPython that is a mess.

2 Likes

Sure, it’s stable on day one, I can’t say I’ve often seen a working python program crash, however a year later though and if you’ve tried to install anything new and you can almost guarantee it will break. In that sense it is very fragile.

In many cases this is fine as python excels at being a prototyping language… but it does not excel at being low maintenance.

To be fair, even C/C++ requires maintenance, the compilers keep getting fussier and C++ itself is an evolving language.

3 Likes

Is it possibly you heavily rely on things outside the core of python?

The kicad footprint generators and also our test scripts run basically without modification for years now.

Or have you been bitten by the transfer from python2 to python3? (Python3 is not backwards compatible on purpose. But within python3 old scripts should always run the same way in newer python3 versions. The same was true for python2.)

If you write python2 programs correctly (meaning using __future__) then it should even be compatible with both python2 and python3 (admittedly it is not worth the additional effort for most scripts)

3 Likes

With developer power available everything is hard to maintain. Yes, the choice of python was probably influenced by the support for it from the wxwidgets and availability of wxpython (along with PyAlaCarte and PyAlaMode) and posibility to bind it to c++ wx widgets. And when it was implemented nobody could foresee GTK2/GTK3 issues. So the choice was probably optimal at the time it was made. Hindsight is 20/20.

I really doubd that with any other language choice you’d get so much for so little. But I can imagine that Linux users can not really appreciate this as it does not work for them.

As KiCad development is open if somebody feels like it he/she can always add Lisp/Lua support to it, but starting along python. But this would require additional developer power and I cannot see where one would get it. And I doubt you could do something like @HiGreg’s https://github.com/HiGregSmith/LayerViewSet or @devbisme’s https://github.com/xesscorp/WireIt

Again this does require additional developer power.

I think it is easier for the python script/plugin developer to acknowledge the fluidity of the interface and prepare herself/himself for long term support where some of the bugs to fix will be caused by API change

1 Like

Another point is that there are far more programmers around familiar with Python than Lisp/Lua.
In the extreme case there would be major problems if a core feature was written in an obscure language by an expert who then dropped out of the project.

Yeah I see how wxPython makes it difficult to find an alternative… and none of the bindings to other languages are well maintained it seems.

There are no plans to introduce any other scripting language than Python (and Lisp in particular - Kicad is an EDA tool, not an operating system like Emacs ;-).

The current Python API is not stable as it directly reflects the internal C++ model. Making a stable API wrapper is on our to-do list, but there’s no manpower. Any Python experts wanting to help here?

Tom

Hi @twl
you may have a look at @hyOzd work here:

Kicad has a python api which is automatically created from the C++ API using a tool called SWIG. This API isn’t very pythonic, it feels like a c++ library and doesn’t have the ease of use and flexibility of python. So, some developers decided to create a wrapper around this API, called kicad-python 7. I believe their aim was to create a simpler and better documented scripting interface for kicad.

Notice that kicad-python, doesn’t call the C++ API directly, so it isn’t a replacement of actual python API of kicad, but is a wrapper around it.

About my fork of kicad-python; it wasn’t actively developed at the time I saw it, so I decided to fork it and add some features. It’s far from complete, and I only add features as I need them.

To be able to use above code snippet, you should download/clone my kicad-python 3 repository and add it to python path so that interpreter can import modules;

import sys
sys.path.append("/path/to/kicad-python/")

A tip; you can use function execfile() function to run python scripts from the python command line.

and the repo from this one is forked

2 Likes

I think @pointhi also has some plans for such an abstraction layer. (I seem to remember either something from the mailing list or from a discussion here.)
Not sure if these two are aware of each others efforts.

1 Like
1 Like

Perhaps no plans but it would be trivial since you are already using SWIG… Lua and javascript,and Rust are all closer to C/C++ than python and would probably not even require a secondary layer to beautify the API since they are designed to interoperate with C/C++. Just a thought.

Beauty or ugliness of the Python interface does not consitute a problem here. We need an abstraction layer to separate the scripting interface from C++ code. Such tight binding makes the scripting API unstable, as every time we modify the data structures or function signatures, they will change the Python API too, effectively breaking the existing scripts.

1 Like

One option could be to write a wrapper C++ API and autogenerate different language bindings from that.

The suggestion is to write an abstraction layer to insulate scripts from the C++ code. However, that just punts the stability problem to the data model. When the KiCad internal data model changes, that will affect the abstraction layer also. You will need to translate KiCad data model to an abstract data model for the script API. That is a lot of work to create and maintain, and will not solve the problem.

A stable script API depends on a stable KiCad data model, and since KiCad data model develops in a pretty ad hoc fashion, that also affects the script API. Until a time where the KiCad data model is stable, the script API will always be changing.

In the meantime, exposing the C++ functions directly is the least amount of work, even if it causes pain.

Also the last thing that is desirable is a whole bunch of different APIs and scripts written in different languages. That would create a horrible mess.

I don’t know if this is sensible for two reasons:

  1. Only a handful of action plugin developers are affected by the API changes
  2. Given the focus of development for V5.1 and V6, pcbnew API should not change much if it will change at all

I like the way you excluded the eeschema API to make your point work :slight_smile:

2 Likes