I think Scheme would be better than lisp in general. Mostly they’re compatible, it’s just scheme doesn’t have cons so doesn’t need to implement cons cells and so can have more efficient internal representation than lisp. Lists for example can be represented as arrays with no need for cdr’s.
Scheme (and lisp) has some big advantages over python. The most obvious is the ability to programmatically construct code, such as data filters or sort ordering functions based on complex criteria. For example, for footprint or component library searches (# of pins, substring, specific library or substring, patterns, etc) - where instead of a monolithic comparison function that takea a gazillion parameters, a function is created that constructs a comparison function based on search criteria, and then this is used by a generic search. It’s also easier to verify since the filter or comparison constructor function can be tested in isolation - feed it inputs and see what code it outputs, without actually have to unit test against a giant test data set.
Of course, the ecosystem is much better for python and there are libraries for virtually any task.
You are probably right, it would be nice if there could be a maintained set of libraries that large that just worked and allowed use to just run tools like KiCost but that’s a huge maintenance burden. Honestly I think python is overkill for what it sounds like people are wanting to do with it.
Scheme/Lisp is a bit too esoteric for most… I’ve never written too much serious in it other than a sudoku solver.
Lua really is the go to language for simple scripting though, thats why it was developed to being with… and there is an updated wxLua bindings library to go along with it. https://github.com/pkulchenko/wxlua
And since Lua is designed to interface with C/C++ you don’t end up with the whole problem of having a secondary wrapper for the swig based API…
Exactly - change of top scripting language, does not solve the underlying problem, but creates many more…
Better to focus on fixing the areas of weakness, whilst keeping what works already.
There is a lot of Python out there already for kicad
I also see a growing number of Python Embedded downloaders, as Python moves to become more of a workbench standard.
Getting back to @cb88’s original point about fragile Python environments, this comic captures the situation well:
Depending on your system, you might have a system package manager (apt-get or homebrew) that installs Python and various versions of Python libraries, and then Python based tools like pip that will install possibly conflicting versions system-wide. It can become a mess when you start using both to upgrade library versions.
And then there are “virtual environments” and virtual environment wrappers that alleviate some of the problem, but they require work to setup. Too much, in my opinion, but I don’t particularly enjoy dependency management, even if it is unavoidable.
The best way to manage paths and dependencies I’ve found so far is with pipenv and it’s Pipfiles. pipenv makes it pretty easy to track the dependencies for an individual Python program and automatically install them into a self-contained virtual environment. The best part is that the Pipfile is part of the source repository maintained by the developer, and the user just needs to run pipenv shell to have the same build and run-time environment as the developer.
Perhaps providing Pipfiles with scripts and/or the KiCAD Python API would be helpful? I haven’t used the scripting API at all, so I don’t know.
All that said, I agree that Python is a fine choice for the KiCAD scripting language. I just wanted to acknowledge the original point that dependency hell is no fun. You just can’t avoid it.
I think improving our current python implementation is the way to go. I started with an abstraction layer as already pointed out (prototype: https://github.com/pointhi/kicad-python). It already works, but I think it would be more important at the moment to get the wxPython implementation to Phoenix as well as adding support for Python 3. If anyone has the interest to improve the current implementation, I’m happy to see contributors.
I think it would be even a good idea to drop Python 2 support completely if possible after getting Python 3, or at least output a big warning when someone tries to use it (Doesn’t make sense to write scripts for it: https://pythonclock.org).
pipenv looks interesting. Don’t know how this cooperates when you have multiple plugins with different virtual environments in one application.
Given the time it takes for development, there seems to be a fair amount of catch-up that occurs as the codebase is shifting to more recent libraries. I think it would save time and energy to be more brutal in dropping support for 2.7 and going straight to 3.
That’s easy to say, if you ignore the large and widespread user established code base.
Reality is, there is a lot of proven 2,7 Python out there, and that is not possible/practical to port overnight.
Best path is to encourage the dual-version support as described in the link above, so that over time V3 can be used more and more.
Some companies may choose to install 2 Kicads, a Py2.7 for legacy script use, and a Py3 for new development.
I’m very aware that there is a large base of support and libraries that use 2.7. I just believe that best path forward with limited resources is to look to the future rather than splitting attention between 2.7 and 3. It’s very much a chicken and egg problem, as we hold onto 2.7 because there is not enough support for 3, but this continues to degrade the usage of 3 as everyone is afraid to let go of 2.7.
Python support will not get better while trying to satisfy both branches. If it comes to committing to a path, I say look to the future. Sometimes things need to break to get better.
“large base of support and libraries that use 2.7”
Which are irrelevant to KiCad itself as it isn’t going to ship with a lot of external libraries such as would be needed to internally support large plugins like KiCost etc… right, just enough to allow it to act as an extension language for Kicad?
It’s not irrelevant to those of us who like being able to use external libraries. For example I am trying to automate many processes using KiCad using dxf placement, optimisation of component location placement and autogeneration of schematics, netlists and pcbs. Hence external library support is important - but I would also like it stable so it’s not going to break with every release.
I do understand this is a lot of work and a long way off.
It’s intended for commercial purposes, but I plan on releasing a more genericised version when I have it more robust and less held together by bandaids and swearing
Did you read the link I gave above ?
That suggests you can actually satisfy both branches and encourages development that is portable.
To assist that approach, it seems kicad could offer an installed choice of both Py2.7 and Py3 consoles.The porting effort multiply effect here is on the user side, not Kicad development.
Companies expect their scripts to continue to work.
I did read it. It sounds good and that would support people with already written modules. I still think that KiCad is in enough flux, that if there was a time to break cleanly the sooner the better, hence my support for just moving to 3. But if it is as easy as the porting link suggests then maybe the developers could follow that instead.
The python 2 official retirement isn’t the most crucial date to KiCad. Even now, the thing which has been problematic and lead to the planned 5.1 release is support for python libraries on Linux distros (specifically wxWidgets support). In the future the adoption of python 3 and abandoning of python 2 in the last distros are the watershed. Usually Debian stable is the last one in everything, unless it is luckily released in a good moment.
I don’t think there’s need to change the programming language. If something would be added, it should probably be javascript. I’m sorry, the fans of more rare languages, but generally speaking (outside KiCad) when a programming language is chosen, other than technical merits may be more important. Existing user base and willingness of people to learn the language are some of the most important criteria. See for example the results of the search “most popular programming languages 2018”:
A quick scan of one directory here shows 122 .py files, and I’m a fairly low-use case. So kicad focused Python will easily run into many tens of thousands out there, a small portion of which will be open source.
I don’t really care about “actively developed”, I just expect my scripts to continue to work.
That’s the litmus test most PCB designer would use.
In all other software I have, source code continues to compile fine on the compilers it is designed for, so the 2020 is more of a token date, than any genuine expiry.
Of course, if KiCad offered two Python consoles, I could test any scripts, as time allows, in both versions.
I get your point – with Python 2.7 seizing development it will in a sense become more stable than anything else . I think API changes in KiCad might be just as big risk as a 2 -> 3 switch.
Do you know if any applications which have two Python versions as part of the execution environment? I can’t really think of anything which would hinder it per say, since pipenv etc already enable such behavior.
. I think API changes in KiCad might be just as big risk as a 2 -> 3 switch.