Numpy, other pip package installs for pcbnew scripting

[nevetts]

How can I install python packages for use with kicad scripting? Are these required to do scripting with pcbnew? I am following this pcbnew tutorial:

Tutorials on python scripting in pcbnew

Hey all, In an effort to help make kicad even better, I’ve put together some tutorials on using the python interface built into pcbnew. This first post serves as a table of contents for this thread. If you are experiencing a segfault when starting pcbnew on linux I recommend Ubuntu 16.04 with the kicad nightly build See this post below for a little more detail Pcbnew script will not run out of the box on Ubuntu 18 or recent linux Mint Getting started: https://kicad.mmccoo.com/kicad-scri…

In particular I am trying to run recipe.py in the scripting console (like at 9:51 of this youtube video: https://www.youtube.com/watch?v=av5HwAFl2VI&feature=youtu.be). The kicad console complains that I don’t have numpy installed. So…

I’ve tried opening a cmd prompt as administrator and

cd \Program Files\KiCad\bin
pip install numpy

which fails. (see output below)

I’ve also tried the suggested work around (Kicad python pip is working?) to run pip from the BOM console, which results in the same errors. In that thread @maui suggests building with mingw might work for other packages, but not numpy (I’m unfamiliar with mingw).

There’s also this (which didn’t seem to give a clear answer): Anyone installed NumPy for use with KiCAD's python?

I find it really weird that an example pcbnew script would require a library that I can’t easily get.

after ‘pip install numpy’ the terminal replies:

> C:\Program Files\KiCad\bin>pip install numpy
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Collecting numpy
  Using cached numpy-1.16.6.zip (5.1 MB)
Using legacy 'setup.py install' for numpy, since package 'wheel' is not installed.
Installing collected packages: numpy
    Running setup.py install for numpy ... error
    ERROR: Command errored out with exit status 1:
     command: 'c:/program files/kicad/bin/python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'c:\\users\\nevetts\\appdata\\local\\temp\\pip-install-w7ks_f\\numpy\\setup.py'"'"'; __file__='"'"'c:\\users\\nevetts\\appdata\\local\\temp\\pip-install-w7ks_f\\numpy\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'c:\users\nevetts\appdata\local\temp\pip-record-jb_kzs\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\program files\kicad\include\python2.7\numpy'
         cwd: c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy\
    Complete output (68 lines):
Running from numpy source directory.
Note: if you need reliable uninstall behavior, then install
    with pip instead of using `setup.py install`:
      - `pip install .`       (from a git repo or downloaded source
                               release)
      - `pip install numpy`   (last NumPy release on PyPi)
blas_opt_info:
    blas_mkl_info:
    Looking for python27.dll
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy\setup.py", line 419, in <module>
        setup_package()
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy\setup.py", line 411, in setup_package
        setup(**metadata)
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/core.py", line 137, in setup
        config = configuration()
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy\setup.py", line 164, in configuration
        config.add_subpackage('numpy')
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/misc_util.py", line 1036, in add_subpackage
        caller_level = 2)
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/misc_util.py", line 1005, in get_subpackage
        caller_level = caller_level + 1)
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/misc_util.py", line 942, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "numpy\setup.py", line 10, in configuration
        config.add_subpackage('core')
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/misc_util.py", line 1036, in add_subpackage
        caller_level = 2)
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/misc_util.py", line 1005, in get_subpackage
        caller_level = caller_level + 1)
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/misc_util.py", line 942, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "numpy\core\setup.py", line 760, in configuration
        blas_info = get_info('blas_opt', 0)
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/system_info.py", line 446, in get_info
        return cl().get_info(notfound_action)
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/system_info.py", line 639, in get_info
        self.calc_info()
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/system_info.py", line 1626, in calc_info
        blas_mkl_info = get_info('blas_mkl')
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/system_info.py", line 446, in get_info
        return cl().get_info(notfound_action)
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/system_info.py", line 639, in get_info
        self.calc_info()
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/system_info.py", line 1088, in calc_info
        info = self.check_libs2(lib_dirs, mkl_libs)
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/system_info.py", line 795, in check_libs2
        exts = self.library_extensions()
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/system_info.py", line 754, in library_extensions
        c = customized_ccompiler()
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/__init__.py", line 32, in customized_ccompiler
        c = ccompiler.new_compiler(plat=plat, compiler=compiler)
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/ccompiler.py", line 765, in new_compiler
        compiler = klass(None, dry_run, force)
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/mingw32ccompiler.py", line 103, in __init__
        build_import_library()
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/mingw32ccompiler.py", line 415, in build_import_library
        return _build_import_library_amd64()
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/mingw32ccompiler.py", line 464, in _build_import_library_amd64
        dll_file = find_python_dll()
      File "c:\users\nevetts\appdata\local\temp\pip-install-w7ks_f\numpy/numpy/distutils/mingw32ccompiler.py", line 285, in find_python_dll
        raise ValueError("%s not found in %s" % (dllname, lib_dirs))
    ValueError: python27.dll not found in ['c:/program files/kicad\\', 'c:/program files/kicad\\lib', 'c:/program files/kicad\\bin', 'C:\\Windows\\System32']
    ----------------------------------------
ERROR: Command errored out with exit status 1: 'c:/program files/kicad/bin/python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'c:\\users\\nevetts\\appdata\\local\\temp\\pip-install-w7ks_f\\numpy\\setup.py'"'"'; __file__='"'"'c:\\users\\nevetts\\appdata\\local\\temp\\pip-install-w7ks_f\\numpy\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'c:\users\nevetts\appdata\local\temp\pip-record-jb_kzs\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\program files\kicad\include\python2.7\numpy' Check the logs for full command output.

If I try another library, like scipy, I get something similar but with different error outputs:

> C:\Program Files\KiCad\bin>pip install scipy
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Collecting scipy
  Using cached scipy-1.2.3.tar.gz (23.3 MB)
    ERROR: Command errored out with exit status 1:
     command: 'c:/program files/kicad/bin/python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'c:\\users\\nevetts\\appdata\\local\\temp\\pip-install-dul1vn\\scipy\\setup.py'"'"'; __file__='"'"'c:\\users\\nevetts\\appdata\\local\\temp\\pip-install-dul1vn\\scipy\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'c:\users\nevetts\appdata\local\temp\pip-pip-egg-info-sctdgs'
         cwd: c:\users\nevetts\appdata\local\temp\pip-install-dul1vn\scipy\
    Complete output (52 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "c:\users\nevetts\appdata\local\temp\pip-install-dul1vn\scipy\setup.py", line 492, in <module>
        setup_package()
      File "c:\users\nevetts\appdata\local\temp\pip-install-dul1vn\scipy\setup.py", line 488, in setup_package
        setup(**metadata)
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/__init__.py", line 144, in setup
        _install_setup_requires(attrs)
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/dist.py", line 719, in fetch_build_eggs
        replace_conflicting=True,
      File "c:\program files\kicad\lib\python2.7\site-packages/pkg_resources/__init__.py", line 782, in resolve
        replace_conflicting=replace_conflicting
      File "c:\program files\kicad\lib\python2.7\site-packages/pkg_resources/__init__.py", line 1065, in best_match
        return self.obtain(req, installer)
      File "c:\program files\kicad\lib\python2.7\site-packages/pkg_resources/__init__.py", line 1077, in obtain
        return installer(requirement)
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/dist.py", line 786, in fetch_build_egg
        return cmd.easy_install(req)
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/command/easy_install.py", line 679, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/command/easy_install.py", line 705, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/command/easy_install.py", line 890, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/command/easy_install.py", line 1158, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/command/easy_install.py", line 1144, in run_setup
        run_setup(setup_script, args)
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/sandbox.py", line 253, in run_setup
        raise
      File "c:\program files\kicad\lib\python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "c:\program files\kicad\lib\python2.7/contextlib.py", line 35, in __exit__
        self.gen.throw(type, value, traceback)
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/sandbox.py", line 166, in save_modules
        saved_exc.resume()
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/sandbox.py", line 141, in resume
        six.reraise(type, exc, self._tb)
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/sandbox.py", line 154, in save_modules
        yield saved
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/sandbox.py", line 195, in setup_context
        yield
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/sandbox.py", line 250, in run_setup
        _execfile(setup_script, ns)
      File "c:\program files\kicad\lib\python2.7\site-packages/setuptools/sandbox.py", line 45, in _execfile
        exec(code, globals, locals)
      File "c:\users\nevetts\appdata\local\temp\easy_install-0yjpvv\numpy-1.20.1\setup.py", line 30, in <module>
    RuntimeError: Python version >= 3.7 required.
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

[Naib]

Unfortunately you are using windows and thus python27 so this comes with quite a few issues, namely that numpy dropped py27 support some time ago

I tried to find a specific version that would install with the specific dependancies… I gave up, mainly because I could use my Linux box as we

[nevetts]

Are you saying I won’t have this problem on linux?

[MitjaN]

TL;DR; Yup.

On Windows, KiCad uses python packaged along with KiCad. Because of wxWidgets/wxPython issues, the python version is 2.7. But version by itself is not a problem. On Windows KiCad and KiCad’s python are compiled with gcc. But numpy is compiled with some other compiler (MSVC IIRC). And python can not run modules compiled with different compiler.

On linux KiCad uses system pyhon (python3 on most distributions now) and system python has no issues installing numpy

[Naib]

Its a shame, there are some really good plugins out there.
The clearance one was the one I was after.
I did consider stripping numpy out since looking over the numpy usage it wasn’t too heavy and more for convenience

[nevetts]

Any particular version of linux? I just tried to install on centos 7 with a package manager and discovered that I received kicad 4.0, which doesn’t have a scripting console…

I also tried a nightly build following: KiCad 5 and nightly builds for Fedora Linux – maybe fedora is different enough from centos that this won’t work…I don’t really know what I’m doing.

dnf copr enable aimylios/kicad-nightly

gave me

Error: This repository does not have any builds yet so you cannot enable it now.

[Russ]

I use SolydK 10 64-bit Debian linux. APT shows no numpy package.

[marekr]

Kicad windows nightlies will soon have python3.9 support

[Naib]

Centos (streams) is great if you want long term stability but not great if you want newer (ie in the last 5-10years) of libraries. Its more geared towards servers and professional workstations rather than desktops

Fedora is a good option if you want to stay with the redhat brand

[aimylios]

I also tried a nightly build following: KiCad 5 and nightly builds for Fedora Linux – maybe fedora is different enough from centos that this won’t work…I don’t really know what I’m doing.

Thanks for trying to use my Fedora nightly builds! Unfortunately, they are currently not up to date. I’m working on some changes to the packaging scheme, and it will take another week or two until they receive the next update.
If you want to use nightly builds of KiCad on Fedora, please use the official Copr repository @kicad/kicad instead.

Independent of that: You can not use the Fedora nightly builds on CentOS (neither the old version 7 nor the current version 8). Some of KiCad’s dependencies are either outdated or not available at all on RHEL/CentOS.

If you want to play with the KiCad Python scripting interface and don’t mind to set up a dedicated (virtual) machine for that, I would recommend to use Fedora 33 and the stable release 5.1.9 (plus numpy, scipy, or whatever else you might need) from the Fedora repositories.

Install on Fedora

[maui]

@MitjaN

On Windows KiCad and KiCad’s python are compiled with gcc. But numpy is compiled with some other compiler (MSVC IIRC).

there is a numpy release even under MSYS2 mingw-w64-x86_64-python-numpy that can be used/distributed even under python2.7 win
https://packages.msys2.org/base/mingw-w64-python-numpy

[nevetts]

Kicad windows nightlies will soon have python3.9 support

@marekr How can I track the status of this? And / or try it out?

Is WSL2 on windows with python 3 an option for me? KICAD_SCRIPTING_PYTHON3=OFF on Windows for 5.1.9-1.9

[marekr]

@nevetts

@marekr How can I track the status of this? And / or try it out?

It’ll be announced, it’s part of our migration to MSVC based builds. Soon…hopefully…the holdup is out of my control

Is WSL2 on windows with python 3 an option for me?

Maybe? I am not a particular fan of using WSL2 for GUI related items and it can introduce bad behavior that we may not get around to fixing

[maui]

@marekr

It’ll be announced, it’s part of our migration to MSVC based builds.

so kicad is leaving open source build MSYS2 platform switching to closed source compiler/environment?

[eelik]

Does it matter when the platform itself, Windows, is closed source? It’s a practical decision, not ideological. Apparently Open Source would have been preferred if it had worked.

[maui]

@eelik

Does it matter when the platform itself, Windows, is closed source?

it does matter for me (and I think I’m not alone)… to get the building chain you need to use a proprietary compiler within all the issue related.

@eelik

It’s a practical decision, not ideological. Apparently Open Source would have been preferred if it had worked.

??? the ‘only’ stopper afaik is wxpython 4 (with py3 support) built for windows msys2
there were some wip

Kicad Python3 support on windows

Hello, I need to script KiCad with python3 for various reasons. Default Kicad windows binaries are not built with “KICAD_SCRIPTING_PYTHON3” option “ON”. I tried to build KiCad using msys2 but it seems wxpython 4 (supporting python3) has no msys packages. I tried to build wxPython on msys, but this look to be quite a hard task as it detect a windows platform and try to build using VS (http://wxpython-users.1045709.n5.nabble.com/Install-wxPython-4-for-MSYS2-td5728732.html) Is python3 support …

is that been abandoned?
[EDIT] it seems it has been solved here:

The snapshot wxPython-4.1.2a1.dev5156+3500ac7a.tar.gz wxPython now compiles on MSYS2 and can be imported in python.

Problem building wxPython 4.1.1 on MSYS2

[nickoe]

There are now some msvc based builds available for testing at https://kicad-downloads.s3.cern.ch/index.html?prefix=windows/testing/

[Naib]

@maui

it does matter for me (and I think I’m not alone)… to get the building chain you need to use a proprietary compiler within all the issue related.

While I do agree with you on this, there is a benefit in using a totally different toolchain… bugfinding since a different toolchain will interpret the src differently. Blizzard built WoW with a native linux version purely for this reason (never released as its one thing to find bugs, its another to release and support a client).
The flipside is if/when compiler specific bugs result in boilerplate code to make it work … think websites and all the IE hacks that had to be added.

[maui]

@Naib

The flipside is if/when compiler specific bugs result in boilerplate code to make it work … think websites and all the IE hacks that had to be added.

How do you know for how long a ‘free’ version of a commercial compiler would be keept ‘free’… then all your effort in changing the code to adapt to the ‘commercial’ compiler would be thrown out and you would be back again in searching a new solution for what you have already discharged.
This happened so often in the past that I cannot understand why it still on the way to move…

[eelik]

AFAIK the original work and decision towards MS tools for KiCad was driven essentially by the python/wx problem. If it happens that MS stops supporting their free compiler tools, I’m pretty sure that it affects a huge amount of Open Source projects which then must move to mingw, including wx. In that case all needed libraries etc. would be supported for mingw again so that KiCad can move back. KiCad wouldn’t be in any worse position than maybe thousands of other projects, and there would be enough pressure so that KiCad project doesn’t need to worry about that.

This is just my personal opinion, anyone can disagree freely and give counterarguments, I may change my mind on this.