Make python lib pcbnew global on mac

dctb · September 18, 2016, 12:05pm

I can run python pcbnew scripts inside the scripting console in the Pcbnew (application). But when I want to access the pcbnew lib outside the sripting console the lib was not found.
Is it possible to change that and how do I change that on mac?

trancecat · February 17, 2017, 5:06pm

I have the same problem. I tested manually moving python libs from kicad bundle and adding them manually in python scripts. I get "Reason: unsafe use of @executable_path in " error for _pcbnew.so all the time.
I don’t know how to fix that. If that problem solved it might be possible to make it global.

wezfurlong · April 5, 2017, 3:39pm

The issue is that the binaries in the macos bundle are relative to @executable_path. This is the right thing to use when scripts are run from inside the kicad application, but when you try to load the modules from your own python script, the shared modules that the pcbnew python depends on are then resolved relative to the apple-provided python binary… and they’re not there.

The true fix for this is some adjustment of the build parameters for the application.

If you just want to try things out, then here’s a simple local hack: Copy the python executable so that its @executable_path matches that of KiCad

cp /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python /Applications/Kicad/kicad.app/Contents/Applications/pcbnew.app/Contents/MacOS

Now instead of running /usr/bin/python to launch your script, you’d use /Applications/Kicad/kicad.app/Contents/Applications/pcbnew.app/Contents/MacOS/Python.

But before you can import pcbnew, you need to set the python path so that it knows where to find the kicad modules:

 $ PYTHONPATH=/Applications/Kicad/kicad.app/Contents/Frameworks/python/site-packages /Applications/Kicad/kicad.app/Contents/Applications/pcbnew.app/Contents/MacOS/Python
Python 2.7.10 (default, Jul 30 2016, 19:40:32)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pcbnew
>>>

John_Pateman · August 4, 2017, 1:52pm

I came across this when trying to set up some scripting in Kicad on macOS. I have successfully run a variety of script within Kicad from the console but am having problems invoking them from outside.

I am trying to launch a kicad python script from a bash shell process. The Python code works when launched directly from the console in Kicad but whenever I try and launch it via a bash script it fails with an import error - ‘no module named pcbnew’.

I have followed the above suggestion above and copied the Python executable from
from /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
to
/Applications/Kicad/kicad.app/Contents/Applications/pcbnew.app/Contents/MacOS
and adjusting my PYTHONPATH but I am still not able to ‘import pcbnew’ (unless it is via the Kicad console).

✘ johnpateman:trial_git [ master ✭ | …6 ] ➭ echo $PYTHONPATH
/Applications/Kicad/kicad.app/Contents/Frameworks/python/site-packages
✔ johnpateman:trial_git [ master ✭ | …6 ] ➭ /Applications/Kicad/kicad.app/Contents/Applications/pcbnew.app/Contents/MacOS/Python
Python 2.7.10 (default, Feb  7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pcbnew
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named pcbnew
>>> import sys
>>> print sys.executable
/Applications/Kicad/kicad.app/Contents/Applications/pcbnew.app/Contents/MacOS/Python
>>>

FWIW The bash script I am hacking extracts two git versions of a board and then passes the two differing .kicad_pcb files through the python script which invokes pcbnew to plot two sets of .svg files. The bash script then does a visual diff on the resulting files. I am trying to adapt the script to work with Fossil SCM rather than Git (which is doable, I think). As far as I am able to tell the rest of the bash script works as advertised. I do not have the coding knowledge to be able to rewrite the whole thing in python which would, of course be the ‘simplest’ and obvious answer

This is on a recent nightly version.

Application: kicad
Version: (2017-07-16 revision e797af3)-master, release build
Libraries: wxWidgets 3.0.2
libcurl/7.54.0 SecureTransport zlib/1.2.8
Platform: Mac OS X (Darwin 16.7.0 x86_64), 64 bit, Little endian, wxMac

Build Info -
wxWidgets: 3.0.2 (UTF-8,STL containers,compatible with 2.8)
Boost: 1.61.0
Curl: 7.43.0
KiCad - Compiler: Clang 7.3.0 with C++ ABI 1002
Settings: USE_WX_GRAPHICS_CONTEXT=ON
USE_WX_OVERLAY=ON
KICAD_SCRIPTING=ON
KICAD_SCRIPTING_MODULES=ON
KICAD_SCRIPTING_WXPYTHON=ON
KICAD_SCRIPTING_ACTION_MENU=ON
BUILD_GITHUB_PLUGIN=ON
KICAD_USE_OCE=ON

EDIT

I have made progress by doing the following

$ /Applications/Kicad/kicad.app/Contents/Applications/pcbnew.app/Contents/MacOS/Python
Python 2.7.10 (default, Feb 7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type “help”, “copyright”, “credits” or “license” for more information.

import sys
sys.path.insert(0, “/Applications/Kicad/kicad.app/Contents/Frameworks/python/site-packages”)`
import pcbnew

But no luck doing the same via the bash script.

brainstorm · August 31, 2017, 11:00pm

Not so lucky using Anaconda on my OSX system either:

$ python
Python 2.7.13 |Anaconda custom (x86_64)| (default, Dec 20 2016, 23:05:08)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import sys
>>> sys.path.insert(0, "/Applications/Kicad/kicad.app/Contents/Frameworks/python/site-packages")
>>> import pcbnew
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Applications/Kicad/kicad.app/Contents/Frameworks/python/site-packages/pcbnew.py", line 42, in <module>
    import _pcbnew
ImportError: dlopen(/Applications/Kicad/kicad.app/Contents/Frameworks/python/site-packages/_pcbnew.so, 2): Library not loaded: @executable_path/../Frameworks/libwx_osx_cocoau_gl-3.0.0.dylib
  Referenced from: /Applications/Kicad/kicad.app/Contents/Frameworks/python/site-packages/_pcbnew.so
  Reason: image not found
>>> sys.path
['/Applications/Kicad/kicad.app/Contents/Frameworks/python/site-packages', '', '/Users/romanvg/.miniconda/envs/py2/lib/python27.zip', '/Users/romanvg/.miniconda/envs/py2/lib/python2.7', '/Users/romanvg/.miniconda/envs/py2/lib/python2.7/plat-darwin', '/Users/romanvg/.miniconda/envs/py2/lib/python2.7/plat-mac', '/Users/romanvg/.miniconda/envs/py2/lib/python2.7/plat-mac/lib-scriptpackages', '/Users/romanvg/.miniconda/envs/py2/lib/python2.7/lib-tk', '/Users/romanvg/.miniconda/envs/py2/lib/python2.7/lib-old', '/Users/romanvg/.miniconda/envs/py2/lib/python2.7/lib-dynload', '/Users/romanvg/.local/lib/python2.7/site-packages', '/usr/local/lib/python2.7/site-packages', '/usr/local/Cellar/protobuf/3.3.2/libexec/lib/python2.7/site-packages', '/Users/romanvg/.miniconda/envs/py2/lib/python2.7/site-packages', '/Users/romanvg/.miniconda/envs/py2/lib/python2.7/site-packages/Sphinx-1.6.2-py2.7.egg', '/Users/romanvg/.miniconda/envs/py2/lib/python2.7/site-packages/aeosa', '/Users/romanvg/dev/toil/src', '/Users/romanvg/.miniconda/envs/py2/lib/python2.7/site-packages/setuptools-27.2.0-py2.7.egg']

Any further hints on how to execute things like OpenFixture via the regular shell, by using its accompanying shell script are very welcome!

More details on issue #1 from OpenFixture.

brainstorm · September 1, 2017, 7:19am

Cross referencing threads, it seems that Mac’s system integrity protection (SIP) gets in the way: Problem running pcbnew python plugin on OS X

EDIT: disabling via recovery mode (CMD+R while booting up) and csrutil disable does not help my case either.

John_Pateman · September 1, 2017, 10:07am

I think the shell script references 'python GenFixture.py '. I think you need to add a ‘source activate name-of-virt-env’ into the script before calling python? I think that anaconda will only switch the active terminal - if you open another terminal and type ‘which python’ you will get the default anaconda python not the virtenv version - so you need to get the script process to activate the correct environment (or specify it directly).

[abc@xyz] trial $ which python
/Users/abc/miniconda3/bin/python
[abc@xyz] trial $ source activate kifield
(kifield) [abc@xyz] trial $ which python
/Users/abc/miniconda3/envs/kifield/bin/python

You could try specifying the python by changing the call to ‘python’ with the full path i.e.
/Applications/Kicad/kicad.app/Contents/Applications/pcbnew.app/Contents/MacOS/Python GenFixture.py .....

I am no longer sure which is the default macOS python on 10.12 but you probably should check ‘which python’ and which version ‘python’ links to. I also have python 3 installed and need to remember to use ‘python2’ to invoke the appropriate 2.7.10 version I need for some of the scripts.

I have managed to get external python scripts working on macOS 10.12 with executable python code in /usr/local/bin.
I have followed @wezfurlong advice to copy the python executable so that its @executable_path matches that of KiCad by doing a

cp /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python /Applications/Kicad/kicad.app/Contents/Applications/pcbnew.app/Contents/MacOS

The shebang line of the python code then needs to be;

#!/Applications/Kicad/kicad.app/Contents/Applications/pcbnew.app/Contents/MacOS/Python

Best of luck!

brainstorm · September 1, 2017, 3:08pm

Thanks John, I did try the copy and shell-bang before but didn’t work just by itself. Instead, I had to rewrite a bunch of relative paths in _pcbnew.so like so (see the shell-script with otool and install_name_tool:

Otherwise I was stuck with Reason: image not found errors when trying to import pcbnew and/or run the script from the terminal…

Now there are no issues with libraries nor python interpreter. Right now it’s about OpenFixture issues with PAD_SMD not being found… work in progress, thanks a lot for your feedback

brainstorm · September 1, 2017, 4:07pm

And success! I had to modify the python code a little bit to fit with some pcbnew API changes, I’ll pullrequest my changes back to the author soon

Take home message if KiCAD release engineers are reading this: OSX python scripting support needs a bit more testing and perhaps some tweaking in the relative paths for _pcbnew.so. Instead of the ugly hack I pulled off in the fix_osx_pcbnew_kicad_libs.sh, there should be neater linker/CMakeFile fixups that would ease this pain.