Cannot register external action plugins in Pcbnew?

Hi all,

I have built KiCad from source, and it worked.

Then I wanted to try python plugins in Pcbnew tools, because I want to try https://github.com/HiGregSmith/KiCommand - so I rebuilt with KICAD_SCRIPTING_ACTION_MENU, as noted in Pcbnew external plugin? and HOWTO: Register a python plugin inside pcbnew Tools menu ; this is my KiCad version info:

Application: kicad
Version: (2017-11-13 revision d98fc85)-master, release build
Libraries:
    wxWidgets 3.0.2
    libcurl/7.35.0 OpenSSL/1.0.1f zlib/1.2.8 libidn/1.28 librtmp/2.3
Platform: Linux 4.4.0-104-generic i686, 32 bit, Little endian, wxGTK
Build Info:
    wxWidgets: 3.0.2 (wchar_t,wx containers,compatible with 2.8) GTK+ 2.24
    Boost: 1.54.0
    Curl: 7.35.0
    Compiler: GCC 4.8.4 with C++ ABI 1002

Build settings:
    USE_WX_GRAPHICS_CONTEXT=OFF
    USE_WX_OVERLAY=OFF
    KICAD_SCRIPTING=ON
    KICAD_SCRIPTING_MODULES=ON
    KICAD_SCRIPTING_WXPYTHON=ON
    KICAD_SCRIPTING_ACTION_MENU=ON
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=OFF
    KICAD_SPICE=OFF

This is on Ubuntu 14.04, 32-bit:

$ echo $(cat /etc/issue) $(lsb_release -idrc); uname -a
Ubuntu 14.04.5 LTS \n \l Distributor ID: Ubuntu Description: Ubuntu 14.04.5 LTS Release: 14.04 Codename: trusty
Linux myPC 4.4.0-104-generic #127~14.04.1-Ubuntu SMP Mon Dec 11 12:44:57 UTC 2017 i686 i686 i686 GNU/Linux

The problem is that; while I do get the new menu item in Pcbnew/Tools/External Plugins, there is only ā€œRefresh Pluginsā€ inside, and the list never gets populated, regardless of how manu times I click ā€œRefresh Pluginsā€:

One possible problem can be, is that Iā€™m not installing in the system directories once I build from source, but in an ā€œout of treeā€, that is ā€œportableā€ folder, as in:

make -j 4
make install DESTDIR=/media/disk/mybin/kicad

As such, I copied the KiCommand folder in plugins/ within that install directory:

cp -a KiCommand_git/kicommand /media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins/

So, I added some debugging statements, which can be seen in this diff - and then rebuilt again:

diff --git a/pcbnew/class_action_plugin.cpp b/pcbnew/class_action_plugin.cpp
index 171e0f2..61c94ee 100644
--- a/pcbnew/class_action_plugin.cpp
+++ b/pcbnew/class_action_plugin.cpp
@@ -102,6 +102,8 @@ int ACTION_PLUGINS::GetActionsCount()

 void ACTION_PLUGINS::register_action( ACTION_PLUGIN* aAction )
 {
+    printf("ACTION_PLUGINS::register_action aA %p nm %s\n", aAction, (const char*)aAction->GetName().mb_str());
+
     // Search for this entry do not register twice this action:
     for( int ii = 0; ii < GetActionsCount(); ii++ )
     {
diff --git a/pcbnew/swig/pcbnew_action_plugins.cpp b/pcbnew/swig/pcbnew_action_plugins.cpp
index ef18cde..2b5f08f 100644
--- a/pcbnew/swig/pcbnew_action_plugins.cpp
+++ b/pcbnew/swig/pcbnew_action_plugins.cpp
@@ -156,7 +156,7 @@ void* PYTHON_ACTION_PLUGIN::GetObject()
 void PYTHON_ACTION_PLUGINS::register_action( PyObject* aPyAction )
 {
     PYTHON_ACTION_PLUGIN* fw = new PYTHON_ACTION_PLUGIN( aPyAction );
-
+    printf("PYTHON_ACTION_PLUGINS::register_action %p\n", aPyAction);
     fw->register_action();
 }

@@ -380,6 +380,8 @@ void PCB_EDIT_FRAME::RebuildActionPluginMenus()
 {
     wxMenu* actionMenu = GetMenuBar()->FindItem( ID_TOOLBARH_PCB_ACTION_PLUGIN )->GetSubMenu();

+    printf("PCB_EDIT_FRAME::RebuildActionPluginMenus actionMenu %p\n",
+      actionMenu);
     if( !actionMenu ) // Should not occur.
         return;

@@ -410,6 +412,9 @@ void PCB_EDIT_FRAME::RebuildActionPluginMenus()
         actionMenu->Delete( item );
     }

+    printf("PCB_EDIT_FRAME::RebuildActionPluginMenus av_mn_sz %d gactcnt %d\n",
+      available_menus.size(), ACTION_PLUGINS::GetActionsCount());
+
     for( int ii = 0; ii < ACTION_PLUGINS::GetActionsCount(); ii++ )
     {
         wxMenuItem* item;
diff --git a/scripting/kicadplugins.i b/scripting/kicadplugins.i
index 1dfaa16..388824e 100644
--- a/scripting/kicadplugins.i
+++ b/scripting/kicadplugins.i
@@ -219,6 +219,8 @@ def LoadPlugins(bundlepath=None):
         plugin_directories.append(os.path.join(os.environ['HOME'],'.kicad','scripting'))
         plugin_directories.append(os.path.join(os.environ['HOME'],'.kicad','scripting','plugins'))

+    print("plugin_directories: {}".format(plugin_directories))
+
     global PLUGIN_DIRECTORIES_SEARCH
     PLUGIN_DIRECTORIES_SEARCH=""
     for plugins_dir in plugin_directories:    # save search path list for later use
@@ -241,6 +243,7 @@ def LoadPlugins(bundlepath=None):
         sys.path.append(plugins_dir)

         for module in os.listdir(plugins_dir):
+            print("module '{}' plugins_dir '{}'".format(module, plugins_dir))
             if os.path.isdir(os.path.join(plugins_dir,module)):
                 LoadOneSubdirPlugin(plugins_dir, module)
                 continue
@@ -256,6 +259,7 @@ class KiCadPlugin:
         pass

     def register(self):
+        print("KiCadPlugin: register {}".format(self))
         if isinstance(self,FilePlugin):
             pass # register to file plugins in C++

So, now when I run kicad, and then start Pcbnew, I get printed in terminal:

plugin_directories: ['/media/disk/mybin/kicad/usr/local/share/kicad/scripting', '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins', u'/home/user/.config/kicad/scripting', u'/home/user/.config/kicad/scripting/plugins', '/home/user/.kicad_plugins', '/home/user/.kicad/scripting', '/home/user/.kicad/scripting/plugins']
module 'kicad_pyshell' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting'
module 'plugins' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting'
module 'zip_wizard.pyc' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'FootprintWizardBase.pyc' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'sdip_wizard.pyc' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'bga_wizard.pyc' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'qrcode.pyc' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'kicommand' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'bga_wizard.py' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'qfp_wizard.pyc' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'qrcode_footprint_wizard.py' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'uss39_barcode.py' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'FootprintWizardBase.py' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'circular_pad_array_wizard.py' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'FPC_wizard.pyc' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'uss39_barcode.pyc' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'sdip_wizard.py' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'touch_slider_wizard.py' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'microMatch_connectors.py' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'qfn_wizard.pyc' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'zip_wizard.py' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module '__init__.py' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'qfp_wizard.py' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'touch_slider_wizard.pyc' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'FPC_wizard.py' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'circular_pad_array_wizard.pyc' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'PadArray.pyc' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'microMatch_connectors.pyc' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'qrcode_footprint_wizard.pyc' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'PadArray.py' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'qrcode.py' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module '__init__.pyc' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
module 'qfn_wizard.py' plugins_dir '/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins'
PCB_EDIT_FRAME::RebuildActionPluginMenus actionMenu 0xb777c18
PCB_EDIT_FRAME::RebuildActionPluginMenus av_mn_sz 0 gactcnt 0
PCB_EDIT_FRAME::RebuildActionPluginMenus actionMenu 0xb76e5e8
PCB_EDIT_FRAME::RebuildActionPluginMenus av_mn_sz 0 gactcnt 0

Pretty much the same printout happens when I run ā€œRefresh Pluginsā€ (except then I have only two PCB_EDIT_FRAME::RebuildActionPluginMenus printouts)

So, the directories are not a problem - they are found and inspected for plugins; plugins are found - even module ā€˜kicommandā€™ is found; actionMenu seems instantiated - however:

  • available_menus.size() and ACTION_PLUGINS::GetActionsCount() are always 0 ?
  • The print in kicadplugins.i in def register is never printed (although the prints in the same file, in def LoadPlugins are printed fine) - meaning that def register here never runs ?
  • Likewise, ACTION_PLUGINS::register_action and PYTHON_ACTION_PLUGINS::register_action never seem to run, as their printouts are also not printed ?

So, if the ā€œregisterā€ functions donā€™t run, that explains why GetActionsCount() would be zero.

But then the question is - why donā€™t the ā€œregisterā€ functions run ?!

The only thing Iā€™ve noticed, is that when I change something in the .i file, and rebuild, I get this at the end of the terminal log:

...
[ 82%] Built target eeschema_kiface
[ 82%] [ 83%] Built target qa_eagle_plugin
Generating pcbnew_wrap.cxx, pcbnew.py
[ 83%] Built target eeschema
/media/disk/src/kicad/kicad_git/pcbnew/../include/geometry/shape_poly_set.h:76: Warning 325: Nested struct not currently supported (VERTEX_INDEX ignored)
/media/disk/src/kicad/kicad_git/pcbnew/../include/geometry/shape_poly_set.h:204: Warning 325: Nested class not currently supported (ITERATOR_TEMPLATE ignored)
/media/disk/src/kicad/kicad_git/pcbnew/../include/geometry/shape_poly_set.h:350: Warning 325: Nested class not currently supported (SEGMENT_ITERATOR_TEMPLATE ignored)
io_mgr.h:122: Warning 325: Nested class not currently supported (PLUGIN_REGISTRY ignored)
io_mgr.h:137: Warning 325: Nested struct not currently supported (REGISTER_PLUGIN ignored)
class_zone.h:839: Warning 325: Nested union not currently supported (WX_VECTOR_CONVERTER ignored)
class_module.h:88: Warning 325: Nested struct not currently supported (VECTOR3D ignored)
/media/disk/src/kicad/kicad_git/pcbnew/../include/class_colors_design_settings.h:48: Warning 401: Nothing known about base class 'SETTINGS'. Ignored.
class_board.h:50: Warning 315: Nothing known about 'std::unique_ptr'.
/media/disk/src/kicad/kicad_git/pcbnew/../include/utf8.h:180: Warning 503: Can't wrap 'operator const std::string&' unless renamed to a valid identifier.
swig_import_helper fixed for /media/disk/src/kicad/kicad_git/build/debugi686/pcbnew/pcbnew.py
Scanning dependencies of target pcbnew_kiface
[ 83%] Building CXX object pcbnew/CMakeFiles/pcbnew_kiface.dir/pcbnew_wrap.cxx.o
Linking CXX shared module _pcbnew.kiface
[100%] Built target pcbnew_kiface
[100%] Creating python's pcbnew native module _pcbnew.so for command line use.
[100%] Built target pcbnew
[100%] Built target qa_geometry
[100%] Built target pcbnew_python_module

In particular, notice that PLUGIN_REGISTRY and REGISTER_PLUGIN are ā€œignoredā€, because of ā€œWarning 325: Nested class not currently supportedā€. I looked up this error, and found:

Compatibility Note: Prior to SWIG-3.0.0, there was limited nested class support. Nested classes were treated as opaque pointers. ā€¦ With proper nested class support now available in SWIG-3.0.0, this feature has been deprecated and no longer works requiring code changes.

You therefore ought to use 3.0 or later for support for nested classes. That will allow you to avoid needing to suppress anything, and might fix your other issues too.

But, my swig version is 3.0+ ? :

$ apt-show-versions -r swig
swig3.0:i386/trusty-backports 3.0.2-1ubuntu1~ubuntu14.04.1 uptodate
$ swig3.0 -version

SWIG Version 3.0.2

Compiled with g++ [i686-pc-linux-gnu]

Configured options: +pcre

Please see http://www.swig.org for reporting bugs and further information

So, now Iā€™m really puzzled.

Would anyone know, what do I need to do, in order to have the Pcbnew action plugins, to register in the tools menu?

1 Like

Probably the problem is in the KiCommand plugin, I didnā€™t get it to be visible in the menu, either, even though some other plugin was visible in the menu.

2 Likes

Many thanks, @eelik:

Indeed - I just got https://github.com/easyw/kicad-action-plugins - and then tried:

cp kicad-action-plugins_git/action_menu_annular_check.py /media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins/

ā€¦ and now the ā€œregisterā€ actions are printed to stdout too:

PYTHON_ACTION_PLUGINS::register_action 0xa9ef280c
ACTION_PLUGINS::register_action aA 0xa7d5570 nm Annular check

ā€¦ and that plugin, action_menu_annular_check.py is shown in the menu:

ā€¦ while the kicommand is still not processedā€¦ So indeed, it must be something with the kicommand addon.

1 Like

@HiGreg Letā€™s get the authorā€™s attention.

2 Likes

Thanks for that, @hermit:

I managed to do some debugging and fixing on my own, hereā€™s the results:

First, I noticed in https://github.com/HiGregSmith/KiCommand :

For KiCad 4.0.7 stable:

Enable Tools > Scripting Console then enter

import kicommand

This will show the KiCommand dialog.

So, that is exactly what I tried - and I got this:

Py 0.9.8
Python 2.7.6 (default, Nov 23 2017, 15:54:50)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Startup script executed: /home/user/.config/kicad/PyShell_pcbnew_startup.py
import kicommand
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins/kicommand/__init__.py", line 2, in <module>
    from .kicommand import *
  File "/media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins/kicommand/kicommand.py", line 2351, in <module>
    os.chdir(USERSAVEPATH)
OSError: [Errno 2] No such file or directory: '/home/user/kicad/kicommand'

Ahha - so the directory in USERSAVEPATH, which turns out to be ~/kicad/kicommand, does not exist! Letā€™s see if that path is hardcoded in /media/disk/mybin/kicad/usr/local/share/kicad/scripting/plugins/kicommand/kicommand.py:

2349 import inspect
2350 USERSAVEPATH = os.path.join(os.path.expanduser('~'),'kicad','kicommand')
2351 os.chdir(USERSAVEPATH)
2352
2353 KICOMMAND_MODULE_DIR = os.path.dirname(inspect.stack()[0][1])
2354 LOADABLE_DIR = os.path.join(KICOMMAND_MODULE_DIR,'loadable')
2355 USERLOADPATH = USERSAVEPATH+':'+LOADABLE_DIR
2356 PROJECTPATH = os.path.dirname(pcbnew.GetBoard().GetFileName())
...
2367 def SAVE(name):
2368     dictname = 'user'
2369     if not os.path.exists(USERSAVEPATH):
2370         os.makedirs(USERSAVEPATH)
2371         output('created ~/kicad/kicommand')
2372     output("saving to %s"%name)

Well, it turns out it is hardcoded - and even if there is a ā€œcreate dir if not existā€ check in def SAVE, the first time os.chdir(USERSAVEPATH) hits is way before this check, so plugin loading fails. Letā€™s see if creating this directory manually helps:

$ ls ~/kicad
ls: cannot access /home/user/kicad: No such file or directory
$ mkdir -p -v ~/kicad/kicommand
mkdir: created directory ā€˜/home/user/kicadā€™
mkdir: created directory ā€˜/home/user/kicad/kicommandā€™

Right, so: open kicad again - again only ā€œAnnular checkā€ is registered at startup; but when I click Tools / External Plugins / Refresh Plugins thereafter, I finally get in stdout log:

PYTHON_ACTION_PLUGINS::register_action 0xa9f203ec
ACTION_PLUGINS::register_action aA 0xc669250 nm KiCommand

ā€¦ and kicommand window starts also:

Well, nice to have this out of the way - thanks for the help, all!

EDIT: Turns out, if I close KiCad, and then reopen KiCad, then Pcbnew, the ā€œKiCommandā€ menu entry in External Plugins is missing again - but all I need to do is ā€œRefresh Pluginsā€ again, and it shows up againā€¦

2 Likes

Awesome denugging! Glad you got it sorted. Unfortunately, registering an Action Plugin fails silently.

Apparently there are two things to fix in KiCommand: create the save folder, and
Try to come up with a way to fail loudly.

Thank you again for your awesome work on this!

2 Likes

Ran into the same issue just now. Iā€™m on Mac OS. Couldnā€™t figure out why KiCommand didnā€™t loadā€¦ went ahead and made a dir at ~kicad/kicommand and KiCommand showed up in the external plugin list. Thanks for the help!

Downloaded KiCommand source from commit fa50e17f795af454f90354ac1a8c08b068415a7c on the GitHub page.

Iā€™m on KiCad 5.1.5-0-10_14.

Application: Pcbnew
Version: (5.1.5-0-10_14), release build
Libraries:
    wxWidgets 3.0.4
    libcurl/7.54.0 LibreSSL/2.6.5 zlib/1.2.11 nghttp2/1.24.1
Platform: Mac OS X (Darwin 18.7.0 x86_64), 64 bit, Little endian, wxMac
Build Info:
    wxWidgets: 3.0.4 (wchar_t,STL containers,compatible with 2.8)
    Boost: 1.69.0
    OpenCASCADE Community Edition: 6.9.1
    Curl: 7.54.0
    Compiler: Clang 9.0.0 with C++ ABI 1002

Build settings:
    USE_WX_GRAPHICS_CONTEXT=ON
    USE_WX_OVERLAY=ON
    KICAD_SCRIPTING=ON
    KICAD_SCRIPTING_MODULES=ON
    KICAD_SCRIPTING_PYTHON3=OFF
    KICAD_SCRIPTING_WXPYTHON=ON
    KICAD_SCRIPTING_WXPYTHON_PHOENIX=OFF
    KICAD_SCRIPTING_ACTION_MENU=ON
    BUILD_GITHUB_PLUGIN=ON
    KICAD_USE_OCE=ON
    KICAD_USE_OCC=OFF
    KICAD_SPICE=ON

This should be fixed either already or soon. Let me check when I fixed it, and Iā€™ll edit this comment to indicate which version of kicommand attempts to fix this. I modified kicommand to now create files in the standard kicad user location instead of ~/kicad/kicommand. However, even with the change, I still look for files in the deprecated location in addition to the new location, for backward compatibility. New installs for kicommand should only create files in the new location. Again, Iā€™ll edit this comment with specifics when I can.

Edit: update with this fix has just been pushed to github. ā€œNow properly uses GetKicadConfigPath() instead of hardcoded ~/kicad. Will create kicommand directory underneath, but now doesnā€™t cd into it. Files can still be found if they are in ~/kicad. Now handles immense output in KiCommand window by only keeping the last 32K characters. Added tests from easy-to-edit text file. If needed, running the tests can auto-generate and update the expected results. Update is triggered by ā€œNoneā€ in the expected field. After generating the new expected results, the new test file in KicadConfigPath()/kicommand must be copied into the install directory. New installations of kicommand will overwrite this file. Please submit/suggest new KiCommand tests youā€™d like to see. Current tests in that file are fairly uneventful, but show the capability to be expanded in the future.ā€

1 Like