Board2Pdf v1.1 released in PCM

albin · March 15, 2023, 9:27pm

I’ve added the ability for the user to choose between PyMuPdf (fitz) and pypdf (currently PyPDF4) in the GUI. The code is found here

The selected library is currently used for coloring. PyPDF4 is always used for merging and adding all the pages to a final file.

The problem is that I’m stuck with a problem that I don’t understand.

All works fine as long as there’s no parentheses in any footprint description. This has to do with Issue #14302 that got fixed today, but I’m still using 7.0.1 because I’d like to find a workaround.

If there are parentheses in at least one footprint description, PyPDF4 is supposed to throw a warning that I just want to ignore. The problem is that I’m getting an exception instead.

What’s mind-boggling is that I only get the exception if I don’t open the Python scripting console before running the plugin!

Can someone help me understand why it matters if I open the Python scripting console or not before running the plugin? It doesn’t matter if I load the plugin from the scripting console or if I press the button in the toolbar. Perhaps @qu1ck knows something? You seem to know most things there is to know about Python in KiCad.

Steps to reproduce a working scenario:

Open PCB Editor.
Open complex_hierarchy.kicad_pcb or some other board with parentheses in at least one footprint description.
Open the Python scripting console.
Open Board2Pdf using the icon in the toolbar.
Press Plot.

This works just fine. It doesn’t matter if the Python scripting console is left opened or closed before Board2Pdf is opened. The only thing that’s important is that it is opened.

Sometimes (not always) I get a windows that lists all warnings, but it always works.

Steps to reproduce a non-working scenario:

Open PCB Editor.
Open complex_hierarchy.kicad_pcb or some other board with parentheses in at least one footprint description.
Do NOT Open the Python scripting console.
Open Board2Pdf using the icon in the toolbar.
Press Plot.

Doing this always throws an AttributeError:

I cannot figure out why it matters if I open the Python scripting console or not. And I cannot figure out a way to solve this. It's not just that I get an AttributeError. The file that is supposed to be created is left empty. Even if I completely remove my error handling in colorize_pdf_pypdf(), the file that is created is still left empty.

tTom · March 16, 2023, 12:43am

Hi Albin …
for me it’s not so clear which fitz is meant …
sudo apt install python3-fitz says:
python3-fitz Version (1.19.2+ds1-1ubuntu1)
Now pip3 install pdfCropMargins is running:
Defaulting to user installation because normal site-packages is not writeable
Collecting pdfCropMargins
Downloading pdfCropMargins-1.1.18-py2.py3-none-any.whl (1.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 6.6 MB/s eta 0:00:00
Requirement already satisfied: wheel in /usr/lib/python3/dist-packages (from pdfCropMargins) (0.37.1)
Requirement already satisfied: PyMuPDF>=1.19.6 in ./.local/lib/python3.10/site-packages (from pdfCropMargins) (1.21.1)
Requirement already satisfied: PySimpleGUI>=4.19.0.6 in ./.local/lib/python3.10/site-packages (from pdfCropMargins) (4.60.4)
Requirement already satisfied: pillow>=9.3.0 in ./.local/lib/python3.10/site-packages (from pdfCropMargins) (9.4.0)
Requirement already satisfied: PyPDF2<3.0.0,>=2.11.0 in ./.local/lib/python3.10/site-packages (from pdfCropMargins) (2.12.1)
Installing collected packages: pdfCropMargins
Successfully installed pdfCropMargins-1.1.18

python3 -m pip list
PyMuPDF 1.21.1
PyPDF2 2.12.1

I was manually copying plugin into /.local/share/kicad/7.0/3rdparty/com_gitlab_dennevi_Board2Pdf/ after i have installed installed V1.1 the plugin runs with selected pypdf(slow) option. Thats great! Something new to play with !!!

Many thanks again for all of your awesome support !!!

qu1ck · March 16, 2023, 5:40am

That happens because kicad scripting console remaps stderr when launched.
It’s technically a bug in a lib to assume that stdout/stderr always exist. But it will only show itself in a very specific circumstances like when python program is running inside embedded interpreter that is a windows UI app and therefore doesn’t have a console io streams at all.

You can verify this with this test plugin:

import pcbnew

class TT(pcbnew.ActionPlugin):
    def defaults(self):
        super().defaults()
        self.name = "stdout test"

    def Run(self):
        import sys
        import wx
        wx.MessageBox(f"stdout: {sys.stdout}\nstderr: {sys.stderr}")

TT().register()

If you launch it before opening scripting console you will see both output streams are None. After scripting console is launched, stderr is a valid stream.

Looking at PyPDF4 code, you can pass custom file-like object to PdfFileReader as warndest parameter, these warnings will end up here. If you pass nothing it will default to stderr, which gives you the exception above.
I suggest in your plugin code check if sys.stderr is None and if it is then pass custom warndest which either voids the output or you can use wx.PyOnDemandOutputWindow. Same as the scripting console does: scripting/kipython_frame.cpp · fc6279a2eaf63ff55fcdf8f110a07e0e41be71ae · KiCad / KiCad Source Code / kicad · GitLab

albin · March 16, 2023, 7:04am

Thanks a lot for a great explanation! You really go above and beyond what I could even dare to ask for. That sounds like a great idea and I’ll start looking at this right away.

ciondo61 · March 16, 2023, 8:04am

Hi @albin , these are the versions of the packages :
python3 -m pip list
pdfCropMargins 1.1.18
PyMuPDF 1.21.1
PyPDF2 2.12.1

However I tried your last modification and by selecting the pypdf library the plugin works correctly and I can generate pdf files.

albin · March 16, 2023, 8:29pm

@ciondo61 PyMuPDF 1.21.1 should work fine as long as none of the footprint descriptions contains parentheses. It should also work with KiCad 7.0.0 and the latest testing build. Happy to hear that the pypdf option works for you!

albin · March 16, 2023, 8:41pm

I’ve now pushed a new version that handles warnings caused by this issue. It’s now possible to choose between PyMuPdf and PyPDF4 for coloring and merging independently.

If you’re not able to install PyMuPDF (fitz) on your system, please use PyPDF4 which is already included.

PyPDF4 is able to color the pdfs created by KiCad 7.0.1, even if footprint descriptions contains parentheses. So if you’re using KiCad 7.0.1 and PyMuPdf isn’t working for you, please select PyPDF4 instead.

If you are able to install PyMuPDF 1.21.1, and you’re not using 7.0.1 or your footprint descriptions doesn’t contain any parentheses, then I really recommend using the PyMuPdf alternative. It’s way faster when plotting larger boards!

@AB007, @tTom, @ciondo61, @canemichael97 and @luppa could you please test the latest committed version? Anyone else also of course! But these are the users that I know have been having problems.

It’s not yet available in the PCM, I’d like it to be tested first. Instructions on how to install manually are found here.

AB007 · March 16, 2023, 10:43pm

Yes it’s working now (the slow option) with no error. The fitz version did not work.
I also have a suggestion: the plugin is nice but you should consider the option to arrange all views in the same sheet as long as it is space (or to choose number of pages/ views/ pages or scaling); under each view can be placed a view label (Top view, Bottom view, etc…).

Anyway very good job! looks very nice on my MAC.

Best Regards,
Andrei

tTom · March 16, 2023, 11:52pm

Hi albin,
just installed the new commit, but i can’t choose. The PY radio buttons are all highlighted :

albin · March 17, 2023, 6:32am

@AB007 Thanks for testing! Glad it worked.

If you have fitz installed, it should work with the latest testing build of KiCad. I don’t think there are any testing builds available for MacOS though. For Windows, the testing builds are found here:
https://downloads.kicad.org/kicad/windows/explore/7.0-testing

@tTom Thanks for testing! I’m glad that you found this and that I didn’t push this version to the PCM yesterday. Not sure why it behaves differently on Windows and Linux, but my implementation of the radio buttons where not that great.

I’ve pushed a new commit where this is hopefully solved. I see no difference in Windows. Please test with the latest commit.

ciondo61 · March 17, 2023, 8:20am

Hi @albin , I confirm that the slow version (pypdf4) works correctly, while the fitz version does not work. However pypdf4 is slow but still usable.

Best Regards,
Stefano

tTom · March 17, 2023, 8:45am

Hi @albin ,
its my pleasure, that i’m able to do something here with my small knowledge. I’m still proud of that i can put your commits to the right place for troubleshooting
Now the radio buttons behave as expected, and pdf files were generated (pypdf4), wonderful !!!

tTom · March 17, 2023, 8:50am

… to be completed …
PyMuPdf/fitz generates this Message:

PyMuPdf/fitz works here for merging

albin · March 17, 2023, 9:02am

Thank you @ciondo61 and @tTom!

The message you get Tom is expected if you’re using 7.0.1 and at least one footprint description contains parentheses. It’s solved by downgrading to 7.0.0, upgrading to latest testing build (if there is a testing build for your platform), or by removing all parentheses from all footprint descriptions. Or of course by simply using PyPDF4 until 7.0.2 is released.

I will continue to:

Release 1.2 in PCM.
Update documentation regarding installation of dependencies.
Continue to work in the plugin.

luppa · March 17, 2023, 1:25pm

I cloned the git to /home/luppa/.var/app/org.kicad.KiCad/data/kicad/7.0/3rdparty/plugins/ , seems to be the correct place under Flatpak install, the icon appears in the toolbar and I can start the plugin.

Same problems still continue, the enabled and disabled layers boxes are empty. I can move templates from the Disabled Templates list to the Enabled Templates list, but I cannot disable any (double-clicking doesn’t work, disable-button does nothing).

I can start KiCad from the command line to see the error output, stuff like this comes out:

Traceback (most recent call last):
  File "/home/luppa/.var/app/org.kicad.KiCad/data/kicad/7.0/3rdparty/plugins/Board2Pdf/dialog/settings_dialog.py", line 253, in OnTemplateEdit
    layers.append(pcbnew.BOARD_GetStandardLayerName(i))
AttributeError: module 'pcbnew' has no attribute 'BOARD_GetStandardLayerName'
Traceback (most recent call last):
  File "/home/luppa/.var/app/org.kicad.KiCad/data/kicad/7.0/3rdparty/plugins/Board2Pdf/dialog/settings_dialog.py", line 240, in OnTemplateEdit
    self.SaveTemplate()
  File "/home/luppa/.var/app/org.kicad.KiCad/data/kicad/7.0/3rdparty/plugins/Board2Pdf/dialog/settings_dialog.py", line 381, in SaveTemplate
    "layers": self.layersColorDict,
AttributeError: 'SettingsDialogPanel' object has no attribute 'layersColorDict'

I think I made an unfortunate choice by starting to use Manjaro. Regular Arch already has packets for 7.0.1, Manjaro is still stuck at 6. Haven’t used (KiCad recommended) FlatPak install before this, looks like a bad choice to sandbox all the system libraries out of it. I think I’m going to starting to move my system to something else than Manjaro and using FlatPak, and only after that start trying KiCad plugins.

albin · March 17, 2023, 2:27pm

I have no idea why you’re getting this error. I’ve never tried Flatpaks so unfortunately not much I can do to help.

albin · March 17, 2023, 10:21pm

Board2Pdf v1.2 is now available in PCM

If you have problem installing PyMuPDF (fitz), or if you’re having problem with PyMuPDF due to the bug in 7.0.1, please use the PyPDF4 option instead.

luppa · March 17, 2023, 11:10pm

I’m not sure if this is helpful or not, but if I start up the python scripting console, and write

>>> import pcbnew
>>> pcbnew.BOARD_getStandardLayerName(0)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
AttributeError: module 'pcbnew' has no attribute 'BOARD_getStandardLayerName'
>>> pcbnew.BOARD.GetStandardLayerName(0)
'F.Cu'

albin · March 18, 2023, 8:49am

@luppa Something is not right with your installation. It’s probably better to start a new thread to get help. Include your full version that can be copied from the Help → About menu, describe your flatpak installation and the problems you’re having. Someone with experience of Flatpak installations may be able to help.

system · June 16, 2023, 8:49am

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.