To exclude Virtual footprints from BOM [SOLVED]

Piotr · October 15, 2019, 5:43pm

I don’t know Python.
Some months ago after asking here I ended with main loop of my BOM plugin as follows:

=======================
for group in grouped:
refs = “”

# Add the reference of every component in the group and keep a reference
# to the component so that the other data can be filled in once per group
for component in group:
    refs += component.getRef() + ", "
    c = component

# Fill in the component groups common data
out.writerow([refs, len(group), c.getValue(), c.getFootprint().split(":")[1], c.getDescription()])

=======================
How to modify it to exclude from BOM lines containing elements which footprint has “Fabrication Attribute” set to “Virtual”.
I think that before refs = “” could be something like if(…) continue,
or last (out.writerow) line could be some way if()-ed.

Michele_Santucci · October 16, 2019, 10:17am

According to this post https://forum.kicad.info/t/checking-footprint-attibutes-in-python-code/3666/2:

you can do something like this:

attrs = c.GetAttributes()
if attrs == 2:
  out.writerow(....)

Piotr · October 16, 2019, 10:56am

First I have tried c.GetAttributes(), c.GetAttribute(), c.getAttributes(), c.getAttribute() and always I have:
AttributeError: comp instance has no attribute ‘GetAttributes’.
Then I see in post you pointed that GetAttributes() there is called for something received from GetModules.
May be here I should someway get Footprint (Module) for Component first.
I have tried to look into kicad_netlist_reader.py as it is called at the begining of my bom.py but there I found only:
def addAttribute(…
def setAttribute(…
and nothing with get and Attribute.

Don’t knowing Python my searches are rather chaotic

Michele_Santucci · October 16, 2019, 11:27am

Hi, try this:

board = pcbnew.GetBoard()
modules = board.GetModules()

for module in modules:
   ....

   attrs = module.GetAttributes()
   if attrs == 2:
      out.writerow(....)

If you can’t use this approach try to understand what kind of object is group (or grouped set).
Maybe a reference to the full code of the plugin could help.

…

Piotr · October 16, 2019, 12:37pm

In my first post I should paste a few lines more then I did.
Before “for group in grouped:” there is:

# Get all of the components in groups of matching parts + values
# (see ky_generic_netlist_reader.py)
grouped = net.groupComponents()

I can’t find ky_generic_netlist_reader.py at my PC. I think may be it would say see kicad_netlist_reader.py.

I assume group is a set of components (symbols) with the same Value and footprint. As in out.writerow there are used c.xxxx functions to get informations common to all components in group I assume c probably has all information about component so among others there should be its footprint. What I wont to get is not directly component Attribute but Footprint “Fabrication Attributes”. So from c I should somewhay get its footprint and than from footprint its Fabrication Attribute.

I don’t know if by using c.get…() functions in Python you get texts, or object, or may be for Python it is don’t care and Python understand from what you do next what you really needed.

I use little modified bom_csv_grouped_by_value_with_fp.py which is installed with KiCad.
I needed some help to made my cosmetic modifications. I mainly wonted to output footprint name without “library:” at the beginning. I have done that in November 2018 and it was my first and last ‘one day’ contact with Python till now.

Michele_Santucci · October 16, 2019, 12:56pm

I found the definition of component here:

github.com

Dygmalab/pcb-tools/blob/master/kicad_netlist_reader.py

#
# KiCad python module for interpreting generic netlists which can be used
# to generate Bills of materials, etc.
#
# Remember these files use UTF8 encoding
#
# No string formatting is used on purpose as the only string formatting that
# is current compatible with python 2.4+ to 3.0+ is the '%' method, and that
# is due to be deprecated in 3.0+ soon
#

"""
    @package
    Helper module for interpreting generic netlist and build custom
    bom generators or netlists in foreign format
"""


from __future__ import print_function
import sys

This file has been truncated. show original

Look for comp class.

I cannot find a method that retrieve module attributes in this class or even in libpart hence you cannot infere if the module is virtual or not in an easy way.

Piotr · October 16, 2019, 2:00pm

Till now I assumed kicad_netlist_reader.py as some library code - not for my modifications. But from part at the beginning I understand that I can in that file specify Values and Footprints to be not included in BOM.
But modifying that file whenever I add new hole footprint to my library is not good solution - I will certainly forget.
The other way described there is to add to my symbol the ‘Installed’ field and give it a value ‘NU’.
I have done it for one element in symbol library, and then updated elements at schematic but generated then BOM contained that elements.

I see in kicad_netlist_reader.py:
if c.getField( “Installed” ) == ‘NU’:
exclude = True

It looks that element is then added to return list only if not exclude.
So I don’t understand why my elements were added even they had field Installed with value NU.
I have checked several times that I didn’t mix any letter.
I am not able to run that script in debugging to find what is going on when get to these lines.

Piotr · October 16, 2019, 2:28pm

I have noticed that function groupComponents() used in bom_csv_grouped_by_value_with_fp.py don’t calls the function getInterestingComponents() in which are those test for exclude.
So it looks that I should modify my version of bom_csv_grouped_by_value_with_fp.py.
The groupComponents has a second parameter. It looks that if I give there a subset of components (probably the result from getInterestingComponents() ) then it will be used (if it is not given then self.components is used. I don’t understand the self parameter which is then not specified when groupComponents is called in bom_csv_grouped_by_value_with_fp.py.
What should I write in call of groupComponents as a first parameter if I wont to use getInterestingComponents as a second parameter and should I use anything as a ‘self’ parameter of getInterestingComponents.

Piotr · October 16, 2019, 2:59pm

Replacing:
gropued = net.groupComponents()
with
grouped = net.groupComponents(net.getInterestingComponents())

gives me BOM without elements marked by field Installed set to NU.

But what is surprising for me. Now I also have elements in each line sorted while previously not.
The function groupComponents() seems to also try to sort elements but using function sorted() and not xxx.sort().

qu1ck · October 16, 2019, 8:13pm

It’s impossible to help you unless you post a link (or attach) the bom plugin you are using.
From what you wrote here so far I can not tell if kicad_netlist_reader.py even has anything to do with it at all.

Also consider using pre-made bom plugins that are quite powerful in terms of various output capabilities and already handle things like skipping virtual components.
See user plugins list at the bottom of the post here:

bobc · October 16, 2019, 8:33pm

You can’t do that with kicad_netlist_reader. That module reads the netlist from an XML file generated from eeschema. It does not have footprint properties in it.

You can get things like fabrication attribute if you write a python script for pcbnew.

Piotr · October 17, 2019, 9:58am

You probably missed that line:

After my yesterday experiments I added my comment to one bug about the wrong sorting of references (marked as fixed, but in my opinion not fixed). But was asked there to report a new bug.
So: https://bugs.launchpad.net/kicad/+bug/1848472
I hope that those of you who knows Python can find what is a source of problem. For me it looks that probably sorted () function has a bug in it or need be called differently than is called in groupComponents().
Could you tell me if my replacing in groupComponents() the line:
g = sorted(g, key=lambda g: sortKey(g.getRef()))
with:
g.sort(key=lambda g: sortKey(g.getRef()))
what make the sorting being correct can be a source of new problems?

Piotr · October 17, 2019, 10:00am

I’ve got it in meantime

bobc · October 17, 2019, 10:13am

This is very confusing to follow. First we were talking about virtual footprints now we are talking about sorting. Can we confirm that the code under discussion is

github.com

KiCad/kicad-source-mirror/blob/master/eeschema/plugins/python_scripts/kicad_netlist_reader.py#L667-L678


        groups.append(newgroup)

# The key to sort the components in the BOM
# This sorts using a natural sorting order (e.g. 100 after 99), and if it wasn't used
# the normal sort would place 100 before 99 since it only would look at the first digit.
def sortKey( str ):
    return [ int(t) if t.isdigit() else t.lower()
            for t in re.split( '(\d+)', str ) ]

for g in groups:
    #g = sorted(g, key=lambda g: sortKey(g.getRef()))
    g.sort(key=lambda g: sortKey(g.getRef()))

Piotr · October 17, 2019, 10:20am

Those are probably “too good” for me. At list now.
If now I have references sorting correct.
Even not correcting kicad_netlist_reader.py, but only replacing in my bom.py (copy from bom_csv_gropued_by_value_fp.py) the line:
gropued = net.groupComponents()
with
grouped = net.groupComponents(net.getInterestingComponents())
I got correct sorting (I think references are sorted twice and one is correct, and second probably does nothing).
And I would have unneeded elements deleted (by adding field ‘Installed’ with ‘NU’) then I get exactly as I wonted to use with my spreadsheed used elements database.

Now I’m not ready to move that database to KiCad symbol library or anywhere else. It is simple and works for me since many years - need not to change.
I suppose that using those user plugins BOM generators I would have a problems with getting what I wont.

bobc · October 17, 2019, 10:25am

Can we mark this “solved” then?

Piotr · October 17, 2019, 10:39am

Yes we went that road.

I wonted to delete from BOM virtual components (just as I am in first time configuration that tool to save me a need of deleting each time few rows from spreadsheet later).
I was showed the kicad_netlist_reader.py I have never look inside before.
I found comment there at the beginning how to exclude elements form BOM.
It didn’t worked untill I have changed one line in bom_csv_gropued_by_value_fp.py.
I had my problem solved not exactly as I wonted but I decided to add field ‘Installed’ to my symbols not to be in BOM.
I noticed that at the same time with deleting unneeded components I got all rest sorted correctly (before I just supposed that KiCad bom generators not sort references).
I found that groupComponents tries to sort references so why it fails when getInterestingComponents succeeds.
The only difference I noticed is to call class method (succeed) or external function (failed)
I have changed that call in groupComponents and it works, but I really know nothing about Python - for example why I need not to write anything as parameter ‘self’ even it is in function definition.

Yes.

Piotr · October 17, 2019, 10:41am

Yes.
I have never modified the title of my threads. I will look how people add that.

system · January 15, 2020, 10:41am

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.