KiBoM - Python BOM Generation tool


#1

Hi all,

For a while I’ve been working on a python BOM generation script, just for my personal use.

I’ve got it to a level where I feel it’s generic enough to be of some use, and I’ve geared it towards being extensible and configurable via a simple config file (no script editing required).

Souce code is available on github - https://github.com/SchrodingersGat/KiBoM - with some examples and documentation.

Currently the CLI version is working well, and I’ve got a GUI version (which runs with WxPython and so does not require any external libraries / software) is on the way.

If anyone wants to suggest or contribute extra features, I’m all ears.


New BOM tool available
Problem with BOM Generation
BOM script to generate a simple list of materials
Import DXF to footprint does not save, KiCAD 4.0.5, bug?
Fields from Schematic Don't Transfer to PCBnew
Components list
Command error. Return code 11 to generate BOM
#2

It looks good thanks.
Good documentation, and sensible choices.
I hope to use it for future projects instead of my unstable scripts.


#3

@SchrodingersGat - You saved my life tonight. Thanks.


#4

Works great!
Thank you so much.


#5

This looks cool.
Suggestions:
a) You could add assembly variants, which work like a string-match type of DNF
Any part with Assembly_Variants field present, checks for supplied name within,
Parts without field, or empty, are on all boards
b) I do not see the DNF string defined, which could be useful, as many variations exists
Some use NF, some use DNF, some tag Do Not Fit…

I think this is currently one way only ? NET -> Report.

It could be useful to allow 2 way, with also a compare/verify pass.

eg Users create a Name.NET, run KiBoM_CLI, then add Distributor part codes for 3 Distis & update voltage and tolerance fields as the part numbers are nailed down.
Some means to pass that changed info back into SCH would be good, as would a re-run that checks a new NET list is OK

I think inserting new field lines into SCH as simple as insert a line ?


#6

Hi Olivier,
I’m using KiBOM and it is very nice!
HTML output is very clear :smiley:
I noticed that if I have i.e. two resistors with the same Value and Footprint, but different Tolerance (i.e. 0.1% and 1%) , they are merged in a single row, which it shouldn’t happen cause they are different components…
Similar behavior should be in case of i.e. capacitors with same Value and Footprint, but different Voltage (i.e. 25V and 50V)…
Is there a configuration option that I’m missing?
Thanks
Maurice


#7

I think that is how it is designed - to allow some tolerance of value, so 0.47k and 470 and 470R are treated the same.
However, it also appears to consider the same - a part with a Digikey supplier field, and one without.
That may be what you meant ? - and I think that is less desirable,
To me Value form tolerance is ok, but only if all other fields match.

Of course, users would likely make them match, in the spreadsheets, then update the SCH and re-do the BOM and now they will correctly merge.
This is why the common request for csv.tsv import or revision path back into SCH, of external spreadsheet revisions.


#8

@maui components are currently matched on the following fields:

  • Part Name / Library
  • Value
  • Footprint

So for example, two parts which are identical in these fields but have different tolerances (or digikey part numbers, etc, etc) will be put into the same group.

I have been thinking about how to improve this. There are two directions I could go:

a) Add the ability for the user to designate extra grouping fields. For e.g. if you have a field named “Digikey” then you can specify that components should be grouped based on this field (as well as all the default ones).

b) Components should be matched based on ALL available field data (so if you have two components with conflicting tolerance values, they will be put into separate groups). I prefer this approach as it means the user does not have to specify.

However, there is a reason I have not implemented this yet - what happens if you have a component with blank fields? As an example case, say you have three resistors, and each component has a “Tolerance” field. One says “1%”, one says “0.1%”, and one is blank. Which group does the blank one go in? Or should a third group be created for the blank one?

The way that I currently specify BoMs for my boards is to select one component within a group, and specify values for various fields. Then all the other components that are grouped with that one get the same field values in the output BoM file. So I only have to specify part numbers (etc) for one component - this makes it very simple to change the part number rather than having to do so for dozens of schematic components.

Having the ability to back-annotate component fields and change those data in the BoM would be very handy for this purpose.

Currently you can work-around the grouping problem by specifying the part value as something like “10K 1%” or “10K 0.1%” (but ideally you would have a tolerance field, or similar).

I’m happy to receive input from any interested parties on this, it’s a non-trivial problem.


#9

Some good thoughts here:

a) I’ve been considering how to implement assembly variants for a while, and I think my approach would be very similar to what you have suggested. I would like to see a field (such as “Assembly” or “variant” or something like that). As you say, if it is blank the part appears in all assemblies. Then you could add tags such as “-production” to indicate that a part will NOT be included in the production assembly, or "+greg to specify that a part will ONLY be loaded if the board is being delivered to Greg. Crazy Greg.

b) DNF checking is currently quite primitive. The script looks at the “Value” field, and also looks for a field called “Notes”. It then checks if these fields are set to any of the following values:

DNF = [“dnf”, “do not fit”, “nofit”, “no stuff”, “nostuff”, “noload”, “do not load”]

(case is ignored)

However this is not yet well documented.

I think there is probably a better way to specify this, and I think that it should be combined with a)

e.g. the script could have a requirement for a “Fit” field to be present, and you can specify “none” or “-all” (etc) for a part to never be fitted.

What are your thoughts on this? I lean towards options that require the least user interaction, and work the same way for everyone. Typing this out now, I am now thinking that the “Fit” field concept would nicely.


#10

Regarding back-annotation to the schematic file, the functionality ‘almost’ exists within the kicad-utils scripts:

In particular the sch.py file allows parsing of a schematic file and it would be fairly easy to take an input CSV file (properly formatted) and update all the part fields.

However, I’m not sure how well this would work if you are working with KiCad open, and it feels pretty hacky - it’s just asking for the file to get overwritten and you lose all your work!

I think this functionality should be included into eeschema.exe personally.


#11

That’s similar to the Assembly Variants rule, which takes a blank field as being equiv the active value (match any), and non-blank are tested for exact match.
In your case, blank is used to mean “can fill with”, and if all other fields match, blanks are auto-filled.
Maybe that becomes a line in the config, for Autofill blanks ?

If there are two different ‘master’ parts, that could still work, if all others have blanks in the fields

example could be 
    R1	100k	1%	0805	Digikey,Mouser,
    R2	100k	1%	0805	,,
    R3	100k	1%	0805	,,
    R4	100k	1%	0805	,,Arrow

becomes 

R1-R4	100k	1%	0805	Digikey,Mouser,Arrow

a future back-BOM, would apply all 3 fields, to all 4 resistors.


#12

I tend to agree - auto matching on ALL fields would seem to present a much cleaner solution (and a much more transparent user experience).

If the script detects that the grouping of a particular component is ambiguous (there are multiple existing groups that it can be placed in) then this can be easily displayed (in the HTML output file) with a particular highlight color meaning that the user has to specify a certain field value for that part.

I’ll have a look into implementing this when I have a moment :slight_smile:


#13

That’s a good link, I think I jumped over that before, because it is called Library-utils.

I think it creates a second SCH file, to avoid the forgot-to-save issue (see path_to_sch2) ?

It has these
sch directory
sch.py: A python class to parse Schematic Files Format of the KiCad.
test_sch.sh: A shell script used to validate the generation of files of the sch class.
add_part_number.py: This script is used to add/edit part numbers fields in the schematic files.
update_footprints.py: This script updates the footprints fields of sch files using a csv as input.

& says

Adding Part Number (PN) to Schematic Files
# first get into sch directory
cd sch
# use the following command to add empty "MPN" fields in the schematic files
./add_part_number.py path_to_sch1 path_to_sch2
# use the following command to add/edit the PN field using the passed csv file
# the default behaviour is search for "Reference(s)" and "MPN" columns in the csv
# the BOM generated by bom_csv_grouped_by_value plugin can be used after
# manually add the MPN field in the Collated Components section (for example)
./add_part_number.py --bom-csv=path_to_bom_csv path_to_sch_files/*.sch
# run the following command to see other options
./add_part_number.py -h

Footprint Checker (wip)
# first get into pcb directory
cd pcb
# run the script passing the files to be checked
./check_kicad_mod.py path_to_fp1 path_to_fp2
# run the following command to see other options
./check_kicad_mod.py -h

#14

OK, that sounds flexible, if a little hidden :wink:

Least interaction and sensible defaults, are always a good idea :slight_smile:

A fit field is the most direct, but some users may want to have one displayed field on the SCH they edit, so for those who want to show it, the tolerant DNF support you show above would seem to cover that ?


#15

Before using KiBOM I was using a script that grouped the elements only if these fields were matching:
Manf#, Voltage, Tolerance, Power, Temperature, Current
In case of blank fields, the element were inserted in a different group…
I would consider useful the ability to configure some extra grouping fields.
Thanks
Maurice


#16

This all seems like a lot of work. And it’s why I do the hard task of parts selection up front with a master parts list. I put a part number in every one of my schematic symbols. A python script post-processes the KiCad BOM against the master parts list, which does basically a one-to-one map from a house part number to a manufacturer part number. (The exception is that it concatenates part values with a base part number for resistors and caps and such.)


#17

not really…
these are my very first scripts I’ve done in python, just before starting coding for kicad StepUp…

Generate a comma delimited list (csv file type).

Components are sorted by ref and grouped by
  Value Footprint  Voltage Tolerance Temp Current
  Fields are (if exist)
        'Ref', 'Qnty', 'Value', 'Footprint', 
        'Manf#', 'Voltage', 'Tolerance', 'Power', 'Temp', 'Current', 'Description'
version 3.3 - 06.2016  [bom_grouped_v33_ky.py]

If I found some missing fields, I just update the schematic and rerun the scripts


#18

I suppose what I don’t see is whether you have manufacturer part numbers in each symbol or not.

If you do, then the other information (tolerance, working voltage, etc) is informative but not necessary for BOM generation.

-a


#19

in many cases (i.e. resistors and capacitors) manufacturer number is not necessary, I prefer having a larger choice for some bare components … In same case the best is price, in other the availability or minimum number requested … that may depend on which type of production I’m going to have…

for ICs normally I have Manf# as a mandatory field
So that is why I use those other info on BOM and I think they are useful
M


#20

I think both approaches have their place, and which you use can even vary on how you want to publish a given design.

That’s why a 2-way BOM system that easily can add-fields to an existing SCH can be useful.

That means someone with a separate master parts spreadsheet, can start there, but they can also update the SCH to supply to some else without needing that master parts spreadsheet.

A good example here would be IC vendors, making eval boards and publishing the designs.
They likely do use some variant of a master parts list BOM manager approach, but they also want their user base to load/harvest/modify any published designs.