KiBoM - Python BOM Generation tool

External Plugins
1

Hi all,

For a while I’ve been working on a python BOM generation script, just for my personal use.

I’ve got it to a level where I feel it’s generic enough to be of some use, and I’ve geared it towards being extensible and configurable via a simple config file (no script editing required).

Souce code is available on github - https://github.com/SchrodingersGat/KiBoM - with some examples and documentation.

Currently the CLI version is working well, and I’ve got a GUI version (which runs with WxPython and so does not require any external libraries / software) is on the way.

If anyone wants to suggest or contribute extra features, I’m all ears.

6 Likes
2

It looks good thanks.
Good documentation, and sensible choices.
I hope to use it for future projects instead of my unstable scripts.

3

@SchrodingersGat - You saved my life tonight. Thanks.

1 Like
4

Works great!
Thank you so much.

5

This looks cool.
Suggestions:
a) You could add assembly variants, which work like a string-match type of DNF
Any part with Assembly_Variants field present, checks for supplied name within,
Parts without field, or empty, are on all boards
b) I do not see the DNF string defined, which could be useful, as many variations exists
Some use NF, some use DNF, some tag Do Not Fit…

I think this is currently one way only ? NET -> Report.

It could be useful to allow 2 way, with also a compare/verify pass.

eg Users create a Name.NET, run KiBoM_CLI, then add Distributor part codes for 3 Distis & update voltage and tolerance fields as the part numbers are nailed down.
Some means to pass that changed info back into SCH would be good, as would a re-run that checks a new NET list is OK

I think inserting new field lines into SCH as simple as insert a line ?

6

Hi Olivier,
I’m using KiBOM and it is very nice!
HTML output is very clear :smiley:
I noticed that if I have i.e. two resistors with the same Value and Footprint, but different Tolerance (i.e. 0.1% and 1%) , they are merged in a single row, which it shouldn’t happen cause they are different components…
Similar behavior should be in case of i.e. capacitors with same Value and Footprint, but different Voltage (i.e. 25V and 50V)…
Is there a configuration option that I’m missing?
Thanks
Maurice

7

I think that is how it is designed - to allow some tolerance of value, so 0.47k and 470 and 470R are treated the same.
However, it also appears to consider the same - a part with a Digikey supplier field, and one without.
That may be what you meant ? - and I think that is less desirable,
To me Value form tolerance is ok, but only if all other fields match.

Of course, users would likely make them match, in the spreadsheets, then update the SCH and re-do the BOM and now they will correctly merge.
This is why the common request for csv.tsv import or revision path back into SCH, of external spreadsheet revisions.

8

@maui components are currently matched on the following fields:

  • Part Name / Library
  • Value
  • Footprint

So for example, two parts which are identical in these fields but have different tolerances (or digikey part numbers, etc, etc) will be put into the same group.

I have been thinking about how to improve this. There are two directions I could go:

a) Add the ability for the user to designate extra grouping fields. For e.g. if you have a field named “Digikey” then you can specify that components should be grouped based on this field (as well as all the default ones).

b) Components should be matched based on ALL available field data (so if you have two components with conflicting tolerance values, they will be put into separate groups). I prefer this approach as it means the user does not have to specify.

However, there is a reason I have not implemented this yet - what happens if you have a component with blank fields? As an example case, say you have three resistors, and each component has a “Tolerance” field. One says “1%”, one says “0.1%”, and one is blank. Which group does the blank one go in? Or should a third group be created for the blank one?

The way that I currently specify BoMs for my boards is to select one component within a group, and specify values for various fields. Then all the other components that are grouped with that one get the same field values in the output BoM file. So I only have to specify part numbers (etc) for one component - this makes it very simple to change the part number rather than having to do so for dozens of schematic components.

Having the ability to back-annotate component fields and change those data in the BoM would be very handy for this purpose.

Currently you can work-around the grouping problem by specifying the part value as something like “10K 1%” or “10K 0.1%” (but ideally you would have a tolerance field, or similar).

I’m happy to receive input from any interested parties on this, it’s a non-trivial problem.

9

Some good thoughts here:

a) I’ve been considering how to implement assembly variants for a while, and I think my approach would be very similar to what you have suggested. I would like to see a field (such as “Assembly” or “variant” or something like that). As you say, if it is blank the part appears in all assemblies. Then you could add tags such as “-production” to indicate that a part will NOT be included in the production assembly, or "+greg to specify that a part will ONLY be loaded if the board is being delivered to Greg. Crazy Greg.

b) DNF checking is currently quite primitive. The script looks at the “Value” field, and also looks for a field called “Notes”. It then checks if these fields are set to any of the following values:

DNF = [“dnf”, “do not fit”, “nofit”, “no stuff”, “nostuff”, “noload”, “do not load”]

(case is ignored)

However this is not yet well documented.

I think there is probably a better way to specify this, and I think that it should be combined with a)

e.g. the script could have a requirement for a “Fit” field to be present, and you can specify “none” or “-all” (etc) for a part to never be fitted.

What are your thoughts on this? I lean towards options that require the least user interaction, and work the same way for everyone. Typing this out now, I am now thinking that the “Fit” field concept would nicely.

10

Regarding back-annotation to the schematic file, the functionality ‘almost’ exists within the kicad-utils scripts:

In particular the sch.py file allows parsing of a schematic file and it would be fairly easy to take an input CSV file (properly formatted) and update all the part fields.

However, I’m not sure how well this would work if you are working with KiCad open, and it feels pretty hacky - it’s just asking for the file to get overwritten and you lose all your work!

I think this functionality should be included into eeschema.exe personally.

11

That’s similar to the Assembly Variants rule, which takes a blank field as being equiv the active value (match any), and non-blank are tested for exact match.
In your case, blank is used to mean “can fill with”, and if all other fields match, blanks are auto-filled.
Maybe that becomes a line in the config, for Autofill blanks ?

If there are two different ‘master’ parts, that could still work, if all others have blanks in the fields

example could be 
    R1	100k	1%	0805	Digikey,Mouser,
    R2	100k	1%	0805	,,
    R3	100k	1%	0805	,,
    R4	100k	1%	0805	,,Arrow

becomes 

R1-R4	100k	1%	0805	Digikey,Mouser,Arrow

a future back-BOM, would apply all 3 fields, to all 4 resistors.

12

I tend to agree - auto matching on ALL fields would seem to present a much cleaner solution (and a much more transparent user experience).

If the script detects that the grouping of a particular component is ambiguous (there are multiple existing groups that it can be placed in) then this can be easily displayed (in the HTML output file) with a particular highlight color meaning that the user has to specify a certain field value for that part.

I’ll have a look into implementing this when I have a moment :slight_smile:

13

That’s a good link, I think I jumped over that before, because it is called Library-utils.

I think it creates a second SCH file, to avoid the forgot-to-save issue (see path_to_sch2) ?

It has these
sch directory
sch.py: A python class to parse Schematic Files Format of the KiCad.
test_sch.sh: A shell script used to validate the generation of files of the sch class.
add_part_number.py: This script is used to add/edit part numbers fields in the schematic files.
update_footprints.py: This script updates the footprints fields of sch files using a csv as input.

& says

Adding Part Number (PN) to Schematic Files
# first get into sch directory
cd sch
# use the following command to add empty "MPN" fields in the schematic files
./add_part_number.py path_to_sch1 path_to_sch2
# use the following command to add/edit the PN field using the passed csv file
# the default behaviour is search for "Reference(s)" and "MPN" columns in the csv
# the BOM generated by bom_csv_grouped_by_value plugin can be used after
# manually add the MPN field in the Collated Components section (for example)
./add_part_number.py --bom-csv=path_to_bom_csv path_to_sch_files/*.sch
# run the following command to see other options
./add_part_number.py -h

Footprint Checker (wip)
# first get into pcb directory
cd pcb
# run the script passing the files to be checked
./check_kicad_mod.py path_to_fp1 path_to_fp2
# run the following command to see other options
./check_kicad_mod.py -h
14

OK, that sounds flexible, if a little hidden :wink:

Least interaction and sensible defaults, are always a good idea :slight_smile:

A fit field is the most direct, but some users may want to have one displayed field on the SCH they edit, so for those who want to show it, the tolerant DNF support you show above would seem to cover that ?

15

Before using KiBOM I was using a script that grouped the elements only if these fields were matching:
Manf#, Voltage, Tolerance, Power, Temperature, Current
In case of blank fields, the element were inserted in a different group…
I would consider useful the ability to configure some extra grouping fields.
Thanks
Maurice

17

not really…
these are my very first scripts I’ve done in python, just before starting coding for kicad StepUp…

Generate a comma delimited list (csv file type).

Components are sorted by ref and grouped by
  Value Footprint  Voltage Tolerance Temp Current
  Fields are (if exist)
        'Ref', 'Qnty', 'Value', 'Footprint', 
        'Manf#', 'Voltage', 'Tolerance', 'Power', 'Temp', 'Current', 'Description'
version 3.3 - 06.2016  [bom_grouped_v33_ky.py]

If I found some missing fields, I just update the schematic and rerun the scripts

19

in many cases (i.e. resistors and capacitors) manufacturer number is not necessary, I prefer having a larger choice for some bare components … In same case the best is price, in other the availability or minimum number requested … that may depend on which type of production I’m going to have…

for ICs normally I have Manf# as a mandatory field
So that is why I use those other info on BOM and I think they are useful
M

20

I think both approaches have their place, and which you use can even vary on how you want to publish a given design.

That’s why a 2-way BOM system that easily can add-fields to an existing SCH can be useful.

That means someone with a separate master parts spreadsheet, can start there, but they can also update the SCH to supply to some else without needing that master parts spreadsheet.

A good example here would be IC vendors, making eval boards and publishing the designs.
They likely do use some variant of a master parts list BOM manager approach, but they also want their user base to load/harvest/modify any published designs.

21

Hello!
I would like to see matching of components on their Manufacturers part numbers, MFPN. This is generally considered a good thing to add. I wish it was a default field in Kicad already, now i add it manually, but since it is a bit tedious for resistors and such, could their be some kind of configuration, so that i migh manually add which fields to group on.

That way i could use the MFPN for most parts, and leave tolerance and such empty, for resistors i could instead leave MFPN empty, If i then create my groupings based upon both MFPN and Tolerance for example, i would get the groupings i wanted.

Thanks for your script and effort anyway!

22

Hey all,

I have made some improvements to the script (the README.md file explains them all) https://github.com/SchrodingersGat/KiBoM

Basically:

a) Grouping columns can now be specified by the user in the .ini preferences file (you can even remove the default grouping columns if required)

b) The regex matching now works on a component level. You can specify various regex to determine if an individual component is included OR excluded (read the docs!)

c) The logic by which a part is “fitted” or “not fitted” is now simplified and explained

d) You can specify that individual components will be included or excluded with “configurations” i.e. only load some components in the “Production” build. This is all laid out in the docs.

e) You can specify in the .ini file if blank fields should count as a “conflict” (making a new line in the BoM) or if they will be merged with a non-blank field where all the other fields match

3 Likes