Why is generation of the BOM such a nightmare?

Schematic
22

That’s a nice tip !! I didn’t know that that XML was an intermediate output. I always saw it there and just completely ignored it, and often just erased it (always wondering: “why does this plugin that is supposed to output the BOM in CSV, outputs it in two different formats?”)

I just quickly opened one, and yes, I from a quick look it seems like a complete report of the list of components. I got started with a quick program that takes as input the “Individual components” section of the generated CSV; and it is a bit of a pain, reading and parsing through it. So, maybe the XML is easier and safer (i.e., less error-prone)

Thanks again for the tip !

23

Huh. Maybe a bug that has been fixed some time between 5.0 and the current 5.1.x ?

I first thought that maybe all of the cases where the “group by value” failed were cases of capacitors that I indicated with a / (e.g., 0.1uF / 16V). But no, I see other cases where this happens. Again, it does look like the “group by value” is not doing group by value, and instead it is doing “group by value,footprint” correctly (as far as I can see, there are never two separate lines that have the same value and same footprint).

Again, the problem for me is: this behaviour (group by the combination value,footprint) doesn’t work for me in some (maybe all?) of my projects.

24

I was sure that it have to be such till v6. So I didn’t checked if I can went back to my standard element values with / in them.

I don’t understand that sentence:

  1. something don’t work or
  2. you don’t wont to have elements sorted by value and footprint.

You try plugin sorting only by value so may be 2.

25

Had not planned to further my posts on this but, after reading the above useful comments and spending a few minutes reviewing the various BOM format/plugins, I decided to redo my BOM plugin.

For my needs, I need only a list of the Ref, Value and Footprint.

So, considering that and, how I use BOM’s, it made sense to use a CSV format and simply Tweak the ‘bom2csv’ plugin.
I further enhanced the process by making a Macro in LibreOffice (Calc) to format it for my needs.

FYI- Libre 6.4 has documented macro “Bug’s” and only Mouse, Arrow and Enter keys are reliable for Cell recording.

Anyway, the BOM process was simplified to basically a few mouse clicks:

  1. Net>myBom2csv
  2. Double-Click the exported .CSV to load into Libre
  3. Run myMacro to Format it
  4. Save and/or Export PNG of selected cells (I use the PNG in tracking database)

Image of PNG result for a 5 item test schematic
(looks like Libre had a hiccup with Bold fonting and Boxing second-row…).Got to love Open-Source!!

BOM_myTweaked_bom2csv

26

it took me 1 min to change the script to output the file the way you wanted and 10min to create a test schematic :stuck_out_tongue:

I am attaching the modified script below.

(there is a custom comparison function that compares value, footprint and partname, I just commented footprint and partname out, lines 35 to 38 from the script.)

image
image942×497 2.84 KB

image
image948×560 15.4 KB
bom_csv_grouped_just_by_value_semicolon.py (5.7 KB)

1 Like
27

Reading it again, I certainly see the ambiguity — I meant to say “doesn’t work for me” in the sense of “it doesn’t do” (specifically, given the way my designs are done, that approach cannot produce a “correct” BOM).

The thing is, I may have the same part (say, some 0.1uF decoupling cap) at several places, with different footprints (e.g., the native C_0603_xxxxmetric and another one with a custom, more compact footprint, as per IPC-7351 for high-density of components). Then, there would be two different lines in the BOM, both having the same actual part number.

So, really, the only way (at least for me, given the way my libraries and projects are built/organized) that I can get a correct BOM is to do a group by Manufacturer Part Number (well, or by Digikey part number).

As far as I can tell, the bom_csv_group_by_value plugin correctly groups by the combination (value,footprint) … Which: (1) does not work (is not useful) for me; and (2) is incorrect in the sense of not doing what it advertises — the plugin says that it groups by value, but it doesn’t: it actually groups by the combination (value,footprint).

It is indeed pretty obvious from the code (thanks @der.ule for pointing it out, and for the modified script!):

result = True
if self.getValue() != other.getValue():
    result = False
#elif self.getPartName() != other.getPartName():
#    result = False
#elif self.getFootprint() != other.getFootprint():
#    result = False*

I should more precisely say: it looks like the script (without the commented lines, which I assume are the modifications by der.ule) actually groups by the combination (value,partnumber,footprint) … I guess part number is either a field that I don’t use, or it always has a value that is strictly a function of the field Value, or the field footprint?)

And yes, I have to admint: hate Python as much as one wants, a quick look at that script makes it very obvious what’s going on and how to modify it :‒)

Man, I feel like “the new guy” at the AA meetings (at least the way they show them in TV or the movies … it’s not like I know what those meetings look like in real-life, right? :‒) ) …

Me: Hi KiCAD forum… my name is Carlos, and I am pathologically Python-phobic …
All: HI CARLOS!!!

;‒)

Cheers!

1 Like
28

Maybe a bit late but there is a very easy way in kicad 5.1 to get reasonably good BOM output without ever going through the BOM tools. Just open the field editor, group as you desire and then select all, ctrl+c and ctrl+v in any spreadsheet application of your liking.

And the 3 fields you list are also easily exported from pcbnew.
For all options that i know about see How to create a bill of materials (BOM)?

1 Like
29 
if python_phobic:
    pass #but remember 4 spaces first
1 Like
30

I have never considered such solution so till now I didn’t understood your problem with grouping by value and footprint.
I think that for manufacturer such BOM with the same element in several lines will be not a problem.
I sometimes put 0pF capacitor in paralel to other when I expect that after checking the actual parameters of designed PCB (RFID antenna) I may be will have to use two capacitors.
So in the first BOM I for example can have 4 rows:

  • 0pF as a free place
  • 12pF for quartz
  • 33pF - some RC filter
  • 47pF - first shot in resonant circuit

After measurements I decide that 47pF would be better 45pF. My manufacturer prefers to get changes then to get the whole new documentation (less rows in BOM - he then assumes he have to check whole his documentation).
So after telling him changes we end with BOM containing:

  • 12pF (replaced for 0pF)
  • 12pF
  • 33pF
  • 33pF (new value for 47pF)

What I modified in plugin first was that footprint was exported including library name and I wonted only footprint. I asked here and qu1ck helped me how to modify plugin to remove the library part of footprint name.
If you distinguish footprints for the same element sizes in the way simple to be removed from footprint name than may be simple modification of plugin …value_with_fp (or only value as it looks is doing the same) can made it grouping such elements in the same row at the same time having 1k 0603 and 1k 0402 in separate rows.
I also know C++ (rather C with classes as I spend programming about 2 weeks per year) and don’t know Python. But I don’t have Python-phobic. I’d like to learn it but time is not made of rubber :frowning:

31

There’s a line missing in your code:

if python_phobic:
    remember the colon at the end of the IF statement
    remember 4 spaces firs

:‒)

32

They can certainly manage, but at least with one manufacturer, I once experienced an unusual delay in getting a response from them after I sent them my Request for quote; they told me that they struggled parsing through the BOM because of duplicate component lines. (that BOM in particular was somewhat large, and the amount of “duplicates” was very high).

So, as a matter of principle, I now aim to always having a clean BOM (“clean” defined from the perspective of no duplicate lines)

Yea, I joke about it (pathological phobia to Python and all), but part of it is the same issue: I never can find the time to sit and learn it, even though in my case, learning it should be pretty quick, as I have had a lot of experience and spent lots of time programming in the past (mostly in C and C++, but also some Visual Basic, some Pascal (in ancient times), bash scripting, Matlab scripting, some (very little) PHP (hate it!), and some (very little) JavaScript… (also hate it, of course! :‒) )

33

I stumbled and stumbled with the BOM making Until I Read The Manual. Then, all became clear and I saw success in about two minutes…

Apparently there are different manual versions. Mine is a PDF dated Nov 17 2019. Here’s a snapshot of the Contents. Truly, Read the Manual!

Screen Shot 2020-10-01 at 8.14.32 AM
Screen Shot 2020-10-01 at 8.14.32 AM1612×542 56 KB

34

In ancient times we were learned to program in Algol. About 1991 we (with my brother) defined our simple language based on Forth. It solved as important problem of no way to upgrade device software those times (we managed to fit program+stack+data buffer in 118 RAM bytes of 8051 (10 bytes left for uC stack). Our devices based on it were in our offer till 2004.

35

Huh… I was sure I had tried that in the past and didn’t work (maybe some other scenario where the copy-n-paste doesn’t work well, and I kept the idea that it doesn’t work?)

Actually, I’m just trying, and I see that one issue is the ... that shows when there are different footprints and/or different values for a given part (although, that is a problem no matter what the approach)

Another thing is: when pasting, I get a dialog to indicate how to separate the values; it looks like selecting TAB as the ONLY delimiter, and unchecking “Merge Delimiters” was the only way to make it work. Can you confirm that this is correct?

Also, another mystery (not so much a real problem; but certainly odd behaviour): in the Edit fields dialog, I am NOT showing the Datasheet field (it is unchecked, and there is no column showing it). However, I select everything, copy and paste into LibreOffice Calc, and there is the column for the Datasheet field … Riddle me that!! :‒)

36

Hahahaha!!! Algol, of course !!! I found myself trying to learn it, something like a year ago (out of personal fun/interest)… after no more than two hours and the first maybe ten pages of the document I was looking at, I gave up (big realization that: hey, we are well into the 21st century!!! :‒) )

Good times, good times !!!

37

About all I can add is this:

Let’s say you want a BOM in .csv format to load into Libre/other spreadsheet.

Reading the manual I referred to, you’ll find this Important Statement: “The currently best XSLT style-sheet for BOM generation at this time is called bom2csv.xsl”

To me, that suggests using it as a starting point to hack. (That is just what I did.)
Looking at the file you’ll see two sections containing the relevant items to change/delete. Particularly, as an example; “Datasheet”

With your gained knowledge, you should be able to get a clean BOM without difficulty.

You can take aspects of that knowledge and apply it to solve formatting of the other BOM plugin’s…

Screen Shot 2020-10-01 at 9.50.25 AM
Screen Shot 2020-10-01 at 9.50.25 AM1680×514 83.1 KB

38

First off, #ihatepythontoo (so nice to have friends!)

A lot of the BOM issues can be helped with some discipline during schematic entry. For me, coming from 30 years of professional EDA, this has become habit - I always add a manufacturers part number to every component as it gets added to the schematic. Atomic libraries make this a whole lot faster - but thats another topic.

For the examples cited above where there are multiple footprints used for a single part type, I would suggest adding a “case” or “package” field to the component, which holds the package style of the part independent of the footprint used in the pcb, Then you could group your BOM based on value+package, and still have multiple footprint styles for that part.
E.g.
value , package, footprint
100nF, 0603, 0603L
100nF, 0603, 0603_handsoldered

would be grouped together

1 Like
39

Maybe it does not directly help you but I’m using PHP in my web application to read KiCad BOMs into the database. I’ll open a new thread for this.

40

Somebody else may have fullfilled your wish already :slight_smile:

41

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.