Version 7 ASCII representation of files (versus version 5)

Schematic
1

I have just migrated to version 7 after using version 5 and earlier.

I had, several years back, written a C++ program to open up and create BOMs from the schematic. I embed stock codes within symbols, other than things like 0603 resistors caps etc (as I don’t want to have to have separate symbol for each value of cap, resistor etc). These are calculated from within my C++ code by a lookup file of symbol and value.

However, I’m having a problem with version 7 with the complete rewrite of how the database is represented by ASCII. There’s a thing where sometimes the ref des is in this line:

(property “Reference” “Cxxx” etc etc

Where xxx is the reference designator.

However, sometimes the reference designator is exasperatingly not in that line and it’s in a line like this:

(reference “X201”) (unit 1)

which is in the “(instances” section of the symbol.

I can’t just look for the latter line because, for some reason, sometimes, there are multiple values in the instance section of symbols; examples that aren’t actually in the schematic at all. :crazy_face:

Does anyone know how to force the ref des that appears in the schematic within kicad itself to put the proper reference designator figure in please? Or how to get rid of the instances where they are not in the actual schematics?

The former is MUCH preferred though.

As an example, all three symbols here do not have their ref des in the

(property “Reference”

line. They have things like:

(property “Reference” “X?” (at 257.81 109.22 90)

image

Thanks!

2

You’ll need to incorporate a S-expression parser in your code. Line separators are just white space.

3

Sorry, I’m not following.

4

In the new format library symbols are embedded into the schematic file and then instantiated separately. Maybe this is so-and-so manageable with one schematic file, but if you have hierarchical sheets, it’s even more difficult.

I very, very strongly suggest trying the KiCad’s own BOM system and handling the result programmatically if needed. Is there any specific reason why you don’t use the BOM system of KiCad itself?

There also may exist several plugins which can create BOM.

5

Using the built-in BOM is a non-starter. As is existing plug-ins, due to my look-up method for caps and resistors.

The thing with the instantiation is it seems to be sometimes one way and sometimes another. That wouldn’t be a problem in itself it wasn’t for the fact it’s including instantiations in the file that aren’t on the schematic.

6

That was then, now is now.
KiCad has changed quite a lot since V5. The schematic and PCB are now based on S-expressions which is a standardized format and I guess that reading such a file into memory is just a few lines of calls to a library function. Sure, you can write your own, but that is probably not a good idea.

There is also some documentation available for KiCad’s file formats.

Another big change is that KiCad now has support for database driven libraries. I do not know what your

looks like, but maybe it’s time to start thinking about some serious maintenance / change. You can of course continue in your own world and do your own thing, but there will not be many people willing to spend time and effort to help you with that. If you switch to “standard” ways of doing things in KiCad, then the code you use will already be there (with a bit of luck) and be maintained by some group as it evolves together with KiCad. Maybe “short term” getting your own method working again is less work, but what if KiCad changes again?

1 Like
7

As I said, using off-the-shelf methods is 100% a non-starter. My method is far too tied in to how my sql database which stores all parts lists, stock codes etc etc to change it. It’s a method used for thirty years through two different CAD systems; that cannot change.

The thing is, the way this is all put together under the hood is inconsistent. Parsing the file is really simple, but sometimes the ref des is in one place in the symbol instance and sometimes in a completely different place. As I say, worst of all, it contains symbols which aren’t even the schematic. I could relatively easily have a work round so if the ref des wasn’t in one part of the instance I take it from another. The fact it has these rogue instances is what makes it impossible.

8

No, that is not “worset of all”. It’s by design. In KiCad V5 graphics for symbols was cached in a [Project]-cache.lib file, and starting from KiCad V6 this graphics is embedded in the schematic file itself, and this is a great improvement. Many projects got damaged in the past because the graphics of schematic symbols got “lost” in one way or another.

The description of the schematic file format to which I linked in my previous post is not very comprehensive, but it’s long enough to show the basic structure of the new schematic file format.

9

If you already have a database then it is probably much easier to have kicad use that database of parts instead of whatever you are doing but if you insist on not going that route…

Then make sure you understand the file format first.

Yeah. Because the first does not describe ref of the component, it describes the symbol’s text property by the name of “Reference”, it’s position, font, effects, etc.
The latter describes the reference of the instances of the symbol and it is always there. But you need to pay attention to the path of the instances if your subsheet is used in other projects too. It will show parts that “are not there” because they are not in your current project. For example here is a list of instances for a subsheet that is used 2 times in 7.0_test and 1 times in 7.0_test2

    (instances
      (project "7.0_tests2"
        (path "/5d4f03cc-fc07-4304-a207-fb51e07bd8fc/62e163e1-5d4e-4b9f-8166-fc2291494ea6"
          (reference "SW1") (unit 1)
        )
      )
      (project "7.0_tests"
        (path "/7d86b326-d60d-4470-8d88-33c9ad614d82/96d18a47-c45a-4bf5-8da8-61b8c9a47ff7"
          (reference "SW1") (unit 1)
        )
        (path "/7d86b326-d60d-4470-8d88-33c9ad614d82/1f2fbde8-99fd-44dd-a414-ac486cc2e98d"
          (reference "SW2") (unit 1)
        )
      )
    )
10

I don’t think you’re following.

I’m saying there’s, for example, C1003 as an instance of a component that is not in the schematic. At all. Anywhere.

11

OK, I think I might have worked out how to simply get round this. I will have to look at the instances section of the symbol, but I’ll have to do a second check to make sure the instance is this project.

Here’s a problem examples from within a “(symbol (lib_id” section:

(instances
  (project "motor_cpu3"
    (path "/94f3375c-5612-466a-9a4d-164b25da23b3/00000000-0000-0000-0000-00006067faa0"
      (reference "C404") (unit 1)
    )
    (path "/94f3375c-5612-466a-9a4d-164b25da23b3/00000000-0000-0000-0000-000060be46a7"
      (reference "C804") (unit 1)
    )
    (path "/94f3375c-5612-466a-9a4d-164b25da23b3/00000000-0000-0000-0000-000060bf3dbb"
      (reference "C1003") (unit 1)
    )
    (path "/94f3375c-5612-466a-9a4d-164b25da23b3"
      (reference "C1003") (unit 1)
    )
  )
  (project "BH0041"
    (path "/da601d62-5c26-4004-8fef-7aef55726f57/38bf4ae0-7541-4658-b16a-603639e73354"
      (reference "C601") (unit 1)
    )
  )
)

The instances that are causing the issue here are everything in project “motor_cpu3”. There is, and never has been, a motor_cpu3 project. There was, in the previous issue of this board a sheet called that, but it was removed from this iteration of the project. So, for some reason, kicad held on to the deleted instances.

What I do want is the instance in BH0041. So, I think that when I’m parsing the instances for each symbol, if I make sure it’s only including the ones for the actual project being parsed and not these phantom ones I’ll be cool.

12

If you ever open the schematic of that sheet in a standalone mode (i.e. by double clicking the file in explorer) instead of opening it through the project it belongs to, then KiCad will create a temporary project by the name of the sheet (same with pcb files). The instances you see are likely remnants of someone doing just that and then reannotating the partial schematic.

13

That’ll be me then! :smile: I’ll’ve likely opened a sheet to copy and paste a section.

Time for bed now, but I’ll try that extra check in the parsing tomorrow and see if that does it for me, but am hopeful it will.

Thanks for the help, everyone.

14

Hello, I am facing a similar problem. I prefer to program in Python, it’s much easier and you don’t have to solve ugly low level problems. Of course, Python has worse performance compared to C++, but in the case of BOM generation, I think it doesn’t matter if it takes a fraction of a second or several seconds. Here on the forum I found a link to a nice kiutils library that solves S-expression stuff.

Here is a link to my thread

There is nothing stopping you from rewriting Python to C++ …

16

OK, have sorted it by, when parsing a symbol, only pulling the ref des out when it matches the current project and it’s all working nicely now.

Thanks for that, that’s kind of you.

I struggle with Python, finding C++, and especially Qt’s implementation, way faster to write applications with. That’s likely partly because I know C and C++ so much better, but also Qt has so many string functions that I can get stuff going in a fraction of the time I can with Python where I’m constantly having to check syntax and look up libraries.