Why does the new KiCad symbol format use s-expression?

one silly question: why s-expression and not for example JSON?

Because S-expression format is the best :slight_smile:

wouldn’t want to make it too easy for new developers to contribute /s

this is a 1980s CS class project, right?

From what i remember from a similar discussion on the mailing list:

Once a project decides on something this central it is in most cases better not to change direction without very good reason (otherwise nothing will ever be done.)

Technology moves on quite fast, s-expression seemed like a good future format when the decision was taken (for the footprint file format). It did not catch on with other projects as was hoped. The trade-off for going back on that decision (for footprints) is not worth it.

It would not make any sense to choose a different format for symbols than footprints. Meaning symbols use the same stuff already existing for footprints.

1 Like

Any structured format would work. JSON, S-EXPR and XML and so. many. others. are effectively the same for our purposes.

S-EXPR is nice because there are open source parsers available for many languages, it is compact and (arguably) readable. The same can be said for other formats but that just means there is no inherent benefit of one.

As @Rene_Poschl said, it is better not to change directions after starting with a format unless there is a very good reason.

For KiCad, part of the benefit is allowing manufacturers and fabricators the ability to extract data directly from KiCad files. If we pulled up and changed formats every time there is a new preferred format, that stability would be lost.

And, while we welcome new contributors to the project (Hi! Send me a message if you’d like help getting involved!), even lead developers tread very carefully when touching anything around the file formats because that affects millions of $/€/¥of business investments around the world.

New developers should not expect to modify basic functionality of the program until they are intimately familiar with all of the aspects of KiCad.

5 Likes

About human readability… S-EXPR as it is implemented by KiCad is “compact” as Seth said. It’s quicker to read and easier to navigate than JSON usually is, because there is more information in one line. For example, usually one object with 10 data fields takes ~11 visual text lines in JSON. But KiCad may put such an object (theoretically, if such an object exists) into less lines. As the result the file is quicker and easier to browse and read.

This isn’t forced by those formats, but in practice is so. JSON is less readable if one object is put in one text line.

If we compare with XML, this is what the wikipedia page about S-expression says:

S-Expressions are often compared to XML, a key difference being that S-Expressions are far simpler in syntax, therefore being much easier to parse.

This also means it’s easier to read by humans. Well, anything is easier to read than XML.

Every structured data language is unreadable in its own way. Mostly due to the unavoidable bracketing for nested data structures. XML for the <> pairs, JSON for the {}s, and S-expressions for the ()s. But the point is that these formats are meant for software-software communication. You can spend years becoming very adept at Kicad without ever knowing what the data format is.

If you are writing software just take advantage of the rich choice of well tested libraries for parsing these formats. If you are trying to decode, then call the libraries then render the format in a more meaningful form for you, the human.

No I won’t, I just read the files. :slight_smile:

((:flushed::wink:)(:flushed::wink:))!

(characters 20)

I think these answers are rationalisations after the fact, i.e. you can make justifications about why it is used now, but that doesn’t answer why S-exp were used in the first place.

I think a lot has to do with the fact that Specctra files used S-expressions, and KiCad has importers/exporters for that. Those were there to support the Freerouter autorouter. I think Dick Hollenbeck used s-expr for pcbnew, so that gives the reason. S-expr code was already there, and Dick liked the format.

The next question is why does Specctra use S-expressions.

1 Like

Well you can go down that rabbit hole to any depth. I’ve read a claim that the standard railway gauge in Europe descends from the width between wheels of Roman chariots.

Indeed, which is why these “Why not X?” discussions are quite pointless, although we often like to entertain them. Like the one about why does the coordinate origin start at upper left.

Often the answer is “it just happened like that but now it’s difficult to change”. Delving into who made the key decision and when is for historians. If you like that sort of thing, I recommend watching QI. A lot of the reasons given for things are folklore, and the real reasons lost.

S-expressions derive from Lisp, which goes back to 1958, but was state of the art then. I assume the same discussions occurred then, “Why free format text and not fixed column text like FORTRAN?”.

I don’t know if the Romans used S-expressions, I will look into it :slight_smile:

I think I’ve heard the same thing, except going further to explain that the width between wheels of Roman chariots was determined by the width of two horses of the time side-by-side. It was an example (probably apocryphal) of which horse’s ass determines a standard.

Pretty strange that none mentioned OrCAD .opj file format. I’m not good in KiCAD history neither an OrCAD fan but S-Expression adoption is probably due to the standard de facto of the era in which KiCAD is born. I don’t think is just a coincidence:

(ExpressProject "Capture Tutorial"
  (ProjectVersion "19981106")
  (ProjectType "PCB")
  (Folder "Design Resources"
    (Folder "Library"
      (File "c:\cadence\spb_17.2\tools\capture\library\capturetutorial 2.olb"
        (Type "Schematic Library")))
    (NoModify)
    (File ".\capture tutorial.dsn"
      (Type "Schematic Design")
      (DisplayName ".\capture tutorial.dsn"))
    (BuildFileAddedOrDeleted "x")
    (CompileFileAddedOrDeleted "x")
    (Board_sim_option "VHDL_flow")
    (ANNOTATE_Scope "0")
    (ANNOTATE_Mode "1")
    (ANNOTATE_Action "0")
    (Annotate_Page_Order "0")
    (ANNOTATE_Reset_References_to_1 "FALSE")
    (ANNOTATE_No_Page_Number_Change "FALSE")
    (ANNOTATE_Property_Combine "{Value}{Source Package}{POWER_GROUP}")
    (ANNOTATE_IncludeNonPrimitive "FALSE")
    (ANNOTATE_PreserveDesignator "FALSE")
    (ANNOTATE_PreserveUserEdits "FALSE")
    (ANNOTATE_Refdes_Control_Required "FALSE")
    (Update_Instace_for_External_Design "FALSE")
    (Annotate_type "Default")
    (width_pages "100")
    (width_start "80")
    (width_End "80")
    (DRC_Scope "0")
    (DRC_Action "1")
    (DRC_Create_Warnings "TRUE")
    (DRC_View_Output "FALSE")
    (DRC_Preserved_Waived "FALSE")
    (DRC_Run_Electrical_Rules "TRUE")
    (DRC_Run_Physical_Rules "TRUE")

Excerpt from an OrCAD tutorial .opj file.

1 Like

@qu1ck proposed using Protocol Buffers library, https://lists.launchpad.net/kicad-developers/msg38772.html. That’s at least one discussion where more reasoning can be found.

No such thing as ‘the best’.

Kicad uses s-expressions because it does.

It’s that simple.

1 Like

Changing it would lead to civil war over JSON vs XML and by the time that there was a victor, some new challenger would appear

3 Likes

Problem with s-expr is not that it’s round brackets vs curly brackets of json or angle brackets of xml (ugh).
It’s that the format is ad-hoc, evolves organically in not very organized manner, documentation is non-existent few months after initial doc is written and it’s benefits are not that good. It’s kinda machine readable except not really (see my point about most languages not understanding s-expr). It’s kinda human readable except not really (you still have to guess what is a keyword and what is a parameter, you have to read code to know for sure which positional parameter is which and what are possible enum values, etc).
Well I outlined most of it in my email and while my proposal wasn’t outright rejected, overall I didn’t get much traction. I don’t give up though and plan to write protobuf support when some of the infrastructure for multi format libraries is there. And hopefully devs will see how much nicer it is to work with protos vs the jungle of lexers/parsers/formatters that is current kicad io code.

So s-expr is good for inline comments? image

1 Like

In natural language, yes :slight_smile: