KiCommand discussion and development - Easy pcbnew command line; 4.07 & 5.1.5/.6

[HiGreg]

Am I being too wordy?

I wanted to present the ability to call python object functions with arguments:

# Move all module's Value text to Dwgs.User layer
r('MODULES Value CALL Dwgs.User LAYERNUMS LIST SetLayer CALLARGS')

# Move only selected module's Value text to Dwgs.User layer
r('MODULES SELECTED Value CALL Dwgs.User LAYERNUMS LIST SetLayer CALLARGS')

These could be saved as a function called ASSIGNVALUELAYER like this:

# Usage: 'Dwgs.User ASSIGNVALUELAYER'
# changes selected module's value object to Dwgs.User layer.
r(': ASSIGNVALUELAYER MODULES SELECTED Value CALL SWAP LAYERNUMS LIST SetLayer CALLARGS ;')

And then used like this:

r('Eco1.User ASSIGNVALUELAYER')

Here’s the help text for CALLARGS:

CALLARGS (Category: Call)
    [OBJECTLIST ARGLISTOFLISTS FUNCTION] Execute python FUNCTION
    on each member of OBJECTLIST with arguments in
    ARGLISTOFLISTS. ARGLISTOFLISTS can be shorter than
    OBJECTLIST, in which case ARGLISTOFLISTS elements will be
    repeated (or truncated) to match the length of OBJECTLIST.
    Returns the list of results in the same order as the
    original OBJECTLIST. The commands LIST and ZIP2 will be
    helpful here.

[HiGreg]

Just added corner extraction from objects (GetBoundingBox() for those who know python):

r('TOPTEXT CORNERS DRAWSEGMENTS')

The help text:

>>> r('CORN HELP')
CORNERS (Category: Geometry)
    [OBJECT] OBJECT is either a single object or a list of
    objects. Converts each OBJECT, either EDA_RECT or OBJECT's
    BoundingBox into vertices appropriate for DRAWSEGMENTS.

More geometry extraction coming…

[HiGreg]

More progress: drawing rotated boxes derived from outlines (aka BoundingBox).

This draws boxes derived from the top-level text’s bounding boxes. Note that I could have used a single rotation point for all objects, or (like the example) used a different rotation point for each object.

r('CLEAR TOPTEXT COPY CORNERS SWAP GetCenter CALL 
   55 ROTATEPOINTS DRAWSEGMENTS')

Here’s a description of what’s happening:

• CLEAR – Clear the stack
• TOPTEXT – Get all the text at the board level (not text within modules)
• COPY – duplicate the top of the stack
• CORNERS – get the corners of the text objects
• SWAP – on the stack, swap the top and second to top.
  • Now the stack is BOTTOM -> corner points -> text objects -> TOP
• GetCenter CALL – Get the center of the text objects.
  • Now the stack is BOTTOM -> corner points -> center points -> TOP
• 55 – the angle of rotation (CCW/LEFTWARD from due right)
• ROTATEPOINTS – rotate corner points around center points, 55 degrees.
• DRAWSEGMENTS – draw individual shapes from the rotated points.

And for reference, here is the help text of the commands.

r("'CLEAR HELP 'TOPTEXT HELP 'COPY HELP 'CORNERS HELP 'SWAP HELP 'CALL HELP 'ROTATEPOINTS HELP 'DRAWSEGMENTS HELP")
CLEAR (Category: Stack)
    Clear the stack.
CORNERS (Category: Geometry)
    [OBJECT] OBJECT is either a single object or a list of
    objects. Converts each OBJECT, either EDA_RECT or OBJECT's
    BoundingBox into vertices appropriate for DRAWSEGMENTS.
SWAP (Category: Stack)
    Switches the two top objects on the stack.
CALL (Category: Call)
    [LIST FUNCTION] Execute python FUNCTION on each member of
    LIST. Return the list of results in the same order as the
    original LIST.
ROTATEPOINTS (Category: Geometry)
    [POINTS CENTER DEGREES] Rotate POINTS around CENTER. POINTS
    can be in multiple formats such as EDA_RECT or a list of one
    or more points.
DRAWSEGMENTS (Category: Draw)
    [POINTSLIST] Points list is interpreted as pairs of X/Y
    values. Line segments aredrawn between all successive pairs
    of points, creating a connected sequence of lines.This
    command uses previously set DRAWPARAMS and the points are in
    native units (nm) so using MM or MILS commands is suggested.

Bonus edit! Here’s how to outline each top level textbox, whether it is rotated or not:

r('CLEAR TOPTEXT COPY GetTextBox CALL CORNERS SWAP COPY GetCenter CALL SWAP GetTextAngleDegrees CALL ROTATEPOINTS DRAWSEGMENTS')

[ikletti]

Thanks for the clarification.

[quote=“HiGreg, post:18, topic:7694”]
So this started out as a sequence of “I wish I had something that could make that easy” to “hmmm, can I do this in a more general way” to “I wonder if this might be useful to anyone else” to “I think I can add even more useful stuff as commands”.

So not really an original intent, but an exploration of the space of what’s possible to make simple scripts a little more accessible.[/quote]
This attitude is what really drives progress. Thanks! (I wish more of my students had this attitude)

@HiGreg

An example of the simplicity of using command_stack. Here is how to select all Module References in command_stack (at some point there will be a command line dialog):

MODULES GetReference CALL SETSELECT

Here is the same thing in python:

for module in pcbnew.GetBoard().GetModules():
module.GetReference().SetSelected()

There’s not a big difference here to an experienced programmer. But to someone who has never programmed, the first is a LOT easier.

Since I belong to the non-programmer user group, I can definitely see myself using your command squence over the python script

[HiGreg]

Here is the proposed README file for KCStack. I’m taking suggestions on the
program name and any comments or questions about anything in this README file,
especially from non-programmers about anything that doesn’t make sense to you.

Is there anything I can make clearer?

KCStack - Kicad Command Stack

Some other options for names: KiCommand, KSCommand,
KiCommandStack, KiStack, KiCommandString, KCString

I like that “Command” implies a command interface since most non-programmers
won’t know what a stack is, or at least won’t identify it with commands.
Clearly I like names that have Command in them.

KCStack allows simple command strings to be executed within pcbnew.
Command strings consist of a variety commands that retrieve, filter, and
process Kicad objects. Commands are very easily added with a simple syntax.

Here are a few short examples:

• PADS SETSELECT
  • select all pads
• : SELECTALLPADS PADS SETSELECT ;
  • define a new command called SELECTALLPADS that select all pads
• MODULES U1 MATCHREFERENCE GETPADS SETSELECT
  • select the pads of the module with reference ‘U1’
• : SELECTMODPADS MODULES SWAP MATCHREFERENCE GETPADS SETSELECT ;
  • define a command that select the pads of the module indicated by the argument.
  • Use the command like this: U1 SELECTMODPADS
• VALUETEXTOBJ DeleteStructure CALL
  • Delete all value objects on the modules
• VALUETEXTOBJ SELECTED DeleteStructure CALL
  • Delete all selected value objects on the modules

Getting Started

Installation

KCStack is (will be) an ActionPlugin and is installed similarly to other Action Plugins:

1. Place the kcstack.py and kcstack_gui.py files in
   C:\Program Files\KiCad\share\kicad\scripting\plugins
   Or the equivalent in MacOS or Linux
   (there may be a user-level directory for such files, but I am not aware of it at the moment.)
2. Within KiCad pcbnew, select the Tools > External Plugins > Refresh Plugins
3. The next time you start pcbnew, the KCStack menu item will already be in
   the External Plugins menu so there is no need to Refresh Plugins.

KCStack dialog box is shown when the Tools > External Plugins > KCStack menu item is selected.

Self Documented Help

Open the Script Console from the Tools menu. Then type

import kcstack
r=kcstack.runcommand
r("HELP")

This will display a short help message, including how to get a list of
commands and how to get more detailed information about each command.

Another useful help commands are:

• r(“Help HELPCAT”) - short list of help commands
• r("'HELP HELPCOM") - details of commands with HELP in the name.
  Note the preceding single quote character.
• r(“ALL HELPCAT”) - all commands listed by category
• r(“HELPALL”) - all commands by category with their details in
  alphabetical order

Overview

With KCStack, arguments to commands are enter before the command. Any results
from the command are then used as an argument to the next command. This way,
you can chain together commands in a way that often makes sense. This
programming structure is
called stack-based programming.

KCStack has several advantages over Python Scripting:

• Simplicity in programming and argument type handling make KCStack more
  accessible than the equivalent KiCad Python scripting.
• Command strings mean less worrying about variables.
• Command strings are often short and easily sharable.
• Many commands accept a variety of input types, and still work as you would expect.
• Programming structure means you don’t have to worry as much about variables.
• KCStack naturally handles lists of objects, so looping over objects is not
  needed: it just happens.
• Being able use pcbnew Python object attributes and functions gives
  KCStack a lot of access to the Kicad object model.
• Defining new commands is simple.
• With KCStack handling of argument types, there’s less worrying about
  exact types.

And several disadvantages:

• Built in commands have flexible argument types, while Python commands
  (accessed with CALLARGS) may require careful argument manipulation.
• Most commands are simple and straightforward, while complex commands are
  possible. The stack-based structure makes some complex strings difficult to
  decipher or create even for experienced programmers.
• While creating entirely new elements from scratch is usually possible,
  command strings are sometimes wordy.
• There are currently no looping or conditional commands.
• Full flexibility is only available with Python scripting. Command strings
  are a short simple interface for some object manipulation or interrogation.
• (Currently) Strings with spaces cannot be created from scratch.

Introduction to Command Strings and Programming Structure

In KCStack, a Command String contains a sequence of arguments and commands
that are executed sequentially. Arguments occur before the command that uses
them. The arguments are consumed by a command and the results of the command
are stored on top of any previously unused arguments or results, making those
arguments and results available to a future commands.

This is implemented and often imagined as a stack structure.
In this structure, the stack holds values (aka operands) that are used in
subsequent commands.

Several important characteristics of the stack structure of programming:

• operands are placed on top of the stack when encountered in the
  command string
• operands are removed from the top of the stack when commands are encountered in the command string
• operands are placed on top of the stack when returned from executed commands
• results from previous commands, when unused, continue to exist on the stack
  and can be used for future commands. In this way, results from past commands
  build up to become arguments for future commands.

Examples

• [MODULES]
  • return the list of modules
• [MODULES SELECTED]
  • return the list of selected modules
• [MODULES SELECTED CLEARSELECT]
  • unselect all selected modules
• [MODULES SETSELECT]
  • select all modules (this seems to have no visual effect)
• [PADS SETSELECT]
  • select all pads
• [PADS CLEARSELECT]
  • unselect all pads
• [MODULES GETPADS SETSELECT]
  • select all pads of all modules
• [MODULES GETPADS CLEARSELECT]
  • unselect all pads of all modules
• [MODULES U1 MATCHREFERENCE GETPADS SETSELECT]
  • select the pads of the module with reference ‘U1’

General Conventions

KCStack follows a general set of conventions:

• COMMANDs are in all capital letters.
• To enter an argument that also happens to be a command, use the single quote
  mark (’) such as in the following string (the brackets are not part of the
  string): ['CALLLIST HELP]
• Access to Python functions and attributes are exactly as documented in
  the
  Python pcbnew documentation,
  which is often in mixed case. Example [MODULES GetCenter CALL]
• Define a new command with the colon, and end with the semicolon.
  • : NEWCOMMAND ARG ARG COMMAND ARG COMMAND ;
• Core commands either place objects on the stack or operate on objects on the
  stack. The commands that place a list of objects on the stack are in the
  category Elements and are listed with the command Elements HELPCAT:
  • MODULES
  • PADS
  • TRACKS - includes vias
  • DRAWINGS
• From these core commands, other commands are defined to
  retrieve certain objects.
  • TEXTOBJ
  • VALUETEXTOBJ
  • REFERENCEVALUEOBJ
  • TOPTEXT
• And in case there is anything missing, you can access the top-level
  board and pcbnew objects.
  • PCBNEW - top level pcbnew Python object
  • BOARD - top level Board Python object from pcbnew.GetBoard()
• And finally, you can filter each of the above objects to choose exactly the
  objects you want, or get at them in slightly different ways.
  • PADS SELECTED
  • MODULES U1 MATCHREFERENCE GETPADS
  • TRACKS VIA FILTERTYPE

[HiGreg]

Here’s an example of the geometry manipulation I’m working on:

Selecting these line segments:

Executing this:

r('5 MM DRAWINGS DRAWSEGMENT FILTERTYPE SELECTED ROUND')

Results in this:

The corners are rounded using arcs of specified radius. In this case, the radius is specified as 5mm.

I’m still working out the details and a lot of bugs.

Comments?

[ikletti]

@HiGreg

Comments?

As a frequent user of a trusty HP48 calculator, I like your RPN approach to commands

I may be a little late to that party, but looking at your examples I find it hard to discern the arguments from the commands.

Would it be possible to have a better distinction between arguments and commands like e.g. brackets for arguments/values? That might make reading a chain of commands easier for humans and maybe you could even omit the single quote mark for commands that are used as arguments.

[HiGreg]

I’m thinking about it. I see a few issues:

Arguments themselves are most often the output of prior commands. In that case if the previous commands are included within the brackets, then it adds the complexity of nested brackets: the whole reason for RPN-style command structure in the first place.

If only arguments that aren’t themselves commands are surrounded by brackets, then there’s the confusion that only some of the arguments for a particular command are bracketed.

The single quote should really be considered “make this a literal”.

I could perhaps change the literal arguments (e.g. the “ALL” used for the HELP command) to be lower case. Then Command Stack commands are all upper, Python commands are mostly mixed case, and Command Stack literal args would be lower case.

Literal strings (i.e. Module Reference names) would still be whatever case they need to be.

How does that sound?

[Shack]

@HiGreg
Hi just wanted to say that this stuff looks really cool and Indeed useful Ill probably give it a go when I have some more time please keep up the good work (y)

[HiGreg]

Even better, I think there’s a way to almost eliminate the need for using single quote (it still may be necessary in some cases).

Assuming that most text on the board is uppercase and most Python functions and variables are all uppercase or mixed case. then it really works well to do the following:

• Command Stack commands are all lower case.
• Command Stack arguments are Mixed Case.
• Python commands within Command Stack are whatever is needed, but mostly will be Mixed Case or UPPER CASE.

This way Mixed Case items are all arguments (Python functions and variable look like arguments to Command Stack).
Mixed case Command Stack arguments means they also do not get interpreted as commands
And all lower case commands conflict with neither the namespaces of most Python commands/variables nor literal Command Stack arguments.

Hopefully I’ll get used to lower case commands. I originally thought that upper case commands would be better.

[HiGreg]

Think I’ve gotten real close on connect and round commands

This command turns the top shape into the bottom shape:

r(‘3 mm drawings selected copy connect round’)

The connect command will connect line segment pairs that have nearby vertices (move vertices to make them overlap), while round will convert all angles in the shape or line to be an arc of the specified radius.

And this command represents the change in case indicated the previous post.

[Rene_Poschl]

This looks very promising. Keep on the good work. Sadly i currently can’t use nightly to test it. (The nightly on fedora has problems. And i need stable anyways to test library contributions.)

[HiGreg]

New command: regular makes a regular polygon of the selected segments.

First you draw the segments, then make the segments connected with the connect command, then finally you can make the polygon regular. The connect command will make the vertices line up on top of the adjoining segment’s vertices.

Each of these almost-shapes was selected in turn. The command to turn the top shape into the bottom shape was.

r('clear drawings selected copy connect regular')

No segments were created or destroyed, just moved around to make a regular polygon (a polygon with equal sides and equal angles).

[HiGreg]

KiCommand is now released on github

Works with nightly

WARNING: The nightly version is not guaranteed to produce files that can be opened in the stable version. There is not even a guarantee that one nightly build can open the files produced by another nightly build. (In most cases there is no problem but you might encounter one. Just know the dangers that come with using a development build.)

Since adding the gui, I’m not sure if I broke the command line: I just haven’t tested it.

There is still a lot of work to do. Be careful docking the window (seems like pcbnew is stealing keystrokes), and ALWAYS save before executing a command. Crashing and creating bizarre objects are to be expected.

Comments and suggestions are welcome. My time is running out until maybe october, so response may be slow.

[TotalKrill]

Interesting project! I like the functionality added here! I do however find that this looks a lot like trying to run SQL querys on a PCB. I do not know if that is good or bad. But maybe you can find ideas for the syntax from there?

[HiGreg]

I like the postfix notation. It makes parsing dead simple, and chaining commands together is easier than dealing with variables. I find SQL hard to compose, and it also means abstracting python objects. I’ll give it some more thought…

[HiGreg]

Update on github to add better error message and command history in the combobox.

[HiGreg]

Updates posted:

Added builtins, fcall, fcallargs to enable getting the ‘range’ function
“clear builtins range index list 5 int list print fcallargs”
For some reason, builtins appears on the stack as a dictionary, which
won’t work with ‘call’ or ‘callargs’, so ‘fcall’, ‘fcallargs’, and ‘sindex’
were created.

Added sindex to allow accessing Python dictionaries with string indexes.

Fixed pick command, was only returning top of stack.

Reworked ‘int’ command so that it returns a single value, not list, if there
is a single string without a comma. This allows the creation of a single
value on the stack (motivated in this case to enable ‘index’ to work well
with lists or dictionaries).
“: range int list builtins range index list swap print fcallargs ;”

Reworked ‘float’, ‘index’, ‘mm’, ‘mil’, ‘mils’ similarly to int.

Added quoted string using “double quotes”. All spaces inside the quote
marks are retained. Words are split on the double quote mark such that the
following are equal:
1 2 3 " 4 5 6 " 7 8 9
1 2 3" 4 5 6 "7 8 9
If in a file, pairs of quote marks must be on the same line.

Added load and save commands for the user dictionary. Lightly tested.

[HiGreg]

Any feedack so far? Has anyone given this a shot? Are there any questions I can answer?

[HiGreg]

Still making progress. Added a few Geometry and Drawing commands, and the Action commands can handle ARC segments (like connect).

There’re still some bugs and I haven’t released the newest version quite yet. I just wanted to post on some of the progress I’ve made.

Draw        - makeangle drawsegments pad2draw drawparams cut regular drawtext drawarc round 
Geometry    - rotatepoints ends angle rotate corners length 
Action      - select deselect rejoin connect 

Note that the selected segment is one piece:

and after the ‘cut’ command, the cut line is now in two pieces:

Here’s the help text for some of the new commands.

makeangle (Category: Draw) 
    [SEGMENTLIST ANGLE] Make the selected segments form the
    specified angle. arc radius is maintained, though angle and
    position are modified, while line segments are moved and
    stretched to be +/- n*angle specified. 
cut (Category: Draw) 
    Cut all segments with the selected segment at the
    intersection. 
regular (Category: Draw) 
    [SEGMENTLIST] Move/stretch the selected segments into a
    regular polygon (equal length sides, equal angles). 
drawarc (Category: Draw) 
     
round (Category: Draw) 
    [RADIUS SEGMENTLIST] Round the corners of connected line
    segments within SEGMENTLIST by adding ARCs of specified
    RADIUS. 
angle (Category: Geometry) 
    [SEGMENTLIST] Return the angle of each segment in
    SEGMENTLIST. 
rotate (Category: Geometry) 
    [SEGMENTLIST DEGREES] Rotate segments by DEGREES around
    additive center. 
length (Category: Geometry) 
    [SEGMENTLIST] Get the length of each segment (works with
    segment and arc types 

Note that I made the text in the screenshots parallel to the segments by using the angle command to find the segment angle.