Vibe coding kipart

There’s been some activity on this forum about using new AI tools to create KiCad plugins, so I thought I’d share my experience.

I wrote kipart back in 2015. It’s not actually a plugin. It’s a Python script that will turn tabular data on part pins into a schematic symbol. It’s very useful for large parts with 100 or more pins. It’s been used for generating libraries of FPGAs with thousands of pins.

But kipart was written for the old-style symbol library format before the switch to S-expressions. I looked at updating it but it was too much work and I had too little interest.

Last year I tried giving the code base to a few of the AI tools and asking “Convert this code to output symbol libraries using the KiCad S-expression format described at this [URL].” The tools would grind away for a while and then give up, usually reporting an overflow on the context window.

Last month, I decided to retry it using a more incremental approach. I asked Grok to create a CSV file with the pin data for a hypothetical part. Then I ran that CSV through the old kipart and got a library symbol. I used KiCad 8 to convert that symbol into the S-expression format. Then I passed the S-expression back to Grok and said “write a program that will create S-expression libraries from CSV files using this as an example.” (I’m heavily paraphrasing here.) And it came up with something that kinda-sorta worked.

From there, it was a matter of iteratively testing the code → reporting errors → getting fixes → requesting new features (incrementally) → repeat. For example, I asked for the ability to work with spreadsheet files and Grok added the option and supporting code. I tested it, found that it worked, and then moved on to the next feature. A large part of the development time was just waiting several minutes for Grok to regenerate up to 700 lines of code.

Throughout the process, Grok was providing unit tests for the code so I could always check that features were not failing as modifications were made. At the end I also had Grok write the comments and docstrings to document the code.

You can see the entire set of interactions between me and Grok here. It’s LONG because Grok prints out all the code for each iteration and then it likes to talk about what it did. It’s more informative, initially, to look at my prompts which are a tiny fraction of the thread.

I didn’t get to a final working version of kipart. Grok had some difficulties with aligning pins along all four sides of a symbol, but the code for that is relatively small and well-contained so I could manually handle it. And I may do that and release it, or I may not. It doesn’t really matter to me. It took so little effort to create the code that discarding it isn’t a big deal. See this for more about this issue.

This has some big implications for KiCad. There may soon be many plugins. It may become easier to make a plugin than it is to search for an existing one that does what you want.

4 Likes

I decided to finish the new version of kipart. Three weeks ago, the AI-generated version was creating part symbols that looked like this:

There were some “problems”, but it was pretty good for an AI that can’t even see.

I reworked the section of code that performs the detailed pin placement, and now kipart produces symbols that look like this:

For everything else I did to get to a working 2.0 version, I used VSCode and Copilot running Claude 3.7 Sonnet. This included:

  • Implementing features: Often I would describe the function of a command-line option, ask for it to be implemented, and that would be it. Less often it took some back-and-forth with some manual tweaking. The AI was great at modifying the code to thread the parameters to where they need to be through several levels of function calls. Once there, it could build the program logic. If it needed some help, it was usually there.
  • Building tests: 100% of my unit tests were built using AI. There are a lot more of them with a lot more coverage because of that.
  • Writing documentation: The AI wrote 100% of the docstrings for the functions and modules in kipart, and it kept them up-to-date as the program logic evolved. I wrote the README myself, but I should have had the AI check it.

As I worked though this project, it showed me a couple of drawbacks that people associate with AI either aren’t true or aren’t big problems:

  • AIs hallucinate: AIs do make mistakes, just like people. But they can also have visions where they invent things you didn’t know you needed. In one case, I mistyped <Enter> and sent the AI only the first line of my prompt. It came back with a function that had features I hadn’t asked for. I kept about 80% of them. But I also had a single instance where the AI injected one line of faulty code while it was rewriting docstrings. That’s why it’s important to have the AI write a lot of unit tests and to keep the code under version control so you can detect a problem and step back to a working version if need be.
  • It would take me longer to write the prompt than to just write the code.: This is only true if you’re micro-managing the AI. Instead of giving it detailed instructions of how to do something, it’s better to give it a short, high-level description of what you want done. Then you can refine it (if need be) using additional prompts. Over time as I’ve worked with AI, my prompts have gotten shorter and shorter.
1 Like

Thanks, nice write up.

My experience with copilot and similar tools is overall positive too, it’s a great tool. And no, it won’t replace actual programmers any time soon.

I found the most effective use of copilot is like a having a very experienced programmer mentor that has erudite level knowledge of almost every area of programming there is. You can ping him with any question any time and he will give you very useful snippets, even tailor made for your context of already existing program, but he won’t actually write a program of any real size for you, you still have to do the legwork.

Also it sometimes acts like an old dog that stubbornly refuses to learn new tricks. For example if you try to use it to help with a program using newer version of a framework than the one it is trained on it will stubbornly give you old snippets using old apis that don’t work, no matter how many times you tell him to use newer version. It will acknowledge the gaps, fix it’s output and then revert back on the very next question.

And of course it has trouble actually understanding complex algorithms which pops up when you need a customized version of one, it just regurgitates standard implementations again and again. But this situation is rare enough that it can be given a pass.

All of the LLMs have these issues currently but they are still incredible accelerators if you use them to their strengths. Using it to help with a github workflow and not needing reference docs, or writing a bash script and not having to dig through mans of awk, xargs, shell built-ins and all that crap for the 10th time was like a breath of fresh air, like going from writing code in notepad to an actual IDE with dynamic highlighting, intellisense and refactoring tools.

2 Likes

Have you seen this other (recent) thread?

It’s for a plugin that (apparently) directly interacts with an AI assistant from within KiCad itself.

The largest danger with using AI tools is complacency.

Was reading about another law firm using AI and was caught by the judge. He slapped their wrist and told them to redo/resubmit. Hallucinations went from 2 to, well, let’s just say ‘more than 2’.

I saw it but I couldn’t understand what was happening. I can never understand videos of people using a GUI: it’s too small to read the text, the mouse moves too fast, and the time between hovering over an entry in a pop-up menu and then having it vanish once it’s clicked on is about 0.1 seconds. It’s probably a great idea, I just can’t process it.

I think you’re echoing what the author of this said: AI is more of a mech-suit than a robot. It enhances your abilities instead of replacing them.

AI won’t replace programmers, it will create more of them. With an AI and some basic knowledge of programming (expressions, conditionals, iteration, functions), you can write in many common languages. It will fill in the details and explain what it did. There was a thread a few weeks ago of someone who built a via-fencing plugin without knowing Python because their experience in C/C++ provided the fundamental knowledge. For KiCad, this means its APIs for interfacing to plugins are more important than ever and that its functionality could start changing more quickly than has previously been possible.

1 Like

The issue is that often people are taking what an AI assistant writes with no critical analysis. We’ve got a world of edge-cases, memory issues, undefined behaviour, and security traps growing in the global codebases as a result of this.

I do use my IDE’s per-line autocomplete, which is driven with an AI model, for KiCad development and it’s an absolute accelerator. But I have to edit its output a lot. All my tests of large code generation models are basically rather poor (see also a recent Reddit post about MS’s use of their code model to auto-generate fixes). I’m sure they’ll improve, but I worry about the technical debt being added to products unwittingly currently.