KiCad growth based on created Github projects

I was trying to extract a complete list of KiCad projects from Github and just decided to see how many had been created each year:

2012: 62
2013: 134
2014: 264
2015: 557
2016: 813
2017: 1058
2018: 1204
2019: 1356
2020: 1681
2021: 1645
2022: 1626
2023: 1846 (extrapolated from 1139 projects started over 61.7% of year)

(For context, GH was founded in 2008 and received a $100M investment from Andreesen/Horowitz in 2012.)

So doubling each year from 2012 - 2015, then kinda linear growth through 2020, and flat since then.

For those with an interest, here is the script I used:

for y in {2012..2023}
do
    created_on="${y}"
    num_kicad_projects=$(
        curl -G https://api.github.com/search/repositories \
        --data-urlencode "q=kicad created:$created_on" \
        -H "Accept: application/vnd.github+json" \
        | jq ".total_count"
    )
    echo "$created_on: $num_kicad_projects"
done
2 Likes

Another interesting info would be Kicad related projects. This would include tools, hacks, scripts, libs, workbenches.

Since I did the search just on ā€œkicadā€, I think I picked up most of those.

Itā€™s fully expected to hit a plateau, KiCad is only filling a niche, GitHub otoh represents a majority of random things, mostly code related rather than hardware designs.

Also some devs moved to GitLab in a last couple of years, so some recent KiCad project are probably there, not on GitHub.

2 Likes

So there are around 10k KiCad projects on github?
Can you search for on github for repositories that have a kicad project file extension? (The old ā€œ.proā€ will have a lot of false positives, but ā€œ.kicad_proā€ should be quite good.

There are quite a lot of sites collecting (partial) schematic / pcb projects. A bunch of PCB manufacturers share (selected?) projects:

https://oshpark.com/shared_projects

https://www.pcbway.com/project/

EDA suites based on the ā€œcloudā€ (Itā€™s just someone elseā€™s computer) do this, such as Easy EDA and even that ā€œaltiumā€ company. I had a look at those sites, there are some nice projects over there, but 90% or more seem to be forks of other projects. And of course, KiCad itself also has made-with-kicad which lists around 50 projects.

And of course there are also websites about electronics that collect and share projects. Most electronics magazines (if still in existence) have an area for this but there are quite a lot of such sites, varying from mostly shoving advertisements down your throat wile trying to attract people with a bare minimum of projects, to quite elaborate collections of projects. https://www.electronics-lab.com/ seems noteworthy for example, and the projects section on Hackaday

Sometimes I wonder itā€™s worth creating a FAQ article to collect such links. Itā€™s a bit similar to the ā€œwebringā€ idea, but apparently such attempts have been deprecated by generic search engines which are continually updated, while hand cured lists are hard to maintain.

The Github search API only scans file extensions when you do a file search within a specific repository. You canā€™t ask it to scan all the repos and select the ones that contain a.kicad_pro file. So you would have to loop across all the repos and do a file scan for each one. Nobody got time for that!

I did re-run the previous search but included a scan of the README files for ā€œkicadā€:

for y in {2012..2023}
do
    created_on="${y}"
    num_kicad_projects=$(
        curl -G https://api.github.com/search/repositories \
        --data-urlencode "q=kicad in:readme created:$created_on" \
        -H "Accept: application/vnd.github+json" \
        | jq ".total_count"
    )
    echo "$created_on: $num_kicad_projects"
    sleep 6s
done

This picks up more KiCad projects than the previous search:

2012	118
2013	280
2014	438
2015	731
2016	1001
2017	1397
2018	1640
2019	1970
2020	2837
2021	3188
2022	3394
2023	3574 (extrapolated from 2214 over 0.619 of a year)

Now the stagnation period is replaced with near-linear growth, but thereā€™s some step-change between 2019 and 2020. Letā€™s just blame it on COVID-19ā€¦

Glancing at random description strings from 2023, Iā€™d estimate >95% of these recent repos are actual KiCad projects versus code or footprint/symbol files. So itā€™s mainly people using KiCad rather than working on it. Even in this small sample, Iā€™ve come across some interesting dark projects that have never been mentioned in this forum or anywhere else. Once I extract the entire list, interested people can click on the repo links and find worthwhile projects to expose to a wider audience.

1 Like

118+280+438+731+1001+1397+1640+1970+2837+3188+3394+3574 = 20568

An annotated and indexed database of 20000 Open Sourced KiCad projects would be a nice thing to have :slight_smile: Especially if it has parametric search to for example exclude the low part count beginner experiments and blinking LED projects, searches for a particular microcontroller (family).

I guess that you can already do something similar like that with githubs search interface. It may be worth creating a FAQ article on how to use github (gitlab, other source code repository sites) effectively for finding KiCad projects or other KiCad related content.

More info on github:

Microsoft acquired Github for USD7.5G (or is that ā€œBillionā€ money numbers are weird) in 2018 for which was then apparently 30x the yearly revenue while Wikipedia now states revenue in 2022 was One Billion USD. That is an impressive growth for just 4 years and completely mind boggling numbers.

Interesting logic. When Paul turned 50 he was 1275 years old: 1+2+3+ā€¦+50

First sentence of this thread:

1 Like

Ah, indeed. I thought it was existing projects year by year, already cumulative.

Hello everybody.

There is the old project Leopart (four years), the component search engine for reference designs used in GitHub. Indexed 8,000 KiCad-projects on GitHub containing 37,000 electronic components as a proof-of-concept.

Explanations : FOSDEM 2020 - Leveraging Open Source Designs
Code on GitHub : GitHub - leopart-hq/leopart: This is a search engine for publicly available KiCad files on GitHub. The KiCad files are parsed and the contained components indexed to make the projects searchable by the components they contain. This project was started as a university lab course and is now an open source project.

Best regards.

Jean-Pierre

Hi, Jean. Good project! I watched your video and saw you had the same problems with Github search I did: search is limited to things like the description and readme and wonā€™t return more than 1000 results.

Iā€™ve got a shell script that replicates the initial web-crawler of your project and stores the repo links in a JSON file. KiCad repos have increased to >19,000 since your talk.

If I decide to push this any further, Iā€™ll definitely look into extending your project. Is the Aisler part lookup service still available?

That is pretty close to 20000 I added up, especially because you added 1350 to extrapolate to the end of this year.

Would it be of use to archive such search data in case microsoft cripples search capabilities?
I think it could also be quite useful to have this as a plugin in KiCad. Maybe preferably with the database somewhere else, so there are no 20.000 people crawling over gitlab every day. What would be a good location for such a database? How big is it?

Hi Dave.

Iā€™m not the original author of this project, itā€™s Lasse Mƶnch :

But you ask good questions.

Best regards.

Jean-Pierre

Yes, itā€™s stored as a 200-Kline, 5MB JSON file. It takes about an hour to generate from scratch (Github rate limits), but would be much faster to update every few days with any new projects. Iā€™m going to store the shell script and JSON on Github along with a simple webapp that displays it as a filterable/sortable table. Anyone can clone it from there and actually run it locally if they want. Iā€™m actually testing it locally now just using python -m http.server in the directory.

1 Like

It may be a very nice addition to the KiCad website. That could limit gitlab crawling to once a week or once a month or so.

Hi Dave.

Wonderful, weā€™re going to have something usable to research Kicad projects.

Best regards.

Jean-Pierre

Hm, for some reason I tought you searched for .kicad_pro

Can you compare those 2 searches for us?

Hereā€™s the data for the search term ā€œ.kicad_proā€:

2012: 0
2013: 2
2014: 5
2015: 8
2016: 18
2017: 18
2018: 23
2019: 27
2020: 42
2021: 59
2022: 60
2023: 48

Note that ā€œ.kicad_proā€ doesnā€™t do what you think. It actually makes Github search look for repos containing both strings ā€œkicadā€ and ā€œproā€.