I extracted data on 19,000 Github projects that mention “kicad” and placed it in a BAT (big-ass table). ChatGPT and I built a single-page Javascript app to search, filter, and sort the projects. You can view them here. (The page takes 3-4 seconds to load.) Some of the projects you can find just by randomly scrolling are quite interesting.
This repo contains the code for the repository data extractor and the Javascript app.
This is great! Many thanks for that!
This is great. Even has a few of my projects in the list.
Nice to see the stars and forks numbers as well.
Found a few useful projects as well when sorted by stars.
That was one reason for doing this: to expose some of the brain power hidden away in Github repos.
I should probably generate an equivalent page for Freecad…
Out of curiosity, why is it that none of my public kicad files on github made it on to that list? Does your thing also search in subfolders or must the phrase Kicad be in the main page or something?
The Github search API doesn’t do any deep searching when you’re scanning across all their 300M+ repos. Basically, it will look for some search terms that appear in the project description, readme, and maybe the title and tags.
If you place “kicad” within your project’s description or readme, they’ll most likely get included when I run another complete scan.
I uploaded an equivalent FreeCAD Project Explorer. There’s a lot fewer FreeCAD repos versus KiCad (5K vs 19K), so the page loads a lot faster.
I built a single-page web app (RepoRecon) that integrates collections of Github repositories across a range of topics that might be of interest to people here:
Let me know if you have further topic suggestions, or you can raise an issue.
I had not responded earlier because I wanted to experiment a bit more with this seemingly simple project that gives quite impressive results. I’ve been going though random suggestions by your script and false positives are rare. But of course, I do not know how many projects it misses.
One small improvement is to remove the popup box with the number of projects found, and just display that somewhere in the header, for example after the search term.
Another improvement is to limit the amount of text in the “description” field of found projects. For example, when you search for Description:midi then it finds: CoCoDragonFlexiMIDI-V1R2 and it’s description spans multiple pages of text. (Maybe limit other fields too?)
I also think this project deserves a better place then just form a forum post. Maybe it (already) deserves a place at: External tools | KiCad EDA (Gosh, that one seems in need of maintenance, Altium imports are pretty much natively integrated in KiCad now, and what I last heard of KiCost is that the project crashed because of active obstruction by websites who do not want their data crawled by bots)
And regarding bask185’s question, when the KiCad project Explorer has a nice and cozy home, it also deserves a small description on what maintainers of repositories can do to ensure your script finds their projects.
Thanks for your suggestions! I made the change to show the number of repos displayed in the table after the table title instead of in the popup, and I limited the size of the description box to 10 lines.
Not sure if I do something wrong, but whatever I put in the filter field (in the required format Column:Value) I have an error: “No column matches Description.” or “No column matches Repo.” even if the search term appears on the list with no filter.
Haven’t checked all types of repos, but at least I get this for KiCAD and FreeCAD repos.
Tried at home and at work - same issue.
What do I do wrong?
(it was working fine before on the first version of the KiCAD project explorer, but now I’ve noticed that also doesn’t work).
Can you show a screenshot of what you typed into the search box exactly? (and the response).
It is a bit unusual that you have to enter the column name, then a semi colon and a keyword (without spaces) but it works for me.
@devbisme Your updates work for “RepoRecon”, but not (yet) for KiCad Project Explorer
Try using “description” (all lower case) instead of “Description”. (I tried to make the column headers look nicer by capitalizing the first letter, but I didn’t correct for this in the code.)
I’ve archived the dedicated KiCad Project Explorer and am only actively updating RepoRecon since that subsumes everything else.
Also, you don’t have to type the full “description”. All you need is enough of the beginning of the string to identify the header. So “d:” would work for the description column, and “o:” for the owner column.
That is of course nice and all, but in two weeks the link to the KiCad project explorer has accumulated 63 clicks, while RepoRecon somewhere in the middle of this thread has 8 of them. I suggest you also update your opening post in this thread so it links to the right place.
Good idea! (I didn’t even know you could do that…)
That did the trick! Many thanks!
I have two simple idea’s for improvements:
https://devbisme.github.io/RepoRecon/
I had to type a few lines of text as a manual how to do a search for raspberry related stuff with RepoRecon. If those two things are changed, I can omit that text, just post the URI with the custom search, and for the reader it reduces a response to a single mouse click on the info bar with your icon.
I’ve searched for an hour with Google and ChatGPT and neither has shown me how to do this. [info]...[/info] was the closest hit, but it doesn’t work within a post. And where would I place this? Inside the first post of this thread?
OK, I can do that. I also need to extend the search string capabilities.
Good suggestions!