Andrew asks
Any chance of a higher release cadence? Say, a major version every year, with relatively fewer features.
2 Likes
How about 6.1 for the “almost ready” or “half ready” features which didn’t make it to 6.0? Within, say, some months from 6.0?
1 Like
I personally think it will be better to do a 7.0 in ~1 year. Doing a release takes a lot of time, you have to switch from building new stuff to testing/fixing. Changing back and forth more quickly between those two modes means much less can get done.
3 Likes
That was my question, glad it was asked and answered.
It really doesn’t have to be this way.
KiCad needs a lot better test coverage and a lot more release automation. These are solved technical problems nowadays.
And then you will not have to switch into testing/fixing mode at all.
If there is some room in the budget from the donations, renting a few instances for build servers and gitlab runners will go a long way.
Well, the thing is that these are not solved for KiCad. We would love to have your help solving them if you are up for it. Building test coverage takes a lot of time, especially since we would need a lot of simulated human interaction coverage. Not only is the open-sourcee tooling situation for that rather poor, we just don’t have time or expertise to set that up right now. We definitely need more developer-hours spent on this before something like renting servers would help.
2 Likes
Releases themselves are automated already for the most part already. It takes no effort to bump a tag and a server packages it. It’s hardly where the problems lie.
The difficulty is taking multiple independent projects such as the kicad application, documentation and libraries and having all the independent actors be ready and prepared. That’s not anything automation can solve and entirely a problem of manpower to “polish” everything off. Once a human declares its done, they make a tag. Then the existing automation takes the tag for release.
Like the “automation” could just take the latest of everything blindly but that’s not going to end well
Also test coverage is a solved problem sure, but somebody needs to write the tests. Feel free to get started 
2 Likes
There’s actually quite a few projects that function this way (and basically the entire JS ecosystem…) and it doesn’t work that badly. It’s not like any of the released version so far have been fully bug free or feature-complete, so just get everyone used to living on master and reserve the super experimental stuff for feature branches.
O please no.
The irresponsible behavior of the dumpster fire of the wider JS ecosystem should never be held as a golden model for professional engineering.
6 Likes
It’s a lot better than this ancient waterfall system with YEARS between releases. Embracing it means a more tested and usable master.
And mistakes that generate bad gerbers and production data will be devastating to the reputation of kicad.
Such “mistakes” aren’t an issue for ecosystems like JS that’ll just NIH another solution in search of a problem.
6 Likes
Point to the last bug like that
And again, if that’s a real concern, we need automated testing yesterday.
2 Likes
There’s alot of structural changes required to do so and not enough developers involved that want to do it. It’s entirely a problem of manpower. People are contributing to kicad in their own free time and/or paid to do it by a company, they don’t have interest in writing tests.
Some newer features are unit tested now but theres infinitely more tests to write.
Also thinking about it, what you want to do is just use the nightly, that’s literally the equivalent of the majority JS ecosystem
It’s automatically built every day, from the latest everything.
2 Likes
No. Because it requires an attitude change between master as testing and master as latest version. It means a lot more scrutiny when merging code and hopefully a more stable build without giving up 23+ months of improvements.
If someone has experience with any open-source tool for writing GUI automation tests, we’d love to get in touch. I don’t think anyone on the team right now has that experience. Writing non-GUI testing doesn’t have nearly as good a payoff given that the test coverage would be so low.
Well I can try something with the pcb_calculator for giggles. But I’m already annoyed that boost::test is being used instead of gtest >_< Just need to write some ducktape…
So many KiCad “bugs” are actually from workrounds for errors in third party frameworks like Boost and wxPython
Slow releases are inevitable when people who are willing to properly investigate and report bugs found using KiCad for real work, like me, won’t touch 5.99 with a bargepole until it finishes what I would call sandbox phase and reaches a certain level of stability
2 Likes
In a perfect world you don’t need a lot of GUI testing. In such ideal situation GUI event handlers are a thin shim translating GUI events into internal business logic data and method invocations. Then you unit test your internal logic without the GUI cruft and write simplistic tests for GUI event handlers with mocks for internal logic to cover the translation code and bam!, you have full test coverage.
Problem is KiCad is far from perfect in that regard, there is no clear separation of business logic and GUI code.
And that’s not something a newcomer can solve, this needs discipline and clear directive from project leadership: move toward testable code and add tests.
I strongly disagree with the last part here. Right now gitlab pipeline takes more than 3hours to run and it only does a linux build.
Imagine for a moment that every time you push a rev to a gitlab branch you get feedback in 3-5 minutes that you didn’t break any tests and you didn’t break any builds on all supported platforms. This allows the team to move completely to a model where every single commit goes through gitlab merge request and code review. It provides confidence that builds are not broken. It moves the bar of code quality higher.
This is doable today with dedicated machines.
Now add to this good test coverage and you are pretty close to being able to cut a release every month if you want. This one will of course take time and dedication.
Nickoe and Adam Wolf will likely disagree. Windows and mac builds still depend on a bunch of manual work.
You are talking about a different issue here. Coordinating codebase readiness with libs and docs and translations etc. will be somewhat manual but it’s not the thing that is impeding fast release cycle. Look at 5.1.7 release, it took what, 1-2 weeks from code being tagged to release.
My issue is with the whole “declare a feature freeze and hunker down for 3 months to hunt down bugs” approach. That kills development velocity.
I completely agree on this. But choosing a bad example doesn’t necessarily take away from @halachal’s point: there are plenty of large projects that adapt rapid release cycle model and enjoy it’s benefits. Not all of them are the dumpster fire that is js ecosystem.
That’s what tests are there to prevent. Or at least, they should be there before such release cycle model is adopted.
That right there is the reason why any individual can’t solve this problem. This needs to be a push from project leadership. Of course a volunteer coming in to add a random feature doesn’t see it as good use of his time to overhaul a bunch of existing code in addition to implementing a feature he cares about just to be able to write a unit test for his feature. So the problem perpetuates and nobody writes tests.
In every software projects life when it gets large enough it becomes a lot more time efficient to write tests than to not write them. KiCad is waay past that point in my opinion.
I touched on this earlier and it’s good to realize that you see the same problem as I. But I think your solution is not the best, you need to separate GUI from business logic first and then you can have a lot of ROI from simple non-gui unit tests.
=======
I apologize if this post reads like a rant of an entitled user who only demands things without taking into account the volunteer nature of the project and limitations of time and resources available for development.
It’s just that I like KiCad, I have invested a good chunk of my own time into it and my plugin for it and I would like to see the project succeed and grow further.
At the same time I look at it as a professional software engineer and can’t help but compare it to the process I am used to at my workplace and it’s just… painful. Kicad development is missing on so much quality of life things that would save you the precious volunteer time and raise their efficiency. But there just isn’t enough movement in that direction.
But I should also praise the movement that does happen. Switch to gitlab was a great step and I know a lot of work went into it.
Now it’s time to utilize the platform fully.
3 Likes
Also to add: I’m actually happy with Wayne’s answer that the team will attempt to move to yearly releases. That’s already much better than what we have now.