Yet another release date 6.x question

Community
12

Bobc is notoriously pessimistic in his predictions and has been right more often than notoriously optimistic other members of this forum :slight_smile:

“The first release candidate hopefully some time early next year” (meaning 2021) has been repeated many times, probably because the project leader has said it.

Unfortunately the time between rc1 and the final has been too long in the past. IMO the project should change the naming to what it actually is: nightly builds are alpha, rc1 is actually what other projects call beta1, i.e. new bugs are actually still expected. Release Candidate means usually “ready to be released as is unless new serious bugs are found”, only serious bugs lead to a new RC, and no known bugs have been postponed after RC1, not even a single one. That would make the expectations more realistic and the end of the waiting cycle more predictable in short term.

But that’s a smallish detail. My observations from the past tell that the most difficult problem for the project is lack of good testing. Every KiCad user could help with that. If KiCad would have got more testing before and after rc1 and more of the bugs would have been found earlier than later, the release schedule would have been tighter. As it is now, it has happened that a new bug is found, then a new one, then a new one… and suddenly we notice that 6 months has gone. Even though the version naming with KiCad Release Candidates have been way too optimistic, it would have been more realistic if testing would have covered more and revealed more of the bugs before the RC1.

Everyone can help to shorten the release cycle in this phase (after the feature freeze), and it will actually work. So everyone, please test, test, test, and report bugs. It will make the release cycle shorter and more predictable.

3 Likes
13

I’m not sure if the 5.99 is in the “feature freeze” stage right now (as it typically means, all the planned features are implemented), it’s rather “specification freeze” (i.e. “we’re not adding any new features for the release”).
But indeed, the more testing in early phase, the better quality of the release.
IMO when the “file formats freeze” will be announced, more projects will be moved to 5.99 and it will result in better testing.

2 Likes
14

Well spotted, another name to change. :slight_smile:
Indeed the real feature freeze and especially file format freeze are practically important, although it has been stated that the file formats won’t be completely frozen because there’s always possibility for bugfixes. Maybe it would be “file format feature freeze”.

1 Like
15

There is only one non-bugfix file format change planned that isn’t in yet (arcs in polygons). As this only impacts the PCB, you can consider the schematic and symbol formats “done” for V6.

As you mentioned, we could still make changes to the file formats to fix bugs, but there aren’t any open bugs requiring format changes that I’m aware of at the moment.

4 Likes
16

When that happens, I will start using 5.99. File format stability is so important and in my opinion marks the start of Beta phase

17

…and the bugs are found earlier only if people test the features. The only file format bugfix after the feature freeze which I’m aware of was the result of a forum discussion and/or a bug report: Discussion about filled polygon implementation details.

It’s intriguing to see how one new feature (rectangle) combined with a less used old feature (create circular pattern with rotation) led to an impossible situation and the file format had to be changed, leading also to better, more consistent behavior and UI and a new feature. What was the chance of this being noticed before the final release?

18

I would like to.

Where can I download the nightly builds for openSUSE Leap 15.2?

19

“I would far rather be happy than right”, the wise man said. :slight_smile:

Previously on forum.kicad.info When will Kicad 5 be available? where we had pretty much the same discussion with the same posters!, I predicted 5.0.0 final to be 2018-08, it was actually released about a month earlier in 2018-07.

Here is the 2021 version.

image
image802×623 28.5 KB

Btw, Happy New Year everyone! Let’s hope 2021 turns out better.

3 Likes
20

The V5 6 month lag from RC1 shows that RC1 was a Beta, not a Release Candidate
I wish that we only used RC for builds that are tentatively fit for launch, it is misleading to users.
I will still happily test a beta, but my expectations are very different

1 Like
21

The old thread is educating and painful reading. I suggest starting right from the RC1 (or almost) to save some time:

Compare that mess with what RC is expected to mean: a candidate for release as is, no bugs which are planned or tried to be fixed before the final release anymore. (KiCad has bad history of adding new bugs with bugfixes.) Even after the 5.0.0 release there were problems with packaging and for a while people didn’t know which version they had, buggy or fixed.

This really should be considered seriously. Beta quality isn’t RC quality and it’s misleading and bad PR to call RC something which isn’t nearly close to a release. Learn from the past, announce beta versions and keep RC for actual release candidates, and play safe so that even packaging is done and tested before the last RC so that only the version string can be changed for the final release (unless new serious bugs are found, of course). All minor bugs should be postponed to 6.0.1 even just before the RC1 and between each RC. Then the 6.0.1 released soon after 6.0.0.

Otherwise we will see another several months cycle between RC and final. I would rather see long beta phase and short RC cycle. That’s how the rest of the world uses the words.

4 Likes
22

I couldn’t agree more… but I guess it’s up to the developers, not the users and/or testers.

23

5.99 is still in development phase, where new implementations are worked upon. To me, that’s an “Alpha” stage.
If everything planned will be implemented, we’ll go to the “Beta” stage which is “feature freeze”. In this stage no features are implemented, and all efforts go to bug fixing.
RC stage to me means, no KNOWN bugs targeted for release are listed (so if there are any bugs left, they’re minor and scheduled for next releases to avoid regressions). RC stage should have as much workload as possible. If any showstopper is found, it will be fixed before release. Otherwise, green light for the release.
In this scenario, RC stage can be rather short as we’re not fixing bugs nor issue new bugreports; we should just confirm the readiness of RC for release. Anything minor goes to next releases, anything major (hopefully not) takes us back on schedule.

EDIT: seems that I’m moreorless right with my assumptions on Alpha/Beta/RC:

24

Tbh, I don’t think changing the names or meanings will change the process much. KiCad developers will continue to fix bugs, and tag releases when they think the code is ready.

Otoh, ‘adventurous’ users will continue to use nightly builds, regardless of release cycle. ‘Conservative’ users will only use builds tagged final release. ‘Helpful’ users will report bugs when they encounter them.

Therefore, declaring builds as “beta” or not is unlikely to significantly change how people use KiCad, nor make it more or less likely that users report bugs.

Unless KiCad starts to include some telemetry, developers don’t have a good way of telling what test coverage has been achieved, so it’s pretty much gut feeling.

8 Likes
25

You may be right. But still using more realistic naming together with better infrastructure (gitlab allows communication, milestones etc. better than launchpad), grown userbase, donations, the new company offering commercial support and bugfixes, more disciplined and strict planning etc. may help people orient better and have stronger trust in the project development cycle than the last time. And that would draw in more testers when new people get interested.

Even a couple of new active testers for beta phase (and of course the RC phase) can make a difference. 20 new active testers would be groundbreaking.

In the v5.0rc cycle the amount of commits which marked some bug in the database fixed was something like 250…300 between RC1 and RC2. Not counting other commits which weren’t marked but which fixed or changed something found by a developer. Anyone can understand that this is not a “release candidate”. I have already ranted too much and I want everybody know I really appreciate and admire the work the developers have done. But still I must say that this is just bad planning.

I know very well that the old cliche “to be taken seriously in professional world KiCad must…” is what it is, but if the project want’s to be professional, the release cycle must be professional, too. Whether or not it affects the actual outcome much. Version strings don’t make that difference alone but are still part of it because they are part of the planning and executing the cycle.

For the last RCs, on the other hand, the testing and new reports by users were much more important. If there comes a new crash report every now and then, say once a week, and the previous one is difficult and takes one week to fix, ten reports is enough to postpone the release two months. That’s not the fault of the testers, either, but it reveals lack of testing. Most of those bugs should be known in the beta phase.

Even when changing the names or meanings may not change the process much, because they are easy to change, why not do it? What happens between tags/names is of course more important and more difficult provided that the changes to the codebase are actually planned beforehand, and planning is what really counts. I have already seen it getting better for gitlab milestones compared to 5.0 and launchpad times.

The rest, testing and reporting, is up to us all. Ideas for making that better are welcome. Maybe we should start a new thread for that (this one has been hijacked for too long already).

3 Likes
26

What I said keeps haunting me. I have to apologize, I’m sorry.

The decisions back then were made using the available knowledge. The situation was unique. It wasn’t “bad planning” then, and that’s not what I meant. Although I obviously think it didn’t go perfectly, but everyone agrees with that.

We have more data and facts now to predict better what may happen. If and only if RC means what it normally means (ready to be released unless new serious bugs are found) to all parties involved and the final is released after six months from the first RC and after hundreds of changes, that would be bad planning now. We can learn from history. I believe it goes differently this time.

If the project wants to keep the aberrant meaning of RC, then OK. I just think that names are not only technical tags in the database. They are communication. Communication is based on common understanding of words. The project is growing bigger and more professional and needs communication which is more and more professional. Therefore I wish that the commonly accepted meanings of terms are accepted in the project even if it itself isn’t critical for the release schedule.

1 Like
27

People keep saying “more professional” but many professional software packages have loads of bugs.

There are always two things to deal with

  1. what people say they want
  2. what they actually need

People want a warm fuzzy feeling about the product, so marketing buzzwords can do that. But what they really need is bug free code and improved features. Marketing buzzwords don’t help with that.

There are several key things about software, which have remained true in the decades I have been in the industry:

  1. catch 22: You don’t want to release code to a wider audience until the bugs are found, but you can’t find bugs until you release the code to a wider audience.
  2. Writing software is really about learning. i.e humans slowly learning what works and what doesn’t. There is no pill to take which gives instant knowledge.
  3. Plans nearly always fail, because we don’t know what we don’t know. Unless you are doing the exact same thing you did before, you can’t be sure. By definition, if you are creating new things, there will be new unknowns.
  4. Often the best way to help developers is just to let them get on with it.
2 Likes
28

This is generally not true. You can catch vast majority of bugs using rigorous automatic testing. But this requires developer resources, machine resources and a polished dev process. Kicad like most open source projects unfortunately doesn’t have any of those.

29

Generally, projects do not have rigorous automatic testing. So yes it is generally true.

QED

I guess there is also
5) despite pointing out obvious realities, people immediately deny them. The first step to solving a problem is recognising the problem.

1 Like
30

It’s not true because it’s not a catch 22 when there is a 3rd valid option available: build rigorous automatic tests. I agree with the general message, specifically if applied to kicad. Still wouldn’t generalize it to all software though.

31

I think you are nitpicking, people seem to love arguing over semantics. My point of “a wider audience” was “to get wider test coverage”. That can be achieved by manual test or automated. An intelligent developer would understand that, and I think you to do.

In an ideal world, automation tests everything. Of course, there is still the problem that you don’t know everything that needs testing. Back in the real world, none of the professional projects I have worked on have had rigorous automatic testing. But we can always dream.

The fact that in real world projects you don’t know what bugs there until you test remains true, which was the point. I am sure you will find a way to disagree with that too.

1 Like