Fedora 19 the Schrödinger’s Cat is out of the box, and alive! I have again gathered some interesting statistics about QA contributions, this time during the Final phase (between Beta and Final release).
Fedora 19 Final – wiki matrices
This is the list of people who filled in our release validation wiki matrices (Install, Desktop and Base), which are posted for every test compose (and release candidate) that we create. The purpose is to see which areas have been thoroughly tested and which were not.
Test period: Fedora 19 Beta – Fedora 19 Final
Testers: 29
Reports: 1035
Unique referenced bugs: 58
1 This is a list of bug reports linked to the wiki results. They don’t have to be reported by that concrete person.
Who would have thought… Andre Robatino (robatino) is at the top again. It’s also very nice to see how many bugs he referenced during his testing. The more referenced bugs the more publicity, the more likeliness that someone reproduces them, the more likeliness someone fixes them. Thanks, Andre!
The community participation list continues with nonamedotc, who provided an amazing number of test case results as well. Then there is a long list of Red Hatters, the highest score split between Adam Williamson (adamwill) and Kamil Páral (kparal) (a.k.a. me). Since I referenced more bugs, I consider myself a winner 😛 “I only believe in statistics that I doctored myself.” 🙂
The community participation continues with Bob Lightfoot (boblfoot), who had been greatly active during Beta, John Reiser (jreiser), Thomas Gilliard (satellit) and others. Thank you, everyone.
Fedora 19 Final – Bugzilla
This is a trimmed list of people who reported bugs into Bugzilla against Fedora 19 in the specified time period. When compared to Fedora 19 Beta, the numbers went up again, especially the number of reporters. That’s really great.
Test period: Fedora 19 Beta – Fedora 19 Final (2013-05-29 – 2013-06-28)
Reporters: 614
New reports: 1486
| Name |
Reports submitted1 |
Excess reports2 |
Accepted blockers3 |
| IBM Bug Proxy |
31 |
2 (6%) |
0 |
| Mark Salter |
31 |
2 (6%) |
0 |
| Adam Williamson |
27 |
1 (3%) |
2 |
| quickbooks.office at gmail.com |
27 |
1 (3%) |
1 |
| Mikhail |
26 |
1 (3%) |
0 |
| David Woodhouse |
23 |
0 (0%) |
0 |
| Kamil Páral |
22 |
2 (9%) |
4 |
| Andre Robatino |
21 |
18 (85%) |
1 |
| Zbigniew Jędrzejewski-Szmek |
17 |
0 (0%) |
0 |
| Dan Mashal |
16 |
4 (25%) |
0 |
| Karel Volný |
16 |
0 (0%) |
0 |
| Vojtěch Boček |
13 |
0 (0%) |
2 |
| Emmanuel Pacaud |
13 |
0 (0%) |
0 |
| Marian |
12 |
3 (25%) |
0 |
| Jan Sedlák |
11 |
0 (0%) |
0 |
| Lukas Brabec |
11 |
1 (9%) |
0 |
| Martin Holec |
11 |
2 (18%) |
0 |
| Tim Waugh |
11 |
0 (0%) |
0 |
| A.J. Werkman |
10 |
0 (0%) |
0 |
| Florian Weimer |
10 |
0 (0%) |
0 |
| Heiko Adams |
10 |
0 (0%) |
0 |
| Joachim Backes |
10 |
3 (30%) |
0 |
| Chris Murphy |
9 |
0 (0%) |
1 |
| Bojan Smojver |
9 |
0 (0%) |
0 |
| Diogo Campos |
9 |
0 (0%) |
0 |
| Jonas Thiem |
9 |
0 (0%) |
0 |
| David Jaša |
8 |
1 (12%) |
0 |
| Dean Hunter |
8 |
1 (12%) |
0 |
| Endi Sukma Dewata |
8 |
2 (25%) |
0 |
| James |
8 |
2 (25%) |
0 |
| Jeff Bastian |
7 |
1 (14%) |
1 |
| Jiří Martínek |
7 |
1 (14%) |
0 |
| Marcus Moeller |
7 |
3 (42%) |
0 |
| Matthew Miller |
7 |
0 (0%) |
0 |
| Simon Lewis |
7 |
1 (14%) |
0 |
| TKS |
7 |
1 (14%) |
0 |
| Mark Hamzy |
6 |
0 (0%) |
1 |
| Nix\ |
6 |
2 (33%) |
1 |
| arnav |
6 |
2 (33%) |
0 |
| Kalev Lember |
6 |
1 (16%) |
0 |
| Krzysztof Daniel |
6 |
0 (0%) |
0 |
| Rolle |
6 |
1 (16%) |
0 |
| Russ Anderson |
6 |
1 (16%) |
0 |
| Siddhesh Poyarekar |
6 |
0 (0%) |
0 |
| James Heather |
5 |
0 (0%) |
1 |
| Jiri Eischmann |
5 |
1 (20%) |
1 |
| Michael Scherer |
5 |
0 (0%) |
1 |
| Alexey Derlaft |
5 |
2 (40%) |
0 |
| Andrea Oliveri |
5 |
1 (20%) |
0 |
| Björn Esser |
5 |
0 (0%) |
0 |
| D. Charles Pyle |
5 |
0 (0%) |
0 |
| Dennis Gilmore |
5 |
0 (0%) |
0 |
| Edgar Hoch |
5 |
0 (0%) |
0 |
| Flóki Pálsson |
5 |
1 (20%) |
0 |
| Francisco de la Peña |
5 |
1 (20%) |
0 |
| Joshua Holm |
5 |
0 (0%) |
0 |
| Konrad Rzeszutek Wilk |
5 |
0 (0%) |
0 |
| Lukas -krtek.net- Novy |
5 |
2 (40%) |
0 |
| mark |
5 |
0 (0%) |
0 |
| Matthias Clasen |
5 |
0 (0%) |
0 |
| nonamedotc at gmail.com |
5 |
2 (40%) |
0 |
| Parag |
5 |
2 (40%) |
0 |
| Paul Whalen |
5 |
0 (0%) |
0 |
| Reartes Guillermo |
5 |
0 (0%) |
0 |
| Vít Ondruch |
5 |
0 (0%) |
0 |
| Álvaro Castillo |
5 |
0 (0%) |
0 |
| Štefan Gurský |
5 |
0 (0%) |
0 |
| …and also 547 other reporters who created less than 5 reports each, but 834 reports combined! |
1 The total number of new reports (including “excess reports”). Reopened reports or reports with a changed version are not included, because it was not technically easy to retrieve those. This is one of the reasons why you shouldn’t take the numbers too seriously, but just as interesting and fun data.
2 Excess reports are those that were closed as NOTABUG, WONTFIX, WORKSFORME, CANTFIX or INSUFFICIENT_DATA. Excess reports are not necessarily a bad thing, but they make for interesting statistics. Close manual inspection is required to separate valuable excess reports from those which are less valuable.
3 This only includes reports that were created by that particular user and accepted as blockers afterwards. The user might have proposed other people’s reports as blockers, but this is not reflected in this number.
The top reporters are nicely and evenly distributed, there are no extreme performers as in the wiki statistics. The top position share IBM Bug Proxy and Mark Salter. Mark reported a number of build failures for ARM architecture. I assume IBM Bug Proxy is probably a shared account among IBM employees who care about Fedora development – they reported a number of anaconda, kernel and boot issues. It’s always nice to see other companies involved in raising the Fedora quality – welcome, IBM 🙂
When it comes to general bug reporting, the top Red Hatter is Adam Williamson. As for the community, quickbooks.office reported the same number of bugs, most of them being SELinux error duplicates. Mikhail closely follows with a much more interesting mix of bug reports, which also applies for David Woodhouse. The community list follows with Andre Robatino (mostly fake bug reports for test purposes), Zbigniew Jędrzejewski-Szmek, Dan Mashal and others.
It’s great to see so many bug reports and so even distribution of the top reporters. Thanks, all of you.
Fedora 19 Beta – updates testing
This is a trimmed list of people who provided feedback in Bodhi for all updates proposed into Fedora 19 in the specified time period. The purpose is to make sure that broken updates do not enter the stable repository and the release continuously stabilizes. The number of testers increased considerably again as compared to Beta.
Test period: Fedora 19 Beta – Fedora 19 Final (2013-05-29 – 2013-06-28)
Testers: 240
Comments1: 1107
If a person provides multiple comments to a single update, it is considered as a single comment. Karma value is not taken into account.
A new star has emerged, Igor Gnatenko (ignatenkobrain) completely dominated here. Great job, Igor! The top performing Red Hatter was, yet again, Adam Williamson (adamwill).
The top community list, apart from Igor, also contains nonamedotc, Kalev Lember (kalev), bitlord, T.C. Hollingsworth (patches), quickbooks, Piotr Drąg (raven), and others. It’s great to see so many people involved, because testing proposed updates and providing karma is one of the easiest way to contribute, yet extremely important at the same time. Thanks to anyone who contributed!
Summary
For most parts, the number of reports stays similar to Beta, but the number of people involved rose considerably. Beta release seems to be very attractive to the Fedora community and rightly so, it’s the last step before everything is pronounced stable. We’re glad to see so many people involved in release testing and we hope that we will see you again in Fedora 20 cycle!
In the meantime you can help as well – for example testing the proposed updates for stable Fedora releases is as useful, maybe even more important, than for releases in development. You can also play with Rawhide and help us find the bugs well in advance. There is a lot of possibilities – if it sounds interesting to you, be sure to have a look at QA/Join, follow the announcements and talk to us in #fedora-qa on IRC and test list. Your involvement will help not only Fedora and its users, but the whole Linux ecosystem. Thanks!
When reading the statistics, please take it with a grain of salt. The numbers are not directly comparable. People might see some reports as more valuable than others. Some people tested a lot of components, but haven’t found many problems (but that also helps). Some people used their skills in other areas than those mentioned. This is not meant to be a comparison chart, but a well-meant “thank you” letter.
The statistics were generated by these scripts.
Another quarter is gone, let’s see how much testing of proposed updates in Fedora we have done. The purpose of this testing is to make sure that broken updates don’t enter our stable (or soon-to-be-stable) Fedora releases. The test feedback is managed inside Bodhi application and here are the statistics for the recent quarter:
Kparal's Fedora Blog 