Fedora 19 the Schrödinger’s Cat is out of the box, and alive! I have again gathered some interesting statistics about QA contributions, this time during the Final phase (between Beta and Final release).
Fedora 19 Final – wiki matrices
This is the list of people who filled in our release validation wiki matrices (Install, Desktop and Base), which are posted for every test compose (and release candidate) that we create. The purpose is to see which areas have been thoroughly tested and which were not.
Test period: Fedora 19 Beta – Fedora 19 Final
Testers: 29
Reports: 1035
Unique referenced bugs: 58
| Name | Reports submitted | Referenced bugs1 |
|---|---|---|
| robatino | 259 | 929177 946964 958426 958427 972025 972046 974050 974052 975227 975230 975324 975325 976280 976281 977669 977671 977715 977732 978114 978124 978125 979171 979172 (23) |
| nonamedotc | 127 | 972225 977879 (2) |
| adamwill | 103 | 858270 892178 975800 978008 (4) |
| kparal | 103 | 973068 974032 974038 977715 977816 977962 (6) |
| Wutao85 | 55 | |
| lbrabec | 49 | 855824 858270 977962 978298 (4) |
| jpospisi | 48 | 964586 975483 (2) |
| eischmann | 39 | |
| lnie | 24 | |
| pkotvan | 23 | 969684 975813 (2) |
| jskladan | 21 | |
| tflink | 21 | 975495 (1) |
| mkrizek | 21 | 977962 (1) |
| jsedlak | 19 | 973747 975375 978346 (3) |
| pschindl | 18 | |
| lnovy | 16 | 743281 858270 (2) |
| mmarhefk | 16 | 892178 959796 975521 (3) |
| Martix | 15 | 976417 (1) |
| boblfoot | 12 | 971191 (1) |
| jreiser | 12 | 976582 (1) |
| kevin | 10 | |
| satellit | 10 | |
| werkman | 5 | 971109 971255 976034 (3) |
| todoleza | 3 | 970988 (1) |
| konradr | 2 | 977974 977987 978036 (3) |
| mattdm | 1 | |
| jreznik | 1 | 705086 (1) |
| mooninite | 1 | |
| dgilmore | 1 |
1 This is a list of bug reports linked to the wiki results. They don’t have to be reported by that concrete person.
Who would have thought… Andre Robatino (robatino) is at the top again. It’s also very nice to see how many bugs he referenced during his testing. The more referenced bugs the more publicity, the more likeliness that someone reproduces them, the more likeliness someone fixes them. Thanks, Andre!
The community participation list continues with nonamedotc, who provided an amazing number of test case results as well. Then there is a long list of Red Hatters, the highest score split between Adam Williamson (adamwill) and Kamil Páral (kparal) (a.k.a. me). Since I referenced more bugs, I consider myself a winner 😛 “I only believe in statistics that I doctored myself.” 🙂
The community participation continues with Bob Lightfoot (boblfoot), who had been greatly active during Beta, John Reiser (jreiser), Thomas Gilliard (satellit) and others. Thank you, everyone.
Fedora 19 Final – Bugzilla
This is a trimmed list of people who reported bugs into Bugzilla against Fedora 19 in the specified time period. When compared to Fedora 19 Beta, the numbers went up again, especially the number of reporters. That’s really great.
Test period: Fedora 19 Beta – Fedora 19 Final (2013-05-29 – 2013-06-28)
Reporters: 614
New reports: 1486
| Name | Reports submitted1 | Excess reports2 | Accepted blockers3 |
|---|---|---|---|
| IBM Bug Proxy | 31 | 2 (6%) | 0 |
| Mark Salter | 31 | 2 (6%) | 0 |
| Adam Williamson | 27 | 1 (3%) | 2 |
| quickbooks.office at gmail.com | 27 | 1 (3%) | 1 |
| Mikhail | 26 | 1 (3%) | 0 |
| David Woodhouse | 23 | 0 (0%) | 0 |
| Kamil Páral | 22 | 2 (9%) | 4 |
| Andre Robatino | 21 | 18 (85%) | 1 |
| Zbigniew Jędrzejewski-Szmek | 17 | 0 (0%) | 0 |
| Dan Mashal | 16 | 4 (25%) | 0 |
| Karel Volný | 16 | 0 (0%) | 0 |
| Vojtěch Boček | 13 | 0 (0%) | 2 |
| Emmanuel Pacaud | 13 | 0 (0%) | 0 |
| Marian | 12 | 3 (25%) | 0 |
| Jan Sedlák | 11 | 0 (0%) | 0 |
| Lukas Brabec | 11 | 1 (9%) | 0 |
| Martin Holec | 11 | 2 (18%) | 0 |
| Tim Waugh | 11 | 0 (0%) | 0 |
| A.J. Werkman | 10 | 0 (0%) | 0 |
| Florian Weimer | 10 | 0 (0%) | 0 |
| Heiko Adams | 10 | 0 (0%) | 0 |
| Joachim Backes | 10 | 3 (30%) | 0 |
| Chris Murphy | 9 | 0 (0%) | 1 |
| Bojan Smojver | 9 | 0 (0%) | 0 |
| Diogo Campos | 9 | 0 (0%) | 0 |
| Jonas Thiem | 9 | 0 (0%) | 0 |
| David Jaša | 8 | 1 (12%) | 0 |
| Dean Hunter | 8 | 1 (12%) | 0 |
| Endi Sukma Dewata | 8 | 2 (25%) | 0 |
| James | 8 | 2 (25%) | 0 |
| Jeff Bastian | 7 | 1 (14%) | 1 |
| Jiří Martínek | 7 | 1 (14%) | 0 |
| Marcus Moeller | 7 | 3 (42%) | 0 |
| Matthew Miller | 7 | 0 (0%) | 0 |
| Simon Lewis | 7 | 1 (14%) | 0 |
| TKS | 7 | 1 (14%) | 0 |
| Mark Hamzy | 6 | 0 (0%) | 1 |
| Nix\ | 6 | 2 (33%) | 1 |
| arnav | 6 | 2 (33%) | 0 |
| Kalev Lember | 6 | 1 (16%) | 0 |
| Krzysztof Daniel | 6 | 0 (0%) | 0 |
| Rolle | 6 | 1 (16%) | 0 |
| Russ Anderson | 6 | 1 (16%) | 0 |
| Siddhesh Poyarekar | 6 | 0 (0%) | 0 |
| James Heather | 5 | 0 (0%) | 1 |
| Jiri Eischmann | 5 | 1 (20%) | 1 |
| Michael Scherer | 5 | 0 (0%) | 1 |
| Alexey Derlaft | 5 | 2 (40%) | 0 |
| Andrea Oliveri | 5 | 1 (20%) | 0 |
| Björn Esser | 5 | 0 (0%) | 0 |
| D. Charles Pyle | 5 | 0 (0%) | 0 |
| Dennis Gilmore | 5 | 0 (0%) | 0 |
| Edgar Hoch | 5 | 0 (0%) | 0 |
| Flóki Pálsson | 5 | 1 (20%) | 0 |
| Francisco de la Peña | 5 | 1 (20%) | 0 |
| Joshua Holm | 5 | 0 (0%) | 0 |
| Konrad Rzeszutek Wilk | 5 | 0 (0%) | 0 |
| Lukas -krtek.net- Novy | 5 | 2 (40%) | 0 |
| mark | 5 | 0 (0%) | 0 |
| Matthias Clasen | 5 | 0 (0%) | 0 |
| nonamedotc at gmail.com | 5 | 2 (40%) | 0 |
| Parag | 5 | 2 (40%) | 0 |
| Paul Whalen | 5 | 0 (0%) | 0 |
| Reartes Guillermo | 5 | 0 (0%) | 0 |
| Vít Ondruch | 5 | 0 (0%) | 0 |
| Álvaro Castillo | 5 | 0 (0%) | 0 |
| Štefan Gurský | 5 | 0 (0%) | 0 |
| …and also 547 other reporters who created less than 5 reports each, but 834 reports combined! | |||
1 The total number of new reports (including “excess reports”). Reopened reports or reports with a changed version are not included, because it was not technically easy to retrieve those. This is one of the reasons why you shouldn’t take the numbers too seriously, but just as interesting and fun data.
2 Excess reports are those that were closed as NOTABUG, WONTFIX, WORKSFORME, CANTFIX or INSUFFICIENT_DATA. Excess reports are not necessarily a bad thing, but they make for interesting statistics. Close manual inspection is required to separate valuable excess reports from those which are less valuable.
3 This only includes reports that were created by that particular user and accepted as blockers afterwards. The user might have proposed other people’s reports as blockers, but this is not reflected in this number.
The top reporters are nicely and evenly distributed, there are no extreme performers as in the wiki statistics. The top position share IBM Bug Proxy and Mark Salter. Mark reported a number of build failures for ARM architecture. I assume IBM Bug Proxy is probably a shared account among IBM employees who care about Fedora development – they reported a number of anaconda, kernel and boot issues. It’s always nice to see other companies involved in raising the Fedora quality – welcome, IBM 🙂
When it comes to general bug reporting, the top Red Hatter is Adam Williamson. As for the community, quickbooks.office reported the same number of bugs, most of them being SELinux error duplicates. Mikhail closely follows with a much more interesting mix of bug reports, which also applies for David Woodhouse. The community list follows with Andre Robatino (mostly fake bug reports for test purposes), Zbigniew Jędrzejewski-Szmek, Dan Mashal and others.
It’s great to see so many bug reports and so even distribution of the top reporters. Thanks, all of you.
Fedora 19 Beta – updates testing
This is a trimmed list of people who provided feedback in Bodhi for all updates proposed into Fedora 19 in the specified time period. The purpose is to make sure that broken updates do not enter the stable repository and the release continuously stabilizes. The number of testers increased considerably again as compared to Beta.
Test period: Fedora 19 Beta – Fedora 19 Final (2013-05-29 – 2013-06-28)
Testers: 240
Comments1: 1107
1 If a person provides multiple comments to a single update, it is considered as a single comment. Karma value is not taken into account.
A new star has emerged, Igor Gnatenko (ignatenkobrain) completely dominated here. Great job, Igor! The top performing Red Hatter was, yet again, Adam Williamson (adamwill).
The top community list, apart from Igor, also contains nonamedotc, Kalev Lember (kalev), bitlord, T.C. Hollingsworth (patches), quickbooks, Piotr Drąg (raven), and others. It’s great to see so many people involved, because testing proposed updates and providing karma is one of the easiest way to contribute, yet extremely important at the same time. Thanks to anyone who contributed!
Summary
For most parts, the number of reports stays similar to Beta, but the number of people involved rose considerably. Beta release seems to be very attractive to the Fedora community and rightly so, it’s the last step before everything is pronounced stable. We’re glad to see so many people involved in release testing and we hope that we will see you again in Fedora 20 cycle!
In the meantime you can help as well – for example testing the proposed updates for stable Fedora releases is as useful, maybe even more important, than for releases in development. You can also play with Rawhide and help us find the bugs well in advance. There is a lot of possibilities – if it sounds interesting to you, be sure to have a look at QA/Join, follow the announcements and talk to us in #fedora-qa on IRC and test list. Your involvement will help not only Fedora and its users, but the whole Linux ecosystem. Thanks!
When reading the statistics, please take it with a grain of salt. The numbers are not directly comparable. People might see some reports as more valuable than others. Some people tested a lot of components, but haven’t found many problems (but that also helps). Some people used their skills in other areas than those mentioned. This is not meant to be a comparison chart, but a well-meant “thank you” letter.
The statistics were generated by these scripts.
Kparal's Fedora Blog 
“IBM Bug Proxy” is more a ‘Red Hat’ thing than a Fedora thing, most reports from IBMers go through some kind of internal reporting system over at IBM which then sends them out ‘upstream’, so bugs to do with RH stuff come out to our Bugzilla. You see a lot of such reports if you look at RHEL bugs, IBM is a major RHEL customer. It is great that they’re testing F19, though, indeed.
What’s up with the Red Hat and community separation? You make it sound like the “community” is a second class member. Please don’t make this distinction and don’t post separate lists.
On the contrary, the point of the blog is to encourage community contributions by highlighting them. People might be happy to see their contributions “recognized”. The distinction being shown here is just to highlight the community contribution, in my opinion.
Unlike the developers, testers are practically invisible. So, this blog by Kamil, in my opinion, will definitely encourage community participation.
I’m trying to highlight community members by explicitly naming them out in the description. I don’t do that for Red Hatters that much, just naming the first one, usually.
Since the public doesn’t know who is a Red Hatter and who is not, people could easily say “all of those are Red Hatters anyway”. I’m trying to show that it’s not true (more often that not the most active members are from the community) and also thank these community people publicly. Red Hatters do these tasks often (although not always) as part of their daily job, which puts even more praise onto the community top-performers.
I think the slight distinction is an interesting information, but if you have some advice how to it better, I’ll love to hear that.