Skip to content
June 6, 2013 / Kamil Páral

The heroes of Fedora 19 Beta testing

1360095720_PrizeFedora 19 Beta was released last week. As usual, here are some interesting statistics from different areas of our testing efforts. No matter how large your contribution was, if you’ve helped us, thank you.

Fedora 19 Beta – wiki matrices

This is the list of people who filled in our release validation wiki matrices (Install, Desktop and Base), which are posted for every test compose (and release candidate) that we create. The purpose is to see which areas have been thoroughly tested and which were not.

Test period: Fedora 19 Alpha – Fedora 19 Beta
Testers: 27
Reports: 837
Unique referenced bugs: 63

Name Reports submitted Referenced bugs1
boblfoot 222 950487 951951 959605 959719 959793 959796 960837 962248 962569 963503 964586 965213 (12)
robatino 195 929177 946964 958424 958426 958427 958430 958436 959610 963547 964069 (10)
tassadar 72 920549 962039 962092 962701 965101 965663 965940 965974 966086 966095 966894 (11)
nonamedotc 71 951951 958436 958512 959796 (4)
adamwill 41 858270 950022 964352 966424 966598 (5)
Wutao85 40 963503 (1)
kparal 35 810112 892178 922885 950022 951951 (5)
satellit 34 922958 960045 96045 960791 (4)
lnie 23
pkotvan 13
fholec 13 929958 964587 965516 965539 965544 965633 965731 (7)
kevin 12 963238 964356 (2)
jskladan 11
mkrizek 11 948099 (1)
lbrabec 9 964147 964176 (2)
vicodan 9
prcek 7 967748 (1)
ausil 4 959610 (1)
pschindl 3 958697 966586 (2)
tflink 2 962831 (1)
mmarhefk 2
jsedlak 2
tasssadar 2
Lalatendu 1 966025 (1)
jthorne 1
adamw 1 958512 (1)
jpospisi 1

1 This is a list of bug reports linked to the wiki results. They don’t have to be reported by that concrete person.

The two titans waged a battle, and in the end, Bob Lightfoot (boblfoot) prevailed, for the first time! :-) Congratulations, Bob. Andre Robatino (robatino) very closely followed. After them, there is Vojtěch Boček (tassadar), a high school student who spent two weeks on internship in Red Hat, and then nonamedotc with nearly exactly the same score. The number one Red Hatter became Adam Williamson (adamwill), closely followed by a few other Red Hat employees and Thomas Gilliard (satellit). Great job, guys, thank you! Of course, everyone else’s help is fully appreciated as well.

Fedora 19 Beta – Bugzilla

This is a trimmed list of people who reported bugs into Bugzilla against Fedora 19 in the specified time period. Compared to Fedora 19 Alpha statistics, the already high numbers went even higher – the number of reporters went up by 50%. Nice to see.

Test period: Fedora 19 Alpha – Fedora 19 Beta (2013-04-24 – 2013-05-28)
Reporters: 463
New reports: 1538

Name Reports submitted1 Excess reports2 Accepted blockers3
Adam Williamson 60 4 (6%) 2
Dhiru Kholia 49 1 (2%) 0
Mikhail 32 6 (18%) 0
Reartes Guillermo 29 6 (20%) 1
Stef Walter 29 1 (3%) 0
Chris Murphy 27 1 (3%) 5
vinesh teotia 27 0 (0%) 0
Andre Robatino 20 11 (55%) 4
Robert Lightfoot 18 2 (11%) 1
Lennart Poettering 18 0 (0%) 0
Ankur Sinha (FranciscoD) 16 1 (6%) 0
Patrik Kis 16 1 (6%) 0
Vojtěch Boček 16 1 (6%) 0
Dean Hunter 15 1 (6%) 0
IBM Bug Proxy 15 1 (6%) 0
Zbigniew Jędrzejewski-Szmek 15 0 (0%) 0
Grosswiler Roger 14 0 (0%) 0
Igor Gnatenko 14 4 (28%) 0
Luis Bazan 14 0 (0%) 0
Mikolaj Izdebski 13 0 (0%) 0
D. Charles Pyle 12 3 (25%) 0
David Spurek 12 3 (25%) 0
Hans de Goede 12 0 (0%) 0
Jeff Bastian 12 1 (8%) 0
Mark Salter 12 0 (0%) 0
Martin Banas 12 1 (8%) 0
Bill Nottingham 10 1 (10%) 0
Flóki Pálsson 10 2 (20%) 0
Jan Stodola 10 1 (10%) 0
John Reiser 10 0 (0%) 0
nucleo 10 1 (10%) 0
quickbooks.office at gmail.com 10 1 (10%) 0
Lawrenc Graves 9 0 (0%) 0
Matthew Miller 9 0 (0%) 0
Michael Scherer 9 0 (0%) 0
Mike FABIAN 9 1 (11%) 0
Need Real Name 9 4 (44%) 0
Nix\ 9 0 (0%) 0
satellit at bendbroadband.com 9 0 (0%) 0
Stephen Gallagher 9 1 (11%) 0
Dan Horák 8 0 (0%) 1
Alex Murray 8 0 (0%) 0
Amit Shah 8 0 (0%) 0
Jens Petersen 8 1 (12%) 0
Joachim Backes 8 2 (25%) 0
Marian Ganisin 8 1 (12%) 0
Mark Hamzy 8 2 (25%) 0
Orion Poplawski 8 1 (12%) 0
Peter Robinson 8 0 (0%) 0
Petr Schindler 8 1 (12%) 0
Simon Lewis 8 1 (12%) 0
Tomas Dolezal 8 0 (0%) 0
A.J. Werkman 7 0 (0%) 0
David Jaša 7 0 (0%) 0
Ed Greshko 7 0 (0%) 0
Richard W.M. Jones 7 0 (0%) 0
Evan Nemerson 6 0 (0%) 0
Jay Finger 6 0 (0%) 0
Jiri Koten 6 0 (0%) 0
Lukas -krtek.net- Novy 6 0 (0%) 0
Parag 6 1 (16%) 0
R P Herrold 6 4 (66%) 0
Vít Ondruch 6 1 (16%) 0
Pavel Holica 5 1 (20%) 1
a554335752 5 0 (0%) 0
Ales Ledvinka 5 0 (0%) 0
Andy Lawrence 5 0 (0%) 0
Cole Robinson 5 0 (0%) 0
Dan Mashal 5 0 (0%) 0
David Woodhouse 5 0 (0%) 0
Filip Holec 5 0 (0%) 0
Hein Kerstgens 5 0 (0%) 0
Joel 5 0 (0%) 0
Kalev Lember 5 0 (0%) 0
kuchiman 5 0 (0%) 0
Matthias Runge 5 0 (0%) 0
Michal Domonkos 5 0 (0%) 0
Patryk Zawadzki 5 0 (0%) 0
Philipp Dreimann 5 0 (0%) 0
Tim Flink 5 1 (20%) 0
Tim Waugh 5 1 (20%) 0
wangjiezhe at gmail.com 5 0 (0%) 0
William Brown 5 3 (60%) 0
…and also 380 other reporters who created less than 5 reports each, but 611 reports combined!

1 The total number of new reports (including “excess reports”). Reopened reports or reports with a changed version are not included, because it was not technically easy to retrieve those. This is one of the reasons why you shouldn’t take the numbers too seriously, but just as interesting and fun data.
2 Excess reports are those that were closed as NOTABUG, WONTFIX, WORKSFORME, CANTFIX or INSUFFICIENT_DATA. Excess reports are not necessarily a bad thing, but they make for interesting statistics. Close manual inspection is required to separate valuable excess reports from those which are less valuable.
3 This only includes reports that were created by that particular user and accepted as blockers afterwards. The user might have proposed other people’s reports as blockers, but this is not reflected in this number.

The gold crown in the number of bug reports is won by our famous “community monkey”, Red Hatter Adam Williamson. Dhiru Kholia continues to report packaging issues, as before. Mikhail and Reartes Guillermo lead the community efforts in standard bug reporting, together with Chris Murphy, vinesh teotia, Andre Robatino, Robert Lightfoot and others. Chris and Andre were very successful in proposing Beta blocker bugs.

It’s always nice to see how many people participate in bug reporting, the numbers are breathtaking. I hope it doesn’t mean that our quality is really low :-) I believe this indicates the opposite – we have an amazing number of people trying to ensure the quality is as high as possible. Thank you.

Fedora 19 Beta – updates testing

This is a trimmed list of people who provided feedback in Bodhi for all updates proposed into Fedora 19 in the specified time period. The purpose is to make sure that broken updates do not enter the stable repository and the release continuously stabilizes.

Test period: Fedora 19 Alpha – Fedora 19 Beta (2013-04-24 – 2013-05-28)
Testers: 164
Comments1: 1054

Name Updates commented
Robert C. Lightfoot (boblfoot) 117
Adam Williamson (adamwill) 102
misc 70
Piotr Drąg (raven) 56
Igor Gnatenko (ignatenkobrain) 55
Kevin Fenzi (kevin) 52
bitlord 47
T.C. Hollingsworth (patches) 45
Ankur Sinha (ankursinha) 42
Kalev Lember (kalev) 36
Alexander Kurtakov (akurtakov) 30
nonamedotc 28
Matthias Runge (mrunge) 25
Michael Schwendt (mschwendt) 18
gwei3 11
Jack Magne (jmagne) 10
Vasileios Keramidas (keramidas) 9
nucleo 9
Rex Dieter (rdieter) 8
mharmsen 8
Dan Mashal (vicodan) 8
Fabio Valentini (fafatheone) 8
Orion Poplawski (orion) 7
Arthur Scott Poore (spoore) 7
Tom Callaway (spot) 7
Heiko Adams (heikoada) 7
Wolfgang Ulbrich (raveit65) 6
Zbigniew Jędrzejewski-Szmek (zbyszek) 6
Axilleas Pipinellis (axilleas) 5
Christopher Meng (cicku) 5
Adam Jackson (ajax) 5
Tim Flink (tflink) 5
Joel Burleson-Davis (sysengbd) 4
Vojtech Bocek (tassadar) 4
Tao Wu (wutao85) 4
Mamoru Tasaka (mtasaka) 4
robatino 4
Endi Sukma Dewata (edewata) 4
Elad Alfassa (elad) 3
David A. Marlin (dmarlin) 3
Mike FABIAN (mfabian) 3
Stef Walter (stefw) 3
Chris Murphy (chrismurphy) 3
David King (amigadave) 3
Daniel J Walsh (dwalsh) 3
halfie 3
John Reiser (jreiser) 3
Patrik Kis (pkis) 3
Petr Schindler (pschindl) 3
Martin Krizek (mkrizek) 3
…and also 114 other reporters who created less than 3 reports each, but 140 reports combined!

1 If a person provides multiple comments to a single update, it is considered as a single comment. Karma value is not taken into account.

When compared to F19 Alpha statistics, the number of reporters increased by half and the number of comments increased more than 3 times. Nice. This is one of the easiest way how to contribute to Fedora quality and I’m very glad that the numbers soared so much. Our hero is, as usual, Robert C. Lightfoot (boblfoot)! The other high-profile community contributors include misc, Piotr Drąg (raven), Igor Gnatenko (ignatenkobrain), bitlordT.C. Hollingsworth (patches), Ankur Sinha (ankursinha) and others. The most active Red Hatter was, once again, Adam Williamson. Great job, everyone.

Summary

I’m very glad to see a visible increase in test activities since F19 Alpha. Surely, the officially released milestone composes (i.e. Alpha, Beta) help to attract more testers than just the regular contributors base. It will be interesting to see how the picture changes between Fedora 19 Beta and Final.

If you are interested in raising the Fedora quality up, help us out. Read QA/Join, follow the announcements and talk to us in #fedora-qa on IRC and test list. We will love to see you!


When reading the statistics, please take it with a grain of salt. The numbers are not directly comparable. People might see some reports as more valuable than others. Some people tested a lot of components, but haven’t found many problems (but that also helps). Some people used their skills in other areas than those mentioned. This is not meant to be a comparison chart, but a well-meant “thank you” letter.
The statistics were generated by these scripts.

Leave a Reply (Markdown syntax supported)

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: