Taskotron: depcheck task replaced by rpmdeplint

If you are a Fedora packager, you might be interested to know that in Taskotron we replaced the depcheck task with rpmdeplint task. So if there are any dependency issues with the new update you submit to Bodhi, you’ll see that as dist.rpmdeplint failure (in the Automated Tests tab). The failure logs should look very similar to the depcheck ones (basically, the logs contain the errors dnf would spit out if it tried to install that package), so there should be no transitioning effort needed.

If you listen for depcheck results somehow, i.e. in FMN, make sure to update your rules to listen for dist.rpmdeplint instead. We have updated the default filters in FMN, so if you haven’t changed them, you should receive notifications for failures in rpmdeplint (and also upgradepath and abicheck) for submitted updates owned by you.

The reason for this switch is that we wanted to get rid of custom dependency checking (done directly on top of libsolv), and use an existing tool for that instead. That saves us time, we don’t need to study all the pitfalls of dependency resolution, and we benefit from someone else maintaining and developing the tool (that doesn’t mean we won’t send patches if needed). rpmdeplint offered exactly what we were looking for.

We will decommission depcheck task from Taskotron execution in the next few days, if there are no issues. Rpmdeplint results are already being published for all proposed updates.

If you have any questions, please ask in comments or reach us at #fedora-qa freenode irc channel or qa-devel (or test or devel) mailing list.

Taskotron: depcheck task replaced by rpmdeplint

Welcome Fedora Quality Planet

Hello, I’d like to introduce a new sub-planet of Fedora Planet to you, located at http://fedoraplanet.org/quality/ (you don’t need to remember the URL, there’s a sub-planet picker in the top right corner of Fedora Planet pages that allows you to switch between sub-planets).

Fedora Quality Planet will contain news and useful information about QA tools and processes present in Fedora, updates on our quality automation efforts, guides for package maintainers (and other teams) how to interact with our tools and checks or understand the reported failures, announcements about critical issues in Fedora releases, and more.

Our goal is to have a single place for you to visit (or subscribe to) and get a good overview of what’s happening in the Fedora Quality space. Of course all Fedora Quality posts should also show up in the main Fedora Planet feed, so if you’re already subscribed to that, you shouldn’t miss our posts either.

If you want to join our effort and publish some interesting quality-related posts into Fedora Quality Planet, you’re more then welcome! Please see the instructions how to syndicate your blog. If you have any questions or need help, ask in the test mailing list or ping kparal or adamw on #fedora-qa freenode IRC channel. Thanks!

Welcome Fedora Quality Planet

‘Package XXX is not signed’ error during upgrade to Fedora 24

Many people hit issues like this when trying to upgrade to Fedora 24:

 Error: Package a52dec-0.7.4-19.fc24.x86_64.rpm is not signed

You can easily see that this is a very widespread issue if you look at comments section under our upgrade guide on fedora magazine. In fact, this issue probably affects everyone who has rpmfusion repository enabled (which is a very popular third-party repository). Usually the a52dec package is mentioned, because it’s early in the alphabet listing, but it can be a different one (depending on what you installed from rpmfusion).

The core issue is that even though their Fedora 24 repository is available, the packages in it are not signed yet – they simply did not have time to do that yet. However, rpmfusion repository metadata from Fedora 23 demand that all packages are signed (which is a good thing, package signing is crucial to prevent all kinds of nasty security attacks). The outcome is that DNF rejects the transaction for being unsecure.

According to rpmfusion maintainers, they are working on signing their repositories and it should be done hopefully soon. So if you’re not in a hurry with your upgrade, just wait a while and the problem will disappear soon (hopefully).

But, if you insist that you want to upgrade now, what are your options?

Some people suggest you can add --nogpgcheck option to the command line. Please don’t do that! That completely bypasses any security checks, even for proper Fedora packages! It will get you vulnerable to security attacks.

A much better option is to temporarily remove rpmfusion repositories:

$ sudo dnf remove 'rpmfusion-*-release'

and run the upgrade command again. You’ll likely need to add --allowerasing option, because it will probably want to remove some packages that you installed from rpmfusion (like vlc):

$ sudo dnf system-upgrade download --releasever=24 --allowerasing

This is OK, after you upgrade your system, you can enable rpmfusion repositories again, and install the packages that were removed prior to upgrade.

(I recommend to really remove rpmfusion repositories and not just disable them, because they manage their repos in a non-standard way, enabling and disabling their updates and updates-testing repos during the system lifecycle according to their needs, so it’s hard to know which repos to enable after the system upgrade – they are not the same as were enabled before the system upgrade. What they are doing is really rather ugly and it’s much better to perform a clean installation of their repos.)

After the system upgrade finishes, simply visit their website, install the repos again, and install any packages that you’re missing. This way, your upgrade was performed in a safe way. The packages installed from rpmfusion might still be installed unsafely (depending whether they manage to sign the repo by that time or not), but it’s much better than to upgrade your whole system unsafely.

To close this up, I’m sorry that people are hit by these complications, but it’s not something Fedora project can directly influence (except for banning third-party repos during system upgrades completely, or some similar drastic measure). This is in hands of those third-party repos. Hopefully lots of this pain will go away once we start using Flatpak.

‘Package XXX is not signed’ error during upgrade to Fedora 24

KVM disk performance: raw vs qcow2 format

Some time ago I compared disk drivers performance in KVM. Today I compared different storage formats – raw and qcow2. Let’s have a look:

Test procedure: Create an empty 10 GB image, attach to VM using VirtIO driver, boot F20 Alpha Live x86_64, measure the time of installation. Repeat installation once again, this time reusing the existing image (instead of creating a new one). Do this for both formats.

Test results:

raw 1st pass          2:36
raw 2nd pass          2:38
qcow2 1st pass        2:36
qcow2 2nd pass        2:44

As you can see, the results are very much the same. It seems it doesn’t matter much which format you use.

But, qcow2 format has some nice additional features, like copy-on-write cloning. If I need to test something very quickly in my existing VM and then revert the changes back, this is the easiest way:

$ cd /var/lib/libvirt/images
$ mv f19.qcow2 f19.qcow2_orig
$ qemu-img create -f qcow2 -b f19.qcow2_orig f19.qcow2
Formatting 'f19.qcow2', fmt=qcow2 size=10737418240 backing_file='f19.qcow2_orig' encryption=off cluster_size=65536 lazy_refcounts=off
$ # Run the VM now and do your tasks
$ mv f19.qcow2_orig f19.qcow2


Flattr this

KVM disk performance: raw vs qcow2 format

Experiment with bleeding-edge GNOME using GnomeOSTree

I have just discovered GnomeOSTree (I’ve heard about it before, but never tried it). It allows you to run an absolutely fresh version of GNOME (checked out from git the very day) in a virtual machine. This is perfect for

  • experimenting with new features
  • checking whether a bug still exists in the development version
  • checking whether a bug fix is correct, without waiting for a distribution package update

I’ve just played with it for 10 minutes, so I might be missing a lot of things, but this seems to be a very useful tool for anyone testing and reporting GNOME bugs. It’s extremely easy to set up, you just download a VM disk image and import it into virt-manager. Later you can update it from inside the system. Try it!


Flattr this

Experiment with bleeding-edge GNOME using GnomeOSTree

The heroes of Fedora updates testing in Q2 2013

1360095720_PrizeAnother quarter is gone, let’s see how much testing of proposed updates in Fedora we have done. The purpose of this testing is to make sure that broken updates don’t enter our stable (or soon-to-be-stable) Fedora releases. The test feedback is managed inside Bodhi application and here are the statistics for the recent quarter:

Test period: Q2 2013 (2013-04-01 – 2013-06-30)
Testers: 669
Comments1: 3891

Name Updates commented
Adam Williamson (adamwill) 231
Reindl Harald (hreindl) 224
Igor Gnatenko (ignatenkobrain) 224
T.C. Hollingsworth (patches) 174
Kevin Fenzi (kevin) 169
Robert C. Lightfoot (boblfoot) 125
misc 124
Kalev Lember (kalev) 111
Alexander Kurtakov (akurtakov) 107
Piotr Drąg (raven) 106
nonamedotc 102
bitlord 99
Ankur Sinha (ankursinha) 72
Kamil Páral (kparal) 69
quickbooks 65
Daniel Dimitrov (dandim) 63
Matthias Runge (mrunge) 46
Michael Schwendt (mschwendt) 39
mharmsen 32
keramidas 30
Peter Robinson (pbrobinson) 26
Rex Dieter (rdieter) 26
bojan 26
Wolfgang Ulbrich (raveit65) 25
Dan Mashal (vicodan) 23
Jack Magne (jmagne) 18
Nick Bebout (nb) 18
gwei3 17
Orion Poplawski (orion) 16
bradw 16
Adam Domurad (adomurad) 16
Matthias Clasen (mclasen) 16
nucleo 16
Endi Sukma Dewata (edewata) 15
Michael Catanzaro (catanzaro) 14
Arthur Scott Poore (spoore) 13
Sérgio Monteiro Basto (sergiomb) 13
Joel Burleson-Davis (sysengbd) 12
Tao Wu (wutao85) 12
Tom Callaway (spot) 12
Adam Jackson (ajax) 12
Abhishek Koneru (kaskahn) 12
Jens Petersen (petersen) 11
Joachim Backes (backes) 11
Ed Greshko (egreshko) 11
Heiko Adams (heikoada) 11
Dan Horák (sharkcz) 11
Freddy Willemsen (freddyw) 11
Christian Lockley (clockley1) 10
Samuel Sieb (ssieb) 10
robatino 10
amessina 10
Mamoru Tasaka (mtasaka) 10
Fabian Deutsch (fabiand) 10
Vojtech Bocek (tassadar) 9
Hans Müller (cairo) 9
Fabio Valentini (fafatheone) 9
Pete Walter (pwalter) 8
mooninite 8
leigh123linux 8
Zbigniew Jędrzejewski-Szmek (zbyszek) 7
Oron Peled (oron) 7
Peter Borsa (asrob) 7
Christopher Meng (cicku) 7
Tim Flink (tflink) 7
rcritten 7
Omair Majid (omajid) 7
Dean Hunter (deanhunter) 7
jerboaa 7
Mike FABIAN (mfabian) 7
Cristian Ciupitu (ciupicri) 7
Dennis Gilmore (ausil) 6
Martin Krizek (mkrizek) 6
Mikolaj Izdebski (mizdebsk) 6
Artyom (artkun) 6
Bruno Wolff III (bruno) 6
Simone Caronni (slaanesh) 6
Pavel Tisnovsky (ptisnovs) 6
Patrick Uiterwijk (puiterwijk) 6
Remi Collet (remi) 6
John Reiser (jreiser) 6
David Juran (djuran) 6
D. Charles Pyle (dcharlespyle) 6
pnemade 6
Josef Stribny (jstribny) 5
nixfas 5
Morten Stevens (mstevens) 5
Lijun Li (lijli) 5
Andrew Azores (aazores) 5
Kevin Kofler (kkofler) 5
avij 5
Axilleas Pipinellis (axilleas) 5
Ville Skyttä (scop) 5
Luke Macken (lmacken) 5
Chris Murphy (chrismurphy) 5
Johan Hedin (jhn) 5
Chad Feller (cfeller) 5
Nils Philippsen (nphilipp) 5
Germano Massullo (germano) 5
Elad Alfassa (elad) 5
David A. Marlin (dmarlin) 5
Martin Kosek (mkosek) 5
Dominic Hopf (dmaphy) 5
Karsten Hopp (karsten) 5
Šimon Lukašík (isimluk) 5
Tim Waugh (twaugh) 5
Vít Ondruch (vondruch) 5
tomspur 5
Samuel Greenfeld (greenfeld) 5
…and also 560 other reporters who created less than 5 reports each, but 848 reports combined!

1 If a person provides multiple comments to a single update, it is considered as a single comment. Karma value is not taken into account.

When compared to Q1 2013, the number of reporters remained steady, but the number of reports increased almost twofold. That is amazing, I suspect it’s directly related to the Fedora 19 stabilization cycle and release.

The top-performers have almost an even score, which is great to see. They are Adam Williamson (adamwill), Reindl Harald (hreindl) and Igor Gnatenko (ignatenkobrain). Thank you guys for your outstanding and steadfast performance!

It’s also interesting to see that in the top 15 list there are only 4 Red Hatters (Adam, Kevin, Alexander and me), all the rest are community participants. I love that fact very much. Testing might not always be fully engaging, but still, paid employees are the minority here. What a strong community.

If you see yourself in the table, or if you are somewhere below the threshold, thank you. Your effort makes Fedora stable and reliable.

If you haven’t participated yet, maybe it’s time to give it a try? You can start today! Just read QA:Updates_Testing and ask any questions in #fedora-qa on IRC or test list. We would love to see even more people involved.

Thanks everyone.

When reading the statistics, please take it with a grain of salt. Not all numbers are directly comparable. This is not meant to be a comparison chart, but a well-meant “thank you” letter.
The statistics were generated by these scripts.

The heroes of Fedora updates testing in Q2 2013

The heroes of Fedora 19 Final testing

1360095720_PrizeFedora 19 the Schrödinger’s Cat is out of the box, and alive! I have again gathered some interesting statistics about QA contributions, this time during the Final phase (between Beta and Final release).

Fedora 19 Final – wiki matrices

This is the list of people who filled in our release validation wiki matrices (Install, Desktop and Base), which are posted for every test compose (and release candidate) that we create. The purpose is to see which areas have been thoroughly tested and which were not.

Test period: Fedora 19 Beta – Fedora 19 Final
Testers: 29
Reports: 1035
Unique referenced bugs: 58

Name Reports submitted Referenced bugs1
robatino 259 929177 946964 958426 958427 972025 972046 974050 974052 975227 975230 975324 975325 976280 976281 977669 977671 977715 977732 978114 978124 978125 979171 979172 (23)
nonamedotc 127 972225 977879 (2)
adamwill 103 858270 892178 975800 978008 (4)
kparal 103 973068 974032 974038 977715 977816 977962 (6)
Wutao85 55
lbrabec 49 855824 858270 977962 978298 (4)
jpospisi 48 964586 975483 (2)
eischmann 39
lnie 24
pkotvan 23 969684 975813 (2)
jskladan 21
tflink 21 975495 (1)
mkrizek 21 977962 (1)
jsedlak 19 973747 975375 978346 (3)
pschindl 18
lnovy 16 743281 858270 (2)
mmarhefk 16 892178 959796 975521 (3)
Martix 15 976417 (1)
boblfoot 12 971191 (1)
jreiser 12 976582 (1)
kevin 10
satellit 10
werkman 5 971109 971255 976034 (3)
todoleza 3 970988 (1)
konradr 2 977974 977987 978036 (3)
mattdm 1
jreznik 1 705086 (1)
mooninite 1
dgilmore 1

1 This is a list of bug reports linked to the wiki results. They don’t have to be reported by that concrete person.

Who would have thought… Andre Robatino (robatino) is at the top again. It’s also very nice to see how many bugs he referenced during his testing. The more referenced bugs the more publicity, the more likeliness that someone reproduces them, the more likeliness someone fixes them. Thanks, Andre!

The community participation list continues with nonamedotc, who provided an amazing number of test case results as well. Then there is a long list of Red Hatters, the highest score split between Adam Williamson (adamwill) and Kamil Páral (kparal) (a.k.a. me). Since I referenced more bugs, I consider myself a winner 😛 “I only believe in statistics that I doctored myself.” 🙂

The community participation continues with Bob Lightfoot (boblfoot), who had been greatly active during Beta, John Reiser (jreiser), Thomas Gilliard (satellit) and others. Thank you, everyone.

Fedora 19 Final – Bugzilla

This is a trimmed list of people who reported bugs into Bugzilla against Fedora 19 in the specified time period. When compared to Fedora 19 Beta, the numbers went up again, especially the number of reporters. That’s really great.

Test period: Fedora 19 Beta – Fedora 19 Final (2013-05-29 – 2013-06-28)
Reporters: 614
New reports: 1486

Name Reports submitted1 Excess reports2 Accepted blockers3
IBM Bug Proxy 31 2 (6%) 0
Mark Salter 31 2 (6%) 0
Adam Williamson 27 1 (3%) 2
quickbooks.office at gmail.com 27 1 (3%) 1
Mikhail 26 1 (3%) 0
David Woodhouse 23 0 (0%) 0
Kamil Páral 22 2 (9%) 4
Andre Robatino 21 18 (85%) 1
Zbigniew Jędrzejewski-Szmek 17 0 (0%) 0
Dan Mashal 16 4 (25%) 0
Karel Volný 16 0 (0%) 0
Vojtěch Boček 13 0 (0%) 2
Emmanuel Pacaud 13 0 (0%) 0
Marian 12 3 (25%) 0
Jan Sedlák 11 0 (0%) 0
Lukas Brabec 11 1 (9%) 0
Martin Holec 11 2 (18%) 0
Tim Waugh 11 0 (0%) 0
A.J. Werkman 10 0 (0%) 0
Florian Weimer 10 0 (0%) 0
Heiko Adams 10 0 (0%) 0
Joachim Backes 10 3 (30%) 0
Chris Murphy 9 0 (0%) 1
Bojan Smojver 9 0 (0%) 0
Diogo Campos 9 0 (0%) 0
Jonas Thiem 9 0 (0%) 0
David Jaša 8 1 (12%) 0
Dean Hunter 8 1 (12%) 0
Endi Sukma Dewata 8 2 (25%) 0
James 8 2 (25%) 0
Jeff Bastian 7 1 (14%) 1
Jiří Martínek 7 1 (14%) 0
Marcus Moeller 7 3 (42%) 0
Matthew Miller 7 0 (0%) 0
Simon Lewis 7 1 (14%) 0
TKS 7 1 (14%) 0
Mark Hamzy 6 0 (0%) 1
Nix\ 6 2 (33%) 1
arnav 6 2 (33%) 0
Kalev Lember 6 1 (16%) 0
Krzysztof Daniel 6 0 (0%) 0
Rolle 6 1 (16%) 0
Russ Anderson 6 1 (16%) 0
Siddhesh Poyarekar 6 0 (0%) 0
James Heather 5 0 (0%) 1
Jiri Eischmann 5 1 (20%) 1
Michael Scherer 5 0 (0%) 1
Alexey Derlaft 5 2 (40%) 0
Andrea Oliveri 5 1 (20%) 0
Björn Esser 5 0 (0%) 0
D. Charles Pyle 5 0 (0%) 0
Dennis Gilmore 5 0 (0%) 0
Edgar Hoch 5 0 (0%) 0
Flóki Pálsson 5 1 (20%) 0
Francisco de la Peña 5 1 (20%) 0
Joshua Holm 5 0 (0%) 0
Konrad Rzeszutek Wilk 5 0 (0%) 0
Lukas -krtek.net- Novy 5 2 (40%) 0
mark 5 0 (0%) 0
Matthias Clasen 5 0 (0%) 0
nonamedotc at gmail.com 5 2 (40%) 0
Parag 5 2 (40%) 0
Paul Whalen 5 0 (0%) 0
Reartes Guillermo 5 0 (0%) 0
Vít Ondruch 5 0 (0%) 0
Álvaro Castillo 5 0 (0%) 0
Štefan Gurský 5 0 (0%) 0
…and also 547 other reporters who created less than 5 reports each, but 834 reports combined!

1 The total number of new reports (including “excess reports”). Reopened reports or reports with a changed version are not included, because it was not technically easy to retrieve those. This is one of the reasons why you shouldn’t take the numbers too seriously, but just as interesting and fun data.
2 Excess reports are those that were closed as NOTABUG, WONTFIX, WORKSFORME, CANTFIX or INSUFFICIENT_DATA. Excess reports are not necessarily a bad thing, but they make for interesting statistics. Close manual inspection is required to separate valuable excess reports from those which are less valuable.
3 This only includes reports that were created by that particular user and accepted as blockers afterwards. The user might have proposed other people’s reports as blockers, but this is not reflected in this number.

The top reporters are nicely and evenly distributed, there are no extreme performers as in the wiki statistics. The top position share IBM Bug Proxy and Mark Salter. Mark reported a number of build failures for ARM architecture. I assume IBM Bug Proxy is probably a shared account among IBM employees who care about Fedora development – they reported a number of anaconda, kernel and boot issues. It’s always nice to see other companies involved in raising the Fedora quality – welcome, IBM 🙂

When it comes to general bug reporting, the top Red Hatter is Adam Williamson. As for the community, quickbooks.office reported the same number of bugs, most of them being SELinux error duplicates. Mikhail closely follows with a much more interesting mix of bug reports, which also applies for David Woodhouse. The community list follows with Andre Robatino (mostly fake bug reports for test purposes), Zbigniew Jędrzejewski-Szmek, Dan Mashal and others.

It’s great to see so many bug reports and so even distribution of the top reporters. Thanks, all of you.

Fedora 19 Beta – updates testing

This is a trimmed list of people who provided feedback in Bodhi for all updates proposed into Fedora 19 in the specified time period. The purpose is to make sure that broken updates do not enter the stable repository and the release continuously stabilizes. The number of testers increased considerably again as compared to Beta.

Test period: Fedora 19 Beta – Fedora 19 Final (2013-05-29 – 2013-06-28)
Testers: 240
Comments1: 1107

Name Updates commented
Igor Gnatenko (ignatenkobrain) 155
Adam Williamson (adamwill) 76
nonamedotc 71
Kalev Lember (kalev) 57
Alexander Kurtakov (akurtakov) 47
bitlord 45
T.C. Hollingsworth (patches) 41
quickbooks 35
Piotr Drąg (raven) 29
Kevin Fenzi (kevin) 28
Ankur Sinha (ankursinha) 20
keramidas 20
misc 20
Michael Schwendt (mschwendt) 16
Peter Robinson (pbrobinson) 15
Wolfgang Ulbrich (raveit65) 13
Dan Mashal (vicodan) 13
Matthias Clasen (mclasen) 12
Matthias Runge (mrunge) 11
bojan 10
Kamil Páral (kparal) 9
leigh123linux 8
D. Charles Pyle (dcharlespyle) 6
Pete Walter (pwalter) 6
Joachim Backes (backes) 6
mharmsen 6
Rex Dieter (rdieter) 5
Dennis Gilmore (ausil) 5
Ed Greshko (egreshko) 5
Dominic Hopf (dmaphy) 5
Endi Sukma Dewata (edewata) 5
Abhishek Koneru (kaskahn) 5
Kai Engert (kengert) 4
Jiri Eischmann (eischmann) 4
pnemade 4
Martin Kho (mkho) 4
Vojtech Bocek (tassadar) 4
Reindl Harald (hreindl) 4
Heiko Adams (heikoada) 4
Ian Malone (imalone) 3
Lukas Brabec (lbrabec) 3
Anton Arapov (aarapov) 3
Dan Horák (sharkcz) 3
Ray Strode (rstrode) 3
Hans Müller (cairo) 3
Greg Schlau (gregschlau) 3
Sérgio Monteiro Basto (sergiomb) 3
Nathan Kinder (nkinder) 3
gil 3
Martin Kosek (mkosek) 3
Artyom (artkun) 3
Roland Grunberg (rgrunber) 3
…and also 188 other reporters who created less than 3 reports each, but 235 reports combined!

1 If a person provides multiple comments to a single update, it is considered as a single comment. Karma value is not taken into account.

A new star has emerged, Igor Gnatenko (ignatenkobrain) completely dominated here. Great job, Igor! The top performing Red Hatter was, yet again, Adam Williamson (adamwill).

The top community list, apart from Igor, also contains nonamedotc, Kalev Lember (kalev), bitlord, T.C. Hollingsworth (patches), quickbooks, Piotr Drąg (raven), and others. It’s great to see so many people involved, because testing proposed updates and providing karma is one of the easiest way to contribute, yet extremely important at the same time. Thanks to anyone who contributed!


For most parts, the number of reports stays similar to Beta, but the number of people involved rose considerably. Beta release seems to be very attractive to the Fedora community and rightly so, it’s the last step before everything is pronounced stable. We’re glad to see so many people involved in release testing and we hope that we will see you again in Fedora 20 cycle!

In the meantime you can help as well – for example testing the proposed updates for stable Fedora releases is as useful, maybe even more important, than for releases in development. You can also play with Rawhide and help us find the bugs well in advance. There is a lot of possibilities – if it sounds interesting to you, be sure to have a look at QA/Join, follow the announcements and talk to us in #fedora-qa on IRC and test list. Your involvement will help not only Fedora and its users, but the whole Linux ecosystem. Thanks!

When reading the statistics, please take it with a grain of salt. The numbers are not directly comparable. People might see some reports as more valuable than others. Some people tested a lot of components, but haven’t found many problems (but that also helps). Some people used their skills in other areas than those mentioned. This is not meant to be a comparison chart, but a well-meant “thank you” letter.
The statistics were generated by these scripts.

The heroes of Fedora 19 Final testing