Kparal's Fedora Blog

This is a short trip report from FUDCon Milan 2011. Josef already described in long detail the joys and challenges of Milan city (and Italy in general), so I’ll stick with just the conference itself.

In the beginning we received a pretty T-shirt. I love receiving T-shirts. Then we explored the university. Our place consisted of two auditoriums and several smaller rooms. I have no idea how many people attended, I guess over a hundred. As we began to fill up the main auditorium, I quickly noticed the insufficient resource – electricity. There was only one usable socket at the end of the room. With a dozen people connected we quickly blew the fuse. Soon I started to envy people with netbooks and their 8 hours on-battery time.

After Jared welcomed us there was a barcamp. People proposed talks and we then voted them. The number of proposed topics and the available time and rooms matched quite well, so almost all talks were given the opportunity.

The first two talks I have attended were about RPM future & alternative packaging and about how Fedora is made from Rel-Eng perspective. The RPM talk was surprisingly relevant to us (us == Fedora QA team), because Jindřich Nový spoke about libsolv (IIRC the name), a library for dependency solving that should be incorporated directly into RPM or serve as a middle-men between RPM and YUM. All Fedora tools could then use this library for depsolving tasks. The API and capabilities should be better then what currently YUM can offer. (We just need to wait some time until everything is implemented, of course). That was extremely interesting, because we fight with depsolving issues in our depcheck test, as part of AutoQA test suite, and it’s a nightmare. This could help us substantially.

The second talk from Dennis Gilmore gave me a little perspective how RelEng team works when they create a new Fedora release. It is quite complex and they touch basically our parts of Fedora infrastructure. But I have now a little better idea and I’m glad for it.

And then… there was the AutoQA talk. Our talk. I talked about the recent changes in Fedora QA and about the near future. Josef Skládanka talked about his baby, ResultsDB. You can see the slides. There were some comments afterwards. There was a request whether we could execute upgradepath sooner, before you actually ask for push to stable in Bodhi. I believe we can’t do that (at least right now), but I’ll try to think about it more. Also we can’t execute your tests right now unfortunately, but it’s our goal in the future. And Josef promised to create KVM images (for server and client machines) for people interested in hacking AutoQA, because the installation is allegedly too difficult. Well, it is difficult. But you’ll have to learn how to use the Autotest+AutoQA framework even with the provided KVM images, so both approaches seem equally difficult for me. Simplification of server+clients deployment is also our near future goal, stay tuned.

I have attended a few more talks and Saturday was over. There was a FUDPub and hackfest day on Sunday. I did not participate directly in any hackfest, but I was present for Git tips&tricks and Java SIG sessions. I used the time to catch up with all the emails that accumulated during the trip.

Overall it was quite short FUDCon. But it kind of matched the experience of the last year’s one in Zurich. I met some people, I learned something new. But I am still curious how the american FUDCons look like. Are they different? I hope I will see some of them too in the future.

Don’t you guys just hate using Bugzilla’s web interface for attaching files? Especially when you have like >= 5 of them? I do. It’s slow, it requires like 20 mouse clicks and whatnot.

I recently learned you can use “bugzilla attach” command from python-bugzilla >= 0.6.2 (I had to install an .fc17 package), but it’s still not as easy as I would like to.

So I created a wrapper on top of that, “bzattach” command. Usage [1]:

$ bugzilla login       #only for the first time
$ bzattach 12345 *.log

Yay!

If you like it, get it while it’s hot!
https://github.com/kparal/fedoraqa

[1] “–help” is also supported, be sure to check that out

If you like this Git tip series, I have something for you. If you ever feel hunger for more Git tips, these are my recommendations:

• Pro Git book — Read this book if you want to work seriously with Git. It’s good.
• Git Ready website — There are so many Git tips that you will feel like in Git heaven. Or Git hell.

Today’s tip will be a little different. I found an interesting information about recovering lost commits in the Pro Git book. And because this book is under CC BY-NC-SA licence, I’ll copy the whole section (slightly modified) here. No sense in re-writing it. Of course the license still applies for that text below.

At some point in your Git journey, you may accidentally lose a commit. Generally, this happens because you force-delete a branch that had work on it, and it turns out you wanted the branch after all; or you hard-reset a branch, thus abandoning commits that you wanted something from. Assuming this happens, how can you get your commits back?

Here’s an example that hard-resets the master branch in your test repository to an older commit and then recovers the lost commits. First, let’s review where your repository is at this point:

$ git log --pretty=oneline
ab1afef80fac8e34258ff41fc1b867c702daa24b modified repo a bit
484a59275031909e19aadb7c92262719cfcdf19a added repo.rb
1a410efbd13591db07496601ebc7a059dd55cfe9 third commit
cac0cab538b970a37ea1e769cbbde608743bc96d second commit
fdf4fc3344e67ab068f836878b6c4951e3b15f3d first commit

Now, move the master branch back to the middle commit:

$ git reset --hard 1a410efbd13591db07496601ebc7a059dd55cfe9
HEAD is now at 1a410ef third commit
$ git log --pretty=oneline
1a410efbd13591db07496601ebc7a059dd55cfe9 third commit
cac0cab538b970a37ea1e769cbbde608743bc96d second commit
fdf4fc3344e67ab068f836878b6c4951e3b15f3d first commit

You’ve effectively lost the top two commits — you have no branch from which those commits are reachable. You need to find the latest commit SHA and then add a branch that points to it. The trick is finding that latest commit SHA — it’s not like you’ve memorized it, right?

Often, the quickest way is to use a tool called git reflog. As you’re working, Git silently records what your HEAD is every time you change it. Each time you commit or change branches, the reflog is updated. You can see where you’ve been at any time by running git reflog:

$ git reflog
1a410ef HEAD@{0}: 1a410efbd13591db07496601ebc7a059dd55cfe9: updating HEAD
ab1afef HEAD@{1}: ab1afef80fac8e34258ff41fc1b867c702daa24b: updating HEAD

Here we can see the two commits that we have had checked out, however there is not much information here. To see the same information in a much more useful way, we can run git log -g, which will give you a normal log output for your reflog.

$ git log -g
commit 1a410efbd13591db07496601ebc7a059dd55cfe9
Reflog: HEAD@{0} (Scott Chacon <schacon@gmail.com>)
Reflog message: updating HEAD
Author: Scott Chacon <schacon@gmail.com>
Date:   Fri May 22 18:22:37 2009 -0700

    third commit

commit ab1afef80fac8e34258ff41fc1b867c702daa24b
Reflog: HEAD@{1} (Scott Chacon <schacon@gmail.com>)
Reflog message: updating HEAD
Author: Scott Chacon <schacon@gmail.com>
Date:   Fri May 22 18:15:24 2009 -0700

     modified repo a bit

It looks like the bottom commit is the one you lost, so you can recover it by creating a new branch at that commit. For example, you can start a branch named recover-branch at that commit (ab1afef):

$ git branch recover-branch ab1afef
$ git log --pretty=oneline recover-branch
ab1afef80fac8e34258ff41fc1b867c702daa24b modified repo a bit
484a59275031909e19aadb7c92262719cfcdf19a added repo.rb
1a410efbd13591db07496601ebc7a059dd55cfe9 third commit
cac0cab538b970a37ea1e769cbbde608743bc96d second commit
fdf4fc3344e67ab068f836878b6c4951e3b15f3d first commit

Cool — now you have a branch named recover-branch that is where your master branch used to be, making the first two commits reachable again. Next, suppose your loss was for some reason not in the reflog — you can simulate that by removing recover-branch and deleting the reflog. Now the first two commits aren’t reachable by anything:

$ git branch -D recover-branch
$ rm -Rf .git/logs/

Because the reflog data is kept in the .git/logs/ directory, you effectively have no reflog. How can you recover that commit at this point? One way is to use the git fsck utility, which checks your database for integrity. If you run it with the --full option, it shows you all objects that aren’t pointed to by another object:

$ git fsck --full
dangling blob d670460b4b4aece5915caf5c68d12f560a9fe3e4
dangling commit ab1afef80fac8e34258ff41fc1b867c702daa24b
dangling tree aea790b9a58f6cf6f2804eeac9f0abbe9631e4c9
dangling blob 7108f7ecb345ee9d0084193f147cdad4d2998293

In this case, you can see your missing commit after the dangling commit. You can recover it the same way, by adding a branch that points to that SHA.

Enjoy!

Rebasing means re-ordering the commits in history. It always changes history. It means rebasing is good for your local branches, but not for branches that many people work on. Because in that case changed history means a lot of annoyed programmers.

Rebasing feature branch to master

The most common use-case is probably with feature branches. You split your feature branch from master. You develop a feature, appending a few commits. Now, you would like to send a patch to the project. But in the meantime the master branch moved. Project developers politely ask you to send the patch based on the current master. What to do? Rebase against master.

$ git checkout feature
$ git rebase master

This will find the common ancestor of feature and master (the split point). Then it will apply all changes from master. And then it will apply all changes you have done on feature since the split point. See the images.

Your code will then seem as if you based it on the current master. You can then do a simple patch git diff master feature and send it.

Rebasing a single branch

Rebasing can also be used on a single branch. It can re-order the commits in history, delete some commits, squash some commits, and more.

Let’s say I have realized I would like to have my last three commits in the feature branch in a different order:

$ git rebase --interactive HEAD~3
pick bb6f59e Update minimum python version requirement to 2.6 in the spec file.
pick 4e929b8 Improve docs in autoqa package.
pick 7796850 changed NEWS

# Rebase d03c059..7796850 onto d03c059
#
# Commands:
#  p, pick = use commit
#  r, reword = use commit, but edit the commit message
#  e, edit = use commit, but stop for amending
#  s, squash = use commit, but meld into previous commit
#  f, fixup = like "squash", but discard this commit's log message
#  x, exec = run command (the rest of the line) using shell
#
# If you remove a line here THAT COMMIT WILL BE LOST.
# However, if you remove everything, the rebase will be aborted.
#

HEAD~3 denotes the fourth most recent commit. That is the base, the last commit that stays intact. It opens an editor with the last three commits in their commit order, most recent being last. You can now easily change commit order by re-ordering the lines. You can also delete commit by deleting the line.

If you change the keyword in the first column, you can edit commit, reword it, squash several commits into one, and many more. Play with it, it’s fun. If you choose to edit some commit, the rebasing process will stop at that point and allow you to do the changes. After you’re done, just execute git rebase --continue to continue with the process. If git find some conflict it can’t solve, it will again stop, you resolve the conflict and use the same command for continuing. You can abort the whole process anytime, if you wish, by issuing git rebase --abort.

Of course, always back-up your code by keeping a back-up branch before performing a difficult rebase process.

Enjoy!

Very often you realize you want to change something right after you’ve committed it. At least I do. How to change the last commit?

It is extremely easy to do. Let’s say you want to change just the commit message. All you need is to run:

$ git commit --amend

That’s all. An editor will open up for you with the last commit message and you can edit it. Piece of cake.

Now something more advanced! You realized you forgot to add some file/remove some file/change some file. You need to change the file contents of the last commit. How to do that?

It’s easy. First, do all the changes you need:

$ vim file_to_be_changed
$ touch file_to_be_added

Now put the desired changes into the staging area:

$ git add file_to_be_changed
$ git add file_to_be_added
$ git rm file_to_be_removed

And now run the same magic command as before:

$ git commit --amend

That’s all. Not only git will allow you to change the commit message, but it will also add the changes in the staging area to the last commit. Work done. Ask for pay raise.

Important warning at the end: Changing last commit (even just the commit message) changes SHA1 checksum of the commit. Never change commits that you already pushed to some remote location if there is a chance somebody else already based his/her work on it. You would cause such person a lot of difficulties. Change commits for your local development only.

Enjoy!

Sometimes you change the code and then you realize you would like to commit it as several separate patches. For example you fixed two bugs at once but then you decided you want to have the commits separated into two logical units – first bug fix and second bug fix.

As long as the changes don’t involve the same file, it’s easy to do. You just git add first file, commit, git add second file, commit. But what will you do if the two changes involved the same file?

There is a simple solution: git add --patch

Let’s see an example. I have made two changes in my README file – deleted one line and added one line. I want to stage it as two separate patches.

$ git add --patch README
diff --git a/README b/README
index 05ad773..2768fc1 100644
--- a/README
+++ b/README
@@ -1,4 +1,3 @@
-Welcome to autoqa!

 Everything here is under heavy development so don't expect anything to
 stay stable - modules, APIs, whatever.  If it's expected to stay stable we
Stage this hunk [y,n,q,a,d,/,j,J,g,e,?]? y
@@ -8,6 +7,7 @@ but if you have questions please feel free to contact us.
 == Architecture ==

 There's four main things in this tree: 'autoqa', 'events', 'watchers', and 'tests'.
+foo!

 'autoqa' is the main binary that kicks off tests (using the autotest framework -
 see http://autotest.kernel.org/ for details). It's typically supposed to be
Stage this hunk [y,n,q,a,d,/,K,g,e,?]? n

Git detects “hunks” (changed sections) in the file and asks for every hunk whether you want to stage it or not (see the prompt “Stage this hunk [y,n,q,a,d,/,j,J,g,e,?]?“) . You can see that I staged the first hunk, and I didn’t staged the second one. (For explanation of the all the letters in the prompts type in the question mark.).

Now I see something like this:

$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#	modified:   README
#
# Changes not staged for commit:
#   (use "git add <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directory)
#
#	modified:   README
#

Some parts of README were staged, some parts were not. I can display the difference between my HEAD and the staging area:

$ git diff --staged
diff --git a/README b/README
index 05ad773..bbc5249 100644
--- a/README
+++ b/README
@@ -1,4 +1,3 @@
-Welcome to autoqa!

 Everything here is under heavy development so don't expect anything to
 stay stable - modules, APIs, whatever.  If it's expected to stay stable we

You can see that only one changed (the deleted line) is part of my staging area. Now I can commit and only this change will be part of it. Then I’ll stage the remaining changes in README and commit again. I’ve just easily split two changes in one file into two different commits. Voila!

Interactive mode

You can do the same way through interactive mode if you prefer:

$ git add --interactive
           staged     unstaged path
  1:    unchanged        +1/-1 README

*** Commands ***
  1: status	  2: update	  3: revert	  4: add untracked
  5: patch	  6: diff	  7: quit	  8: help
What now>

You are presented with the list of changed files and you can do different operations on them. Don’t forget to press h or 8 in this step. Action patch or 5 is exactly the same we saw with git add --patch. After you select which file to work on (and press Enter once more), you select all the desired hunks:

What now> p
           staged     unstaged path
  1:    unchanged        +1/-1 README
Patch update>> 1
           staged     unstaged path
* 1:    unchanged        +1/-1 README
Patch update>>
diff --git a/README b/README
index 05ad773..2768fc1 100644
--- a/README
+++ b/README
@@ -1,4 +1,3 @@
-Welcome to autoqa!

 Everything here is under heavy development so don't expect anything to
 stay stable - modules, APIs, whatever.  If it's expected to stay stable we
Stage this hunk [y,n,q,a,d,/,j,J,g,e,?]? y
@@ -8,6 +7,7 @@ but if you have questions please feel free to contact us.
 == Architecture ==

 There's four main things in this tree: 'autoqa', 'events', 'watchers', and 'tests'.
+foo!

 'autoqa' is the main binary that kicks off tests (using the autotest framework -
 see http://autotest.kernel.org/ for details). It's typically supposed to be
Stage this hunk [y,n,q,a,d,/,K,g,e,?]? n

*** Commands ***
  1: status	  2: update	  3: revert	  4: add untracked
  5: patch	  6: diff	  7: quit	  8: help
What now>

Now I can display the status:

What now> s
           staged     unstaged path
  1:        +0/-1        +1/-0 README

*** Commands ***
  1: status	  2: update	  3: revert	  4: add untracked
  5: patch	  6: diff	  7: quit	  8: help
What now>

In the README file I have 0 lines added and 1 line removed in the staging area, and one line added and zero lines removed in the unstaged area.

I can easily display the diff between HEAD and the staging area:

What now> d
           staged     unstaged path
  1:        +0/-1        +1/-0 README
Review diff>> 1
diff --git a/README b/README
index 05ad773..bbc5249 100644
--- a/README
+++ b/README
@@ -1,4 +1,3 @@
-Welcome to autoqa!

 Everything here is under heavy development so don't expect anything to
 stay stable - modules, APIs, whatever.  If it's expected to stay stable we
*** Commands ***
  1: status	  2: update	  3: revert	  4: add untracked
  5: patch	  6: diff	  7: quit	  8: help
What now>

That’s exactly what I want. Now I can just quit the interactive mode and commit.

The first approach (non-interactive) seems faster to me, but interactive mode can be also appealing for some people.

Enjoy!