Project

General

Profile

Gerrit » History » Revision 71

Revision 70 (laforge, 08/19/2017 10:16 AM) → Revision 71/130 (neels, 08/24/2017 06:03 PM)

h1. Contributing using Gerrit 

 {{>toc}} 

 At [[OpenBSC:OsmoDevCon2016]] we discussed problems with our past contribution / patch submission process using mails on the mailing list as well as patchwork.    The result is that we want to give Gerrit a try for some time and see if it helps us to have a better process 

 Gerrit is a review tool that integrates nicely with git and ssh. You can find general information about Gerrit at https://www.gerritcodereview.com/ 

 The advantages of Gerrit are: 
 * patch submission status is automatically tracked, also with several revisions for a patch set. 
 * patches are build-tested (and possibly even further tested) by jenkins before they are applied 
 * developers + maintainers can formally vote on a patch (developer: -1/0/+1, maintainer: -2/0/+2) 
 * once a patch has +2 score, it can be (automatically) merged into master 
 * patch sumissions not via git send-email but direcly from git 

 h2. Osmocom Subprojects using Gerrit 

 The following projects use Gerrit to contribute changes: 

 * libosmocore.git 
 * libosmo-abis.git 
 * libosmo-netif.git 
 * libosmo-sccp.git 
 * libsmpp34.git 
 * openbsc.git 
 * osmo-bts.git 
 * osmo-iuh.git 
 * osmo-pcu.git 
 * cellmgr-ng.git 
 * osmo-sip-connector.git 

 h2. Configuring Gerrit/Account 

 You will need to sign-up at https://gerrit.osmocom.org/login/. If you have an Osmocom Redmine account you can use https://osmocom.org/openid as OpenID provider. 

 * first sign in on https://osmocom.org. Do this before logging in on gerrit (the redmine login process loses the gerrit login data and you'd have to do the same thing twice if not logged in on osmocom.org already). 
 * go to https://gerrit.osmocom.org and click the "Sign in" link. 
 * click the "Sign in with Osmocom":https://gerrit.osmocom.org/login/%23%2Fq%2Fstatus%3Aopen?id=https://osmocom.org/openid link (can be bookmarked). -- This is the same as entering https://osmocom.org/openid as OpenID provider and hitting the "Sign in" button. 

 *careful:* enter 'https' to ensure that your openid credentials are passed on encryptedly. 
 *pitfall:* if you're logged in on 'projects.osmocom.org' (including the 'projects.' part), you should also use the openid provider: https://projects.osmocom.org/openid; the 'projects.' part may be omitted, what's important is that redmine login and OpenID URLs match. Also, decide for one of those URLs once, because when picking a different OpenID URL next time, you will create a new user instead of logging in as yourself. 
 *note:* gerrit will create a distinct user for each openid URL you pass. If you logged in successfully but your user seems to have lost permissions, you may have created an evil twin user: contact us on the mailing list so we can fix it in the user database. 

 If you have no Osmocom redmine account, you can simply create one online at the "Register" link in the upper right corner. 
 Even without an existing or new redmine account, you should also be able to use any other OpenID provider to authenticate against gerrit (untested). 

 After the initial sign-up you will need to: 

 * Pick a username (can not be changed) 
 * Add your public ssh key(s) 
 * Add email addresses you intend to use as author/comitter 

 If you would like to push private branches to the Gerrit repository, you also need to be added to the "known users" group. 
 Please send a short requesting email to openbsc@lists.osmocom.org. 

 h2. Setting up Gerrit for commits and pushing 

 *Note:* it is easiest to work with gerrit when gerrit is the only remote in your git clone. 
 When you clone from git.osmocom.org and add the gerrit remote, git will have two remotes, 
 so when you first checkout a branch you have to supply the remote explicitly (cumbersome). 
 The gerrit repositories and git.osmocom.org are constantly synced, so it is sufficient 
 to clone from gerrit only. 

 h3. Simplest: new clone 

 * Create a new clone from gerrit 
 * Fetch the commit hook that adds Change-Id to each commit to uniquely identify a commit 

 <pre> 
 git clone ssh://$USERNAME@gerrit.osmocom.org:29418/$PROJECT.git 
 scp -P 29418 $USERNAME@gerrit.osmocom.org:hooks/commit-msg $PROJECT/.git/hooks/ 
 </pre> 

 h3. SSH config 

 In '~/.ssh/config', add these lines: 
 <pre> 
 Host go 
 Hostname gerrit.osmocom.org 
 Port 29418 
 User $USERNAME 
 </pre> 
 ('go' means gerrit.osmocom, replace with your favorite shortcut name, 
 replace '$USERNAME' with your user name as used on the gerrit website) 

 Then you can shorten above commands to 
 <pre> 
 git clone ssh://go/$PROJECT.git 
 cd $PROJECT 
 scp go:hooks/commit-msg .git/hooks/ 
 </pre> 

 h3. Commit hook: Always put Change-Id at the bottom of the log message 

 The commit-msg hook places a Change-Id tag in the footer, often above other tags like 'Depends:' or 'Related:'. Since the Change-Id is an implementation detail for Gerrit, I personally prefer it always placed right at the bottom. This simple edit changes the commit-msg hook to add Change-Id at the bottom unconditionally: 

 <pre> 
 cd $PROJECT 
 sed -i 's/if (unprinted /if (0 \&\& unprinted /' .git/hooks/commit-msg 
 </pre> 

 The goal is to disable the condition in line 163 with an 'if (0...': 

 <pre> 
                         if (0 && unprinted && match(tolower(footer[line]), changeIdAfter) != 1) { 
                                 unprinted = 0 
                                 print "Change-Id: I'"$id"'" 
                         } 
 </pre> 

 Then the Change-Id will be placed by line 170 instead. 

 h3. Committer must match 

 Your email address on gerrit and the email address git places in your 
 commits must match, or you will get rejected with an error message like 
 "invalid commiter". You can add email addresses on the gerrit web UI. 

 h3. Add gerrit to an existing clone 

 * Add the remote to be able to fetch and push to gerrit 
 * Fetch the commit hook that adds Change-Id to each commit to uniquely identify a commit 

 <pre> 
 USERNAME=gerrit_user_name 
 PROJECT=$(basename $PWD) 
 git remote add gerrit ssh://$USERNAME@gerrit.osmocom.org:29418/$PROJECT.git 
 scp -P 29418 $USERNAME@gerrit.osmocom.org:hooks/commit-msg .git/hooks/ 
 </pre> 

 h2. Push for review 

 Checkout the revision or branch that you want to submit for review, then 

 <pre> 
 git push gerrit HEAD:refs/for/master 
 </pre> 

 You can optionally add a topic name with 

 <pre> 
 git push gerrit HEAD:refs/for/master/my_topic 
 </pre> 

 h2. Merge patch to master 

 A patch can be merged when it has CR+2 and V+1 votes, and if, in case of a 
 series of patches pushed from a branch, when its ancestor patches can also be 
 merged. 

 Sometimes the reviewer that gives CR+2 also hits the "Submit" button right away  
 to merge the patch to master. Sometimes it is left up to the owner of the patch 
 to decide when to hit "Submit" (who needs to be in the "Known Users" group). 

 The V+1 vote means "build is verified" and is usually given by our jenkins 
 gerrit builds: https://jenkins.osmocom.org/jenkins/view/Jenkins-Gerrit/ 

 The CR+2 vote means "code reviewed and ready for merge to master branch". 
 Accounts with the "Reviewer" role for a given project are allowed to give CR+2 
 votes. Others are allowed to give CR+1 (and CR-1). CR votes _don't_ add up. 

 _Fixed by gerrit 2.12.6, see https://bugs.chromium.org/p/gerrit/issues/detail?id=4158:_ 
 -Sometimes hitting the "Submit" button results in an error message saying 
 "Change is New", which is a bug related to a private branch with the same 
 patches being present. Can be fixed e.g. by an admin's manual push to master.- 

 h2. Push a "private" user branch 

 *Note* that you must be a member of the "known users" group, see above. 

 If your local branch name is of the form 'your_name/topic', you can just 
 <pre> 
 git push 
 </pre> 
 and git will tell you what to do. 

 To push from a "nonstandard" local branch name, do 
 <pre> 
 git push gerrit HEAD:refs/heads/user/$USERNAME/branch_name 
 </pre> 


 h2. List changesets in gerrit 

 <pre> 
 git ls-remote gerrit changes/* 
 </pre> 

 h1. Tips and Tricks 

 h2. Throw-away branch 

 If you need to adjust and re-submit patches, it may be handy to create a throw-away branch ("R D" in magit-gerrit in emacs for example), 
 make your changes/amendments and then send patch(es) back to gerrit while removing temporary branch automatically with "git review -f". 

 h2. Fetch a patch from gerrit 

 This script (I called it @P@) makes fetching a patch set from gerrit a breeze: 
 <pre> 
 #!/bin/sh 
 # fetch gerrit patch into new branch named like the patch number. 
 # 
 # Usage: go to a git clone and pass a patch number: 
 # 
 #     cd openbsc 
 #     P 973 
 # or 
 #     P 973/2 
 # 
 # Will create new local branches '973_4' (if 4 is the latest patch set) 
 # or '973_2', respectively. 

 patch="$1" 

 if [ -z "$patch" ]; then 
   echo "Usage: P 1234[/5]" 
   exit 1 
 fi 

 if [ -z "$(echo "$patch" | grep '/')" ]; then 
   patch="/$patch/" 
 fi 

 if [ -z "$(echo "$patch" | grep '^/')" ]; then 
   patch="/$patch" 
 fi 

 last_set="$(git ls-remote origin "changes/*" | grep "$patch" | sed 's#.*/\([^/]*\)$#\1 &#' | sort -n | tail -n 1)" 
 if [ -z "$last_set" ]; then 
   echo "Not found: $patch" 
   exit 1 
 fi 

 change_name="$(echo "$last_set" | sed 's/.*\(refs.*\)/\1/')" 
 branch_name="$(echo "$change_name" | sed 's#refs/changes/../\([0-9]*\)/\([0-9]*\)#\1_\2#')" 

 set -x 
 git fetch origin "$change_name" 
 git co -b "$branch_name" FETCH_HEAD 
 </pre> 

 h2. Re-submit a Branch with Amended Commits 

 On a feature branch, one typically has numerous commits that depend on their preceding commits. 
 Often, some of the branch commits need to be amended for fixes. You can re-submit changes to 
 patches on your branch by pushing in the same way that you first submitted the branch. 

 Note: if you modify the Change-Ids in the commit logs, your push would open entirely new 
 review entries and you would have to abandon your previous submission. Comments on the first 
 submission are "lost" and you cannot diff between patch sets. 

 (There used to be a bug in gerrit that required editing the first patch to be able to 
 re-submit a branch, but that's fixed.) 



 h2. Re-submit Previously Abandoned Changes 

 You have to edit the Change-Ids, on a branch that would be every single commit log message. 

 <pre> 
 cd openbsc 
 git co my-branch 
 git rebase -i master 
 # replace all 'pick' with 'r' (or 'reword'), exit your editor 
 # git presents each commit log message for editing 
 </pre> 

 h2. Submit a "private" branch for master 

 If you've pushed a branch to refs/heads/* somewhere, gerrit will already know the Change-Ids on it. 
 Make sure the option [[Gerrit#Private-Branches-Create-a-new-change-for-every-commit|Create a new change for every commit not in the target branch]] is _TRUE_ for your project, 
 or gerrit will refuse to accept your submission. 

 h2. 502 Bad Gateway 

 When getting a "Bad Gateway" error message upon trying to login on gerrit, you probably just need to restart your web browser. The reason is not clear. 

 h1. Reasons for Particular Configuration 

 h2. Rebase if necessary 

 There are different merge strategies that Gerrit performs to accept patches. 
 Each project can be configured to a specific merge strategy, but unfortunately you can't 
 decide on a strategy per patch submission. 

 It seems that the "Merge if Necessary" strategy is best supported, but it creates non-linear 
 history with numerous merge commits that are usually not at all necessary. 

 Instead, the "Cherry Pick" strategy puts each patch onto current master's HEAD to create 
 linear history. However, this will cause merge failures as soon as one patch depends on 
 another submitted patch, as typical for a feature branch submission. 

 So we prefer the "Rebase if Necessary" strategy, which always tries to apply your patches to 
 the current master HEAD, in sequence with the previous patches on the same branch. 
 However, some problems still remain, including some bugs in "Rebase if Necessary". 

 There's a problem with "Rebase if Necessary": If your branch sits at master's HEAD, Gerrit 
 refuses to accept the submission, because it thinks that no new changes are submitted. 
 This is a bug in Gerrit, which holger has fixed manually in our Gerrit installation: 

 https://bugs.chromium.org/p/gerrit/issues/detail?id=4158 


 h2. Private Branches: Create a new change for every commit... 

 Say you have an extensive feature in development, and you want to keep it on the 
 upstream git repository to a) keep it safe and b) collaborate with other devs on it. 
 So, of course, you have regularly pushed to refs/heads/yoyodyne/feature. 

 Since you have the gerrit commit hook installed, your feature branch already has 
 Change-Id tags in all commit log messages. 

 Now your feature is complete and you would like to submit it to master. 
 Alas, Gerrit refuses to accept your patch submission for master, because it 
 knows the Change-Ids are also on a different branch. 

 Gerrit by default enforces that a Change-Id must be unique across all branches, 
 so that each submission for review is separate for each branch. Instead, we 
 want to handle Change-Ids per-branch, so that you can have the same change 
 submitted to different branches, as separate patch submissions, without having 
 to cosmetically adjust the Change-Id. 

 Solution: set the option  
 _Create a new change for every commit not in the target branch_ to _TRUE_ 

 h2. Allow content merges 

 By default, gerrit compares patches only by the files' paths. If two paths are the same, 
 it immediately shows them as conflicts (path conflicts). 

 In software development, a conflict usually means an actual content conflict, so if the 
 edits are in two entirely separate places in the file, we don't consider this a conflict. 

 By setting _Allow content merges_ to _TRUE_ in the git project config, we tell Gerrit to 
 perform text merges of the submitted patches and only complain about actual content 
 conflicts, in the usual software engineering sense. 

 h1. Admin 

 h2. Adding users to groups 

 Normally, the gerrit UI auto-completes a user name in the edit field. It has happened 
 though that an existing user is not auto-completed, as if it didn't exist. In that case, 
 find out the user ID (seven digit number like 1000123) and just enter that. 

 The user ID can be found on the user's "Settings" page, or in the database (s.b.). 

 

 h2. Querying the database directly 

 If your user has permission to access the database, you can place SQL queries using the 
 'gerrit gsql' commands over ssh: 

 <pre> 
 ssh go "gerrit 'gerrit gsql -c \"show tables\"" "show tables"' 
 ssh go "gerrit 'gerrit gsql -c \"select "select full_name,account_id from accounts\"" accounts"' 
 </pre> 

 (see ~/.ssh/config above for the 'go' shortcut) 

 This seems to be the MySQL dialect. 

 The "...\"...\"" quoting allows including single-quotes in the SQL statements. 

 h2. Fix evil twin users 

 If differing openid URLs have lead to evil twin users shadowing the same email address just without the permissions, you can fix it like this: 

 <pre> 
 ssh go "gerrit gsql -c \"select * from account_external_ids where email_address like '%foo%'\"" 
 # ACCOUNT_ID | EMAIL_ADDRESS     | PASSWORD | EXTERNAL_ID 
 # -----------+-----------------+----------+---------------------------------- 
 # 100004       | foo@example.com | NULL       | https://osmocom.org/openid/user/777 
 # 100021       | foo@example.com | NULL       | https://projects.osmocom.org/openid/user/777 

 ssh go "gerrit gsql -c \"update account_external_ids set account_id = 100004 where email_address like '%foo%'\"" 

 ssh go "gerrit gsql -c \"select * from account_external_ids where email_address like '%foo%'\"" 
 # ACCOUNT_ID | EMAIL_ADDRESS     | PASSWORD | EXTERNAL_ID 
 # -----------+-----------------+----------+---------------------------------- 
 # 100004       | foo@example.com | NULL       | https://osmocom.org/openid/user/777 
 # 100004       | foo@example.com | NULL       | https://projects.osmocom.org/openid/user/777 
 </pre>
Add picture from clipboard (Maximum size: 48.8 MB)