Project

General

Profile

Feature #5005

script for automatically updating redmine issues from commitlog

Added by laforge 3 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
Low
Assignee:
Category:
-
Target version:
-
Start date:
02/02/2021
Due date:
% Done:

100%

Spec Reference:

Description

In our git commit log we already put annotations like:

Related: OS#1234, SYS#8888
Fixes: OS#5678
Closes: OS#9999
However, nothing is interpreting those commit log messages yet. Ideally one would expec that
  • Related, Closes and Fixes would add some comment to the redmine issue like "Related patch has been posted to gerrit" once the patch is uploaded to gerrit
  • Closes and Fixes would mark the related redmine issue as _Resolved* *once the patch is merged to master.

It would be great if we could somehow automatize this. Posting the related updates to redmine is probably easy, especially with how much steviehs now knows about the redmine API?

The more intresting question is probably how to get triggered from gerrit.

It seems the jenkins-gerrit integration has a similar "problem" and uses an approach described at https://stackoverflow.com/questions/16106898/how-does-the-gerrit-trigger-plugin-in-jenkins-works as
  • connect to gerrit via SSH using teh "stream-events" command
  • receive a stream of events, parse them and then perform related actions

The stream is described at https://gerrit-review.googlesource.com/Documentation/cmd-stream-events.html and it seems to be a stream of json objects describing what happens. it looks like patchset-created and change-merged are the relevant events. The entire commit message seems to be present in those events, ready for parsing.

I'm wondering if we really should only add "comments', or if we actually should add a custom field to redmine issues, allowing us to track the relationship more structured. That custom field could contain a list of gerrit change-ids, for example.

Let's use this ticket to collect some ideas. And let's see if steviehs thinkks this would be something he could implement.


Related issues

Related to Osmocom.org Servers - Feature #3291: gerrit - redmine integrationNew05/25/2018

Associated revisions

Revision 57b6011e (diff)
Added by laforge 3 months ago

redmine: Make changeset keywords work with OS# annotation

In Osmocom we annotate osmocom issues as OS#1234 and not just as #1234,
in order to distinguish them from redmine or coverity issues.

Change-Id: I04a97434433a022f47a759a8219458e8772ae71e
Related: OS#5005, OS#3291

History

#1 Updated by laforge 3 months ago

#2 Updated by laforge 3 months ago

related note: Redmine actually does have our git repositories configured for each project, and it has configurable referencing/fixing keywords for commitlog parsing. We even configured it for "Closes", "Fixes" and "Related".

However, it expects the issue number like #5005 and not as OS#5005. And that's not configurable, so we'd have to patch redmine I suppose :/

The almighty regex in redmine seems to be in scan_comment_for_issue_ids()

    comments.scan(/([\s\(\[,-]|^)((#{kw_regexp})[\s:]+)?(#\d+(\s+@#{TIMELOG_RE})?([\s,;&]+#\d+(\s+@#{TIMELOG_RE})?)*)(?=[[:punct:]]|\s|<|$)/i) do |match|

wehre "kw_regexp" is what we can configure in the redmine UI, but the part ahead of the "#" cannot be configured and hence anthing not matching "[\s\(\[,-]|^)" will be disregarded.

#3 Updated by laforge 3 months ago

See https://gerrit.osmocom.org/c/docker-playground/+/22815 - this should now be working for the most relevant projects (where we have the git repo as "repository" tab here in redmine).

#4 Updated by osmith 2 months ago

Works great for most repositories.

Daniel added docker-playground.git here, now that works as well: But for osmo-ci.git it does not work. We've tried to add it here:

See also: https://projects.osmocom.org/projects/core-testing-infra/settings/repositories

Seems to be an issue on the server, about /usr/local/git/repositories/osmo-ci.git not being there (though other repos appear to be in that directory), or wrong permissions?

#5 Updated by laforge 2 months ago

On Tue, Mar 02, 2021 at 12:37:33PM +0000, osmith [REDMINE] wrote:

But for osmo-ci.git it does not work. We've tried to add it here:

this relates to the way how we manage filesystem ACLs to provide access. Ironically,
there are a number of refs/heads/osmith/* branches where the redmine user has no read access
to.

What we currently do is something like

setfacl -m default:user:999:rX -R /path/to/repositories/osmo-ci.git

however, that only recursively sets the default ACL to all directories including
...../repositories/osmo-ci.git/refs/heads/osmith

which looks like this:

# owner: 100
# group: 101
user::rwx
group::r-x
other::---
default:user::rwx
default:user:999:r-x
default:group::r-x
default:mask::r-x
default:other::---

The individual files underneath however are somehow not affected by that default acl
inthe same way? See:

# file: /repositories/osmo-ci.git/refs/heads/osmith/centos-install-test
# owner: 100
# group: 101
user::rw-
group::r--
other::---

# file: /repositories/osmo-ci.git/refs/heads/osmith/dependency-check
# owner: 100
# group: 101
user::rw-
group::r--
other::---

# file: /repositories/osmo-ci.git/refs/heads/osmith/fix-python3
# owner: 100
# group: 101
user::rw-
group::r--
other::---

# file: /repositories/osmo-ci.git/refs/heads/osmith/obs-fixes
# owner: 100
# group: 101
user::rw-
group::r--
other::---

# file: /repositories/osmo-ci.git/refs/heads/osmith/obs-latest-fix
# owner: 100
# group: 101
user::rw-
user:systemd-coredump:r-x       #effective:r--
group::r-x                      #effective:r--
mask::r--
other::---

# file: /repositories/osmo-ci.git/refs/heads/osmith/rpm
# owner: 100
# group: 101
user::rw-
group::r--
other::---

# file: /repositories/osmo-ci.git/refs/heads/osmith/ttcn3-centos
# owner: 100
# group: 101
user::rw-
group::r--
other::---

# file: /repositories/osmo-ci.git/refs/heads/osmith/ttcn3-latest
# owner: 100
# group: 101
user::rw-
group::r--
other::---

# file: /repositories/osmo-ci.git/refs/heads/osmith/wip
# owner: 100
# group: 101
user::rw-
group::r--
other::---

Maybe the "default" only applies to newly-created files, and the old, existing
ones still have no ACLs at all?

#6 Updated by laforge 2 months ago

Indeed, something like

find repositories/osmo-ci.git -type f -exec setfacl -m user:999:r \{\} \;

seems to have resolved it.

#7 Updated by daniel 2 months ago

laforge wrote:

Maybe the "default" only applies to newly-created files, and the old, existing
ones still have no ACLs at all?

Yeah, you need the default so newly created files and directories will get this permission automatically, but that doesn't change the permission of existing files (I guess this also means that default:* is only important for directories).

#8 Updated by laforge about 2 months ago

  • Status changed from New to Resolved
  • Assignee changed from steviehs to laforge
  • % Done changed from 0 to 100

Also available in: Atom PDF

Add picture from clipboard (Maximum size: 48.8 MB)