git.osmocom.org: error: garbage at end of loose object
Right now, I can't clone libosmo-sccp from https://git.osmocom.org/libosmo-sccp. It fails with:
$ git -C /home/user/code/osmo-dev/src/osmo-ci/_temp/repos clone https://git.osmocom.org/libosmo-sccp Cloning into 'libosmo-sccp'... error: garbage at end of loose object 'a70b4599f63b50c4d654eb9e1b1a099a7c8cdc36' fatal: loose object a70b4599f63b50c4d654eb9e1b1a099a7c8cdc36 (stored in /home/user/code/osmo-dev/src/osmo-ci/_temp/repos/libosmo-sccp/.git/objects/a7/0b4599f63b50c4d654eb9e1b1a099a7c8cdc36) is corrupt
I have noticed this multiple times with various repositories in the past week.
A few minutes later, it is working again. So maybe this is related to some sync job?
This is still happening, e.g. here:
There is a "synch job" and that's the gerrit replication job that pushes/mirrors
gerrit.osmocom.org to git.osmocom.org. We're using the built-in gerrit replication
service for this, which in turn AFAIK just does a git+ssh push, which shouldn't introduce
any races. After all, it happens quite frequently that somebody clones a public git
repo while somebody else is pushing something.
However, pulling/cloning over https means there's a nginx reverse proxy between the server
and the client, and there may be some problem introduced here.
It would be useful to have exact timestamps to check if there's anything in the log. So
maybe it's worth to modify related scripts to print a timestamp before doing the clone?
The interesting bit from our config is that the http+https access to git.osmocom.org
are actually redirected to cgit.osmocom.org
To be honest, I never really looked too deep into how git over http works, given that
in the past I've only used it with the native git protocol and with ssh. For a long
time AFAIR it was even discouraged to use http* with it.
laforge, here is one set of timestamps:
+ date '+%Y-%m-%d %H:%M:%S' 2019-07-29 07:01:59 + git clone https://git.osmocom.org/libosmo-sccp Cloning into 'libosmo-sccp'... error: garbage at end of loose object '70751fce24f989d7334cba180e34ca65840ed0f9' fatal: loose object 70751fce24f989d7334cba180e34ca65840ed0f9 (stored in /home/osmocom-build/jenkins/workspace/Osmocom_OBS_nightly/nightly-3g_QnWQVB/repo/libosmo-sccp/.git/objects/70/751fce24f989d7334cba180e34ca65840ed0f9) is corrupt + date '+%Y-%m-%d %H:%M:%S' 2019-07-29 07:02:10 + exit 1