Bug #5670
closed
git.osmocom.org: cloning via https:// is slow
Added by fixeria over 1 year ago.
Updated over 1 year ago.
Description
I just noticed that cloning from git.osmocom.org via https:// is significantly slower than cloning from Gitea or Gerrit.
$ time git clone https://gerrit.osmocom.org/libosmo-sccp
Cloning into 'libosmo-sccp'...
remote: Counting objects: 127, done
remote: Total 5302 (delta 0), reused 5302 (delta 0)
Receiving objects: 100% (5302/5302), 1.94 MiB | 37.49 MiB/s, done.
Resolving deltas: 100% (3049/3049), done.
real 0m0.204s
user 0m0.211s
sys 0m0.056s
$ time git clone https://gitea.osmocom.org/osmocom/libosmo-sccp
Cloning into 'libosmo-sccp'...
remote: Enumerating objects: 5302, done.
remote: Counting objects: 100% (5302/5302), done.
remote: Compressing objects: 100% (2149/2149), done.
remote: Total 5302 (delta 3134), reused 5102 (delta 2984), pack-reused 0
Receiving objects: 100% (5302/5302), 1.74 MiB | 36.38 MiB/s, done.
Resolving deltas: 100% (3134/3134), done.
real 0m0.244s
user 0m0.242s
sys 0m0.034s
$ time git clone https://git.osmocom.org/libosmo-sccp
Cloning into 'libosmo-sccp'...
Fetching objects: 5302, done.
real 0m14.780s
user 0m1.147s
sys 0m1.166s
Note that even the output looks different when cloning from git.osmocom.org.
Files
Running git clone https://gitea.osmocom.org/osmocom/libosmo-sccp with -vv:
$ git clone -vv https://gitea.osmocom.org/osmocom/libosmo-sccp
Cloning into 'libosmo-sccp'...
POST git-upload-pack (164 bytes)
want e7228dbdd27c16b65a0ed26373187b8418820204 (HEAD)
want c288852320dd09b3511396729c4f5d05c5c04cee (refs/heads/2021q1)
want 5cfda55c70865455433e14716acf7e86b7f8375b (refs/heads/2021q4)
want 64dda770c3035b7d5e2549eef8e8c4ce86ab34c4 (refs/heads/2022q1)
want e9cc5c937663bb9ad0c408c65acda017b25b7520 (refs/heads/2022q2)
want 6dbe76a021740cb9ac10d1a42dab6f92a97c26e0 (refs/heads/35c3)
want b6ce5ee7ba4b9f5fe3a9ffda3b26895929d6e02a (refs/heads/daniel/onwaves)
want 655cd19c9d99fbbfc0895339d1c7fcdf4f5109ea (refs/heads/debian)
want 686877c7f32f8f17511f913139ef1e41c4c04a93 (refs/heads/fairwaves/production)
want b6232f90cfcc8b923d50f9bbebf73d53831491aa (refs/heads/laforge/benchmark)
want 5489ccc37d55f25c750e287b5639215771bb492d (refs/heads/laforge/hack)
want 888d062068721c11927d64f03f8cf98faf343572 (refs/heads/laforge/ipa)
want 32c53c50175ca2d84217ef74df1cd40f30a65089 (refs/heads/laforge/ludt)
want 5a701ac80bc219d85ad3cb7c6877566fb7499894 (refs/heads/laforge/xua-memdebug)
want e7228dbdd27c16b65a0ed26373187b8418820204 (refs/heads/master)
want cd5507b7acad83472f628fdd00d2be8f38eed3d7 (refs/heads/neels/big_endian)
want 7a4f614ba912241fabfaa73d1efc8f17ad8d6275 (refs/heads/neels/lcs)
want eb8f34b69f8bbb6593da21ffdb9406ba9059e75c (refs/heads/neels/ni)
want af39c7a78a37058ec4a2f2dc0d4e588aad46b3a1 (refs/heads/neels/sccp_timers)
want d27c1186fe8f6786bb41de6888f23fa34d8f7705 (refs/heads/neels/sccplite)
want 9b5423b5f438ecb00aa67cf131c98b745ee1c0e2 (refs/heads/neels/tmp_hacks)
want 8e75e6388c93ea2dc2e9ad1a08989b603ac5cbf2 (refs/heads/osmith/conn-id-scoping)
want dc8451de82afe711bcd86e3828bc4ae3d80393b9 (refs/heads/osmith/fix-python3)
want ad695405d99cb9964981e76a5abeade7cd88e0f8 (refs/heads/osmith/rpm)
want d4998abd4632d82d598f11b5d4775d364ec36416 (refs/heads/osmith/wip)
want 5cac653a986487470444b81132305aeae0a3e045 (refs/heads/pmaier/fixup)
want 5a834f300c2603a595390f6247167e07770e8a86 (refs/heads/zecke/0.7.0-xudt)
want 0cdee842d9a46b9aa56588477d95432f7db003e4 (refs/heads/zecke/ancient/debian6.0)
want 7570263227bed4d28756f1bf2d0363edfbd7ceff (refs/tags/0.0.1)
want 366090b1f9e411d62f281fa248a5a9ee4e78d315 (refs/tags/0.0.2)
want b6df10774731e0225d7f78ffe649fbc26a0bf465 (refs/tags/0.0.3)
want 2824003b5f5340c67291173a8d20270423bee39a (refs/tags/0.0.4)
want da4f36863bef955922449d1bbcc84a43405acb01 (refs/tags/0.0.5)
want 600c2e7fd6e41d679b6d210fa17b97146458eff3 (refs/tags/0.0.5.1)
want fc5f3348f02d7f9d7708a91ece4bc47b758be936 (refs/tags/0.0.6)
want 49f7e56a35092e7cbf1624ddda788374faceead8 (refs/tags/0.0.6.1)
want 825c5cfd05104d089174a386aa5f480b037ae26d (refs/tags/0.0.6.2)
want ec14eaca9e21cc74173e2a9646171e6717c8e88c (refs/tags/0.0.6.3)
want 8cabaf7dea0d9c4eddea8526b7b58db514bc77ce (refs/tags/0.10.0)
want 46c2945af2f7d6741fdba7185241845f1d2c7421 (refs/tags/0.7.0)
want 35f936d797aaee397928a5f5ab21ccc34af50a66 (refs/tags/0.8.0)
want 798d6355ac5b7f6916d1a5b581f4648dcc09325a (refs/tags/0.8.1)
want d7989d140b1ffb65df9abb52bfec35b943b6c186 (refs/tags/0.9.0)
want c99143b5113ff78cc956ca5d2622e67f2cfc26fe (refs/tags/1.0.0)
want 54e232bb5accf2af705c8d21c3e970404f9a23b0 (refs/tags/1.1.0)
want 3d7ef3ab9744d44c25ddfebcdc2e713308443716 (refs/tags/1.2.0)
want 8d411cefada4f634708d5e929f39d6b7a303820d (refs/tags/1.2.1)
want 723de8781780e035523a0c3c75094843abebf9cf (refs/tags/1.3.0)
want 339fb9e695e323a391a3fa6326a3eeb8db8385f4 (refs/tags/1.4.0)
want 49b16a11d2c2f7df11a8ae92a06f89f2cd6daac2 (refs/tags/1.5.0)
want 56f605795f529ca35806e3aef5b68cc7de7bd15e (refs/tags/1.6.0)
want c1307ee64d25f4b19397bcf4791ba4c85d1dbe79 (refs/tags/3G_2016_09)
want 686877c7f32f8f17511f913139ef1e41c4c04a93 (refs/tags/fairwaves/1.2.0-fw.1)
want 8e708d1f2da1b187f631bf08172a5194a85b1a23 (refs/tags/old_sua)
POST git-upload-pack (gzip 2802 to 1403 bytes)
remote: Enumerating objects: 5302, done.
remote: Counting objects: 100% (5302/5302), done.
remote: Compressing objects: 100% (2149/2149), done.
remote: Total 5302 (delta 3136), reused 5100 (delta 2984), pack-reused 0
Receiving objects: 100% (5302/5302), 1.74 MiB | 38.69 MiB/s, done.
Resolving deltas: 100% (3136/3136), done.
Running git clone https://git.osmocom.org/libosmo-sccp with -vv produces a lot of output (~4.8k lines), see attached.
Running git clone https://git.osmocom.org/libosmo-sccp with GIT_TRACE=1 GIT_TRACE_CURL=1. See attached.
The redirects currently only work for URLs terminating in the repository name, but not for URLs beneath it. I changed it for libosmo-sccp.
Please rewrite the config file to make all lines look like that of libosmo-sccp
- Related to Bug #5658: sporadic failures when cloning from https://git.osmocom.org/ added
unfortunately this breaks some of our builds like https://jenkins.osmocom.org/jenkins/view/All%20no%20Gerrit/job/nplab-m3ua-test/1884/console
Cloning into 'libosmo-sccp'...
Removing intermediate container 5e19e7bee50b
---> 69a268dfeb30
Step 9/14 : ADD http://git.osmocom.org/libosmo-sccp/patch?h=$OSMO_STP_BRANCH /tmp/commit
ADD failed: failed to GET http://git.osmocom.org/libosmo-sccp/patch?h=master with status 404 Not Found: Not found.
../make/Makefile:90: recipe for target 'docker-build' failed
make: *** [docker-build] Error 1
make: Leaving directory '/home/osmocom-build/jenkins/workspace/nplab-m3ua-test/osmo-stp-master'
So we need to make sure http[s]://git.osmocom.org/libosmo-sccp/patch.* still ends up at cgit until we have found a gitea replacement...
I've now replaced the regex map with the attached one.
Also available in: Atom
PDF
