Using actions - is ghcr.io suppose to be faster than docker.io

[andy-brainome]

I’m trying to cut down the build/test time in a python based container.
I have created a base image from a docker file that does all the repeated apt-get and pip installs.
I have created a second image that imports the custom code and performs the build using nuitka/pyinstaller.
This all works using github actions and docker.io as the container repository.
I tried migrating these two workflows to using ghcr.io as the container repository and have successfully built everything.
Only issue is that - in an action context - ghcr.io is no faster (and marginally slower) than docker.io on a loading a 2.23 GB compressed image.
Is my expectation of a transfer speed increase by staying within the same ecosystem mislaid?
I’m willing to pop the $$ for near-local storage if it could trim load time to less than public storage.

[Simran-B]

I can’t recall where I read it, but someone said that the servers for GitHub Actions and GitHub Packages would be in the different data centers. That would explain why GHCR isn’t faster in workflow runs. It might not be a local data transfer at all, and Docker might happen to be closer to the servers that run the workflows or just be a tad faster or have more bandwidth alotted.

[andy-brainome]

Thank you for the response. Separate data centers is the only situation that makes sense of the telemetry I’ve received. I have not tried to measure the network latency between ghrc and actions but whatever the case - I lost a bit of time porting over and reverting back to dockerhub along with the 2 months of dead silence from this community - lost me as an adopter.
There is no upside to going with ghrc.io with github actions.

[Simran-B]

GHCR still has benefits over Dockerhub, mainly images not expiring and no hard pull limits. But images not being hosted near where actions run is still a bit odd. I believe there was a mention of more caching options. Could be years away, however.

[andy-brainome]

I’m a long way off of tripping hard pull limits although I’ll keep ghrc in mind as a mirror in the event I get popular.
Did they say when they will start billing for ghrc storage? Seems like ancient images feature is a good revenue generator.

[Kurt-von-Laven]

Thai Pangsakulyanont reported improved performance building and pushing to GHCR.

[Mugane]

I've been sitting here waiting over an hour and it's not even half way downloaded I would say run for the hills, do not use, you will retire before it downloads.
docker pull ghcr.io/homeassistant/home-assistant:stable

[sammcj]

Painfully slow pulling images from ghcr.io here in Australia, pulling a 5GB layer takes over an hour on a 1Gbit link!

[sammcj]

?

[alesb2010]

Any solution for that ??
I push a image for ghcr.io sized 800Mb take 1 hour ! That's not good for this service !
The Github team need to improve that

[zastrixarundell]

yep, also an issue in Serbia. Not that 800MB is one hour long, but it definitely is longer than expected. Running a gigabit network and can only pull ~60Mbps.

[ScottMcNaught]

I'm also in Australia an ghcr.io is so ridiculously slow. 🐌

Running this command for an 8GB image:
docker pull ghcr.io/ggerganov/llama.cpp:full-cuda

In a quality datacenter, with a gigabit fiber link, it looks like it will take around 4 hours to download. Around 10mbits per second.

[devnoname120]

Pull speed from ghcr.io is abysmal here too (Switzerland).

[smerschjohann]

yes, and it seems the bigger the image is, the slower it gets over time. If you push an image, the first few GBs are fast, then it becomes very slow...
I'm not sure if that was always the case

[alvitali]

is your ISP Init7 by any chance?

[sladg]

pulled image "ghcr.io/immich-app/immich-server:v1.129.0" in 8m14.546s, takes over 8min to pull 1.5G image (tested twice, same result).

On the other hand, using docker.io as registry, it took 14sec.

Logging in with PAM, does not influence the speeds anyhow.

[boppy]

The problem seems to be the ipV6 connectivity. See this link for details. After disabling ipV6 temporarily my download speeds up to docker-hub levels.

Commands used:

sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1

After setting back (changing =1 to =0) I have 1996-download-speeds again...

[kush-gupt]

I experienced the same results (and fix) here

[DKT69]

Malaysia ISP pull ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release} and ghcr.io/lunatechlab/moontv:latest extremly slow.

[+] Running 10/12
 ⠸ immich-machine-learning [⣿⣿⣿⣿⣿⣿⣿⣿⡀⣿⣿] 48.46MB / 272.3MB Pulling                                                                                           415.4s 
   ✔ dad67da3f26b Already exists                                                                                                                               0.0s 
   ✔ 799440a7bae7 Already exists                                                                                                                               0.0s 
   ✔ 9596beeb5a6d Already exists                                                                                                                               0.0s 
   ✔ 15658014cd85 Already exists                                                                                                                               0.0s 
[+] Running 10/12 Pull complete                                                                                                                [+] Running 10/12.7s 
 ⠸ immich-machine-learning [⣿⣿⣿⣿⣿⣿⣿⣿⣄⣿⣿] 125.7MB / 272.3MB Pulling                                                                                              1234.4s 
   ✔ dad67da3f26b Already exists                                                                                                                                   0.0s 
   ✔ 799440a7bae7 Already exists                                                                                                                                   0.0s 
   ✔ 9596beeb5a6d Already exists                                                                                                                                   0.0s 
   ✔ 15658014cd85 Already exists                                                                                                                                   0.0s 
   ✔ 9b7325825452 Pull complete                                                                                                                                    2.7s 
   ✔ c6d22b64a506 Pull complete                                                                                                                                    2.8s 
   ✔ 4f4fb700ef54 Pull complete                                                                                                                                    2.9s 
   ✔ db713e2c38c8 Pull complete                                                                                                                                    3.0s 
   ⠧ 23b55c87dc07 Downloading     [=======================>                           ]  125.2MB/271.8MB                                                        1231.7s 
   ✔ 39f3a2065336 Download complete                                                                                                                                3.1s 
   ✔ 625f04eb54fe Download complete                                                                                                                                3.4s

even i have faster internet speed. (no ipv6 enable.) any idea to fix?