I’d like (or rather have to) to setup Windows gitlab-runner. And I try to follow the KDE way.
In gitlab-templates · master · Sysadmin / CI Utilities · GitLab in the “Windows-Hosts” section the setup for a Windows VM capable of running Windows Docker images for the CI is described.
But with Port Windows images to VM based CI setup (!441) · Merge requests · Sysadmin / CI Images · GitLab there seems to be a move away from Docker on Windows
Two questions on my mind:
- is there documentation (or something I can read up on) why the move away from Docker on Windows?
- is there documentation on how the new qcow based images are started/ran (how to setup the host to run the qcow VM image)
Thanks in advance!
There were two email threads about this:
You can also ask questions on Matrix in #kde-sysadmin:kde.org I believe.
I’ve now added some documentation to detail this. See README.md · master · Sysadmin / CI Images · GitLab
Note that you will probably want to master your own Windows image. See windows11-base · master · Sysadmin / CI Images · GitLab for my instructions on that particular one.
vm-runner is designed to work with any publicly accessible S3 storage and will work most optimally if that is available but it can also import local disk images by being passed a path to the qcow2 file. Note that in this case it will be using a full deep copy (rather than a thin provisioned copy-on-write image which is what happens with S3 downloaded images)
In terms of why we abandoned Docker on Windows it came down to a couple of things:
- Windows Docker images are large, and cannot be easily cleaned up.
- If you do not reboot a Windows host prior to trying to remove images then you can end up with a significant amount of lost disk space that is impossible to free using normal tools.
- In these situations only GitHub - moby/docker-ci-zap will be able to help you, and that tool completely wipes the entire Docker data folder from the system, requiring everything to be downloaded again
- As a consequence, you need to arrange downtime, stop processing jobs on that runner and reboot it each time you need to clean it up
- Windows Docker images have extremely poor I/O:
- If your build process involves touching a significant number of small files, then your build time will be significantly longer within a Windows Docker container
- For Craft (which makes provides Boost as some software relies on it) we ended up providing a host volume (which circumvents the issue) during the period we used Windows Docker containers otherwise builds would time out during the setup phase.
- Windows Docker images are lacking key system facilities.
- Some Windows APIs rely on system services (such as anything COM+ related) which are not running in the Docker containers, so they don’t work.
- In addition many GUI related APIs also fail completely as that is also disabled in Docker.
- Windows Docker images mean looking after Windows hosts, which for us was in addition to the Linux hosts we already had. Moving to ephemeral images saves us this maintenance cost.
We were also experiencing issues with the flexibility of the build pool, as previously it was fixed to 2x Linux, 1x FreeBSD and 1x Windows job per node. While we can now only run 3 jobs per node (64GB RAM build hosts, 16GB RAM per VM) we can now run whatever mix of VMs is needed to get jobs completed which results in builds finishing faster overall.
Thank you for the thorough writeup! This is pure Gold for me! And drives me completely away from using Docker on Windows (thank you for sparing me the agony!)
One last question: how do you handle the Windows Licenses? Do you have 6 Licenses, as you can run up to 6 concurrent jobs (3 on each of your 2 nodes). How are those licenses and VMs linked?
Or just one for the base image? (no idea with the Microsoft licensing jungle)
Sorry I didn’t get notified of your reply.
We hold sufficient licenses for the maximum number of VMs running.
1 Like