#5126 RFR: OSBS/docker system requests
Closed: Fixed None Opened 8 years ago by smooge.

= phenomenon =

Fedora 24 release engineering needs to stand up 4 systems for the Open Shift Build System (OSBS) and Docker registry.

1 stage systems/1 production [osbs]

RAM: 16 GB
CPU: 4 cores
Storage:
20 GB of system (/)
100 GB of separate localized storage disk (/var/lib/docker) for docker (not NFS).
Networking:
* Build network

1 stage systems/1 production [docker-registry]

RAM: 16 GB
CPU: 4 cores
Storage:
20 GB of system (/)
100 GB of separate localized storage disk (/var/lib/registry) for docker (not NFS).
Networking:
* Build network


any help needed when this once deployed (testing.etc.)? please keep me in the loop...doteast

So, some questions:

  • What OS do these need to be?

  • Can they be load balanced if we had two of them, or are they one at a time active?

  • Would it make sense to have any caching in front of these?

  • What all folks are involved in this? Maxamillion and smooge? Any others?

  • I am assuming these can be VMs or is there any bare hardware requirement?

  • OS: Fedora 23 please

  • Unfortunately there is no load balancing involved in any of this yet, OSBS is a single-node system at the moment. The layout of the machines in the request are: osbs-stg, docker registry stg, osbs-prod, docker registry prod

  • Probably not, the osbs builders wouldn't benefit from it and the cach end point would have to be pulp crane or something else that implements the docker-registry endpoint rest api to really be useful.

  • maxamillion

  • All VMs are fine, we wouldn't really gain much from bare metal deployment.

ok, thanks for the info.

A few other random questions:

  • I assume people would download from the registery? Is that likely to be a lot of BW/storage? I assume there's no way to mirror this content? and it cannot be nfs?

  • Do either or both of these things use a database? ie, where is the information stored. If we had content and saved the /var/lib/registery we could rebuild everything and it would be back as before?

  • Do both of these systems need to be accessable from the outside? or can they just be internal? I assume the registery might need to allow https to let people download from it? does the osbs ?

In order:

  • People will download from the registry, probably yes to both. We're trying to figure out how to mirror content, but that's likely something we'll sort out in Fedora 25 time frame. It can be NFS, but it can not be shared NFS. I've been told that causes intermittent issues with docker-distribution registry.

  • The docker-distribution registry does not use a database. Technically OpenShift uses an etcd database, but at this time the osbs system is being considered a "stateless builder" because all the relevant metadata should be stored in koji and we can reproduce builds if necessary.

  • registry needs external connectivity, osbs can be internal only but will need to be able to talk to the koji builders and the koji hub.

My apologies for the churn on this but I was incorrect on the storage requirements needed for the osbs master nodes, instead of having the storage mounted, I would like to request separate logical volumes and possibly a pre-defined volume group name (not a hard requirement, just a nice to have).

This is what I did in my test VM here locally:

{{{
$ pvcreate /dev/vdb
$ vgcreate vg-docker /dev/vdb
$ lvcreate -L 95G -n data vg-docker
$ lvcreate -L 5G -n metadata vg-docker
}}}

However, I could do with just the data and metadata logical volumes using whatever the default volume group is created as part of the kickstart.

Thank you,
-AdamM

This is now all in production. Thanks!

Login to comment on this ticket.

Metadata