#6366 missing metadata about nightly rawhide/24 composes
Closed: Fixed 6 years ago Opened 8 years ago by bpeck.

Currently there is no metadata about a compose in the rsync share. This makes it very difficult to know what is available to rsync/import.

rsync rsync://dl.fedoraproject.org/fedora-linux-development/rawhide/


Fedora Master Mirror Servers

Modules for Fedora Core and Extras have been removed, as this content is
no longer updated. See the instructions below for how to mirror current
content.

See http://fedoraproject.org/wiki/Infrastructure/Mirroring for
instructions.


drwxrwxr-x 4,096 2016/03/09 13:26:55 .
drwxrwxr-x 4,096 2016/03/08 11:08:04 Atomic
drwxrwxr-x 4,096 2016/03/08 07:33:16 CloudImages
drwxrwxr-x 4,096 2016/03/08 07:33:17 Docker
drwxrwxr-x 4,096 2016/03/08 07:33:20 Everything
drwxrwxr-x 4,096 2016/03/08 07:36:56 Server
drwxrwxr-x 4,096 2016/03/08 07:37:03 Spins
drwxrwxr-x 4,096 2016/03/08 07:37:04 Workstation

In the past we used to have a .composeinfo file which not only told us which trees were available in a compose but whether the compose completed successfully.

Both of these attributes are important to keep testing. We don't want to overwrite a working release with a broken one.

It appears that the .composeinfo file format is going away and being replaced with metadata/composeinfo.json which is fine. We can update our code to work with this but according to dgilmore its not as easy as simply copying this file over since the compose gets carved up.

It sounds like the process that does the carving should produce a new composeinfo.json which describes the carved up version.

Until this ticket is resolved we will not be able to mirror Fedora inside Red Hat for testing.


I am honestly not sure the best way to resolve this. A big part of the problem is taht we split the compose up after it is done.

Lets look at a nigtly f24 compose, The compose is done in

https://kojipkgs.fedoraproject.org/compose/branched/Fedora-24-20160318.n.0/compose/

We have a bunch of directories that have the various parts of the compose
{{{
Atomic Cloud CloudImages Docker Everything Labs metadata Server Spins Workstation
}}}
the metadata directory has a composeinfo.json file in it that describes nicely all the content of the compose

however, when we put the compose on the mirrors, we go and carve it up.
{{{
Atomic CloudImages Docker Everything Server Spins Workstation
}}}
all go into the main public location
http://dl.fedoraproject.org/pub/fedora/linux/development/24/

we then put the remianing deliverables over in alt
{{{
Cloud Labs
}}}
http://dl.fedoraproject.org/pub/alt/development/24/

throwing metadata on the floor as it now is incorrect.

there is .treeinfo files all through the compose but you need to tell beaker where to look for them.

we have a ticket https://pagure.io/pungi/issue/98 open to enable us to change things up to better support our use cases.

AFAIK the only use case for .composeinfo is beaker. dmach would be the person that made the change from .composeinfo to composeinfo.json

another way to solve this is pungi gives a message when it is done composing, it is follow up with a message later that says the rsync is done. If you use the message from pungi you could fetch the composeinfo.json and prep things for when the rsync is done. knowing how we carve things up. It is not as ideal as it could be as we would need to let you know if we change how we carve things up, but I do not anticipate changing that before we get pungi being able to better support our use cases.

Secondary arches are a whole different problem. Since we are merging three composes into one.

Metadata Update from @bpeck:
- Issue set to the milestone: Fedora 24 Alpha

7 years ago

I believe this is all okay now

Metadata Update from @ausil:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

6 years ago

Login to comment on this ticket.

Metadata