index
index

That is an index file that works for the 64 bit image:

{{{
$ virt-builder --no-check-signatures --source file:///tmp/index fedora-20-alpha
[ 0.0] Downloading: file:///tmp/x86_64/Fedora-x86_64-20-Alpha-20130918-sda.raw.xz

################################################################## 100.0%

[ 2.0] Creating disk image: fedora-20-alpha.img
[ 2.0] Uncompressing: file:///tmp/x86_64/Fedora-x86_64-20-Alpha-20130918-sda.raw.xz
[ 16.0] Opening the new disk
[ 41.0] Setting a random seed
[ 41.0] Random root password: XhEaeDvy972k3eBg [did you mean to use --root-password?]
[ 41.0] Finishing off
Output: fedora-20-alpha.img
Total usable space: 1.8G
Free space: 1.3G (69%)
}}}

It needs the 32 bit fields filling in (checksum & compressed_size).

The whole file needs to be GPG clearsigned.

I didn't bother with all the ARM images.

We need to have tooling to create the indexes, and we do not do detached signatures. we can do inline signing like we do for the CHECKSUM files

Replying to [comment:2 ausil]:

We need to have tooling to create the indexes, and we do not do detached signatures. we can do inline signing like we do for the CHECKSUM files

I can chuck together a sed script or something, but not right this minute.

I didn't mean detached signatures, I meant:

{{{
gpg --clearsign --armor index
}}}

which is what is (probably) used for CHECKSUM.

Dennis, if we got the tooling made in a relatively short amount of time, do you think it's reasonable to have this in place for f20? And, if so, what would "a relatively short time" be?

we would need it and integrated into the compose process before Final TC1. note, we only sign the gold release after its declared such.

I think when you say "tooling" you mean that this has to be generated as part of some existing process or program? If so could you point me to what process it is that generates the current files & directories? I don't even know where to start.

The release engineering scripts live at https://git.fedorahosted.org/cgit/releng/tree/scripts, but I'm not sure exactly how they all fit together. Maybe it would be called from https://git.fedorahosted.org/cgit/releng/tree/scripts/build-cloud-images?

what we need is a stand alone script that generates the data in the correct format, then we can call it with the paths etc in https://git.fedorahosted.org/cgit/releng/tree/scripts/run-pungi which is the overview compose script. we would call it after we gather the whole tree together. I do beleve we should generate the data for all arches, i.e. the armhfp, i386 and x86_64 trees.

We composed Final TC1 yesterday, so at this point we will have to work on it for F21

I'm going to host an index file pointing to the F20 images to mitigate this. It requires a small change to virt-builder (since currently it only allows relative links).

This needs a script to create the index file.

Attached is an example of a shell script that could be used to create the index for this directory: http://www.mirrorservice.org/sites/dl.fedoraproject.org/pub/fedora/linux/releases/21/Cloud/Images/x86_64/

make-index.sh
make-index.sh

we need to work out a way to integrate this into Fedora 23

Some extra info. we use sigul to do all signing of things, we need to have a tool that we point at the Cloud images tree and it has to figure out everything from teh contents of the directory. we can pass along things like the fedora version. but the filenames, size of the image, etc needs to be calculated on the fly.

Probably too late to do this for F24, but it'd be nice to figure out for F25. Additionally, it'd be nice to coordinate with CentOS -- https://lists.centos.org/pipermail/centos/2016-April/158794.html

FWIW, I notice that virt-builder now supports simplestreams on its own.

... although Simplestreams metadata isn't sufficient on its own. See: https://www.redhat.com/archives/libguestfs/2016-April/msg00171.html

By which I mean Simplestreams would be fine, but those extra fields are required, and we'll probably need to make some changes to virt-builder to actually use them.

FYI: We're looking into the virt-builder index format over in CentOS, but not set in stone yet. Happy to collaborate on this.

@bstinson have you guys done anything here yet?

We have a script that walks a directory and generates the virt-builder index format from any of the images it finds:
http://cloud.centos.org/centos/7/images/image-index

Will need to set up autosigning to make this work seamlessly. Confirm if we still want 1 index file for everything. If so, this will be not be trivial.

We don't need one index file for everything. However it does need to be signed (which unfortunately http://cloud.centos.org/centos/7/images/image-index is not).

We don't need one index file for everything. However it does need to be signed (which unfortunately http://cloud.centos.org/centos/7/images/image-index is not).

Very true, however: http://cloud.centos.org/centos/7/images/image-index.asc is signed

Since this is an RFE and not a break-fix, moving to https://taiga.fedorainfracloud.org/project/acarter-fedora-docker-atomic-tooling/us/996?kanban-status=145 and closing this ticket.