Ticket #4986 (new task)

Opened 5 years ago

Last modified 3 months ago

Create a 'Deliverables' page / SOP listing exactly what images, torrent files, signatures, checksums etc should be provided with (pre)-releases

Reported by: adamwill Owned by: rel-eng@…
Milestone: Component: other
Keywords: Cc: ausil, robatino, nirik, jdulaney, jkurik, acarter
Blocked By: Blocking:

Description

Going through the F16 QA retrospective, there's this series of linked points from Andre Robatino:

"robatino - Alpha and Beta torrents often have unsigned checksum files - this just happened with 16 Alpha. It also happened during F13 and F14 - see http://lists.fedoraproject.org/pipermail/test/2010-April/090106.html for F13 Beta, and https://bugzilla.redhat.com/show_activity.cgi?id=638738 for F14 Alpha and Beta. When it's pointed out, we're told that it's not feasible to change them once people have started downloading, but if this is true, there needs to be better checking (at least as much as for mirror content, which can be changed) to make sure the torrents are correct before being posted. Filed https://fedorahosted.org/fedora-qa/ticket/237 and https://fedorahosted.org/rel-eng/ticket/4906 . No response from releng as of yet. Unless QA is given access prior to public posting, it is powerless to prevent this. robatino - Also, it's possible that the checksum files can be signed more than once. There are two versions of the signed checksum files for F15, one on the torrents (the original ones) and a different one on the mirrors. See http://robatino.fedorapeople.org/checksums/15-Final/ for the two versions - for example, Fedora-15-i386-CHECKSUM.orig and Fedora-15-i386-CHECKSUM. While not nearly as serious as the above issue (since the checksums are the same and both signatures are good), it could be confusing. "

A quick mail discussion prompted the suggestion from Andre that the best way to cover all these issues would be for there to be a Deliverables wiki page / SOP which clearly lists all the 'bits' that should be delivered for each image drop - Alpha, Beta, Final, TC and RC. This would provide a simple reference for whoever's doing the build to make sure everything has been provided.

It would make most sense for this to live in releng's wiki space, I think, so filing against releng trac, but a QA person could certainly take care of creating the page in theory, I think.

We should have this in place prior to F17 Alpha TC1.

Change History

comment:1 Changed 5 years ago by adamwill

  • Cc dennis@…, robatino, nirik added

Apologies - the suggestion was from Kevin Fenzi, not Andre.

comment:2 Changed 5 years ago by ausil

  • Cc ausil added; dennis@… removed

comment:3 Changed 5 years ago by jdulaney

  • Cc jdulaeny added

comment:4 Changed 5 years ago by jdulaney

  • Cc jdulaney added; jdulaeny removed

comment:5 Changed 3 years ago by till

  • Component changed from koji to other

this does not seem to be a koji-related ticket

comment:6 Changed 4 months ago by acarter

PGM has begun publishing release deliverables [1] but the list that's maintained is not kept up to date and does not cover the concerns outlined here. Because the list is not well maintained it's not really used by many stakeholders. We need to identify an owner for this list and enable that owner to keep it updated in a sane way. The list should be systematically queryable by release and QE tooling at least so we can validate expectations. A likely candidate for storing this data is PDC.

[1] https://fedoraproject.org/wiki/Fedora_Program_Management/ReleaseBlocking/Fedora24

comment:7 Changed 3 months ago by acarter

  • Cc jkurik added

comment:8 Changed 3 months ago by acarter

Jan, as noted above, we do have a doc but it's not updated and not enforced as part of the release process which means we have no canonical source of what we're shipping for a release. We'd like for this to be a gating list with a defined process around it. Ideally this would be systematically queryable in a tool like PDC eventually.

comment:9 Changed 3 months ago by jkurik

I brought this topic to the FESCo [1] for their last meeting [2]. There was no decision made and the topic has been re-scheduled for the meeting on 2016-Apr-22 due to pending details from rel-eng team.

[1] https://fedorahosted.org/fesco/ticket/1566
[2] https://meetbot.fedoraproject.org/teams/fesco/fesco.2016-04-15-17.00.log.html

comment:10 Changed 3 months ago by adamwill

hah. this ticket amuses me greatly, since I've been going round for the last week saying what a dumb idea it was to keep this list in a wiki page. guess which idiot's dumb idea it was. ;)

Last edited 3 months ago by adamwill (previous) (diff)

comment:11 Changed 3 months ago by jkurik

There was quite extensive discussion about this release deliverables list on FESCo meetings during F23 live cycle. The current format presented on [1] is the format FESCo finally agreed on. There was also agreement on the way how this list can be changed, which is via Change Process, so FESCo has full control on it. As far as I know the list [1] is currently up to date.

Using PDC instead of this wiki page will be IMO welcomed by many parties. I am going to point to this ticket during the FESCo meeting on 2016-Apr-22, so we can decide how we would like to track this.

[1]https://fedoraproject.org/wiki/Fedora_Program_Management/ReleaseBlocking/Fedora24

comment:12 Changed 3 months ago by adamwill

We do have some discussion ongoing around this in pungi tickets, ref. https://pagure.io/pungi/issue/249 and https://pagure.io/pungi/pull-request/269 .

comment:13 Changed 3 months ago by acarter

I understand from DGilmore that there are in fact things that we're shipping that aren't on that list (see today's flurry around Live USB amongst others). Jan, please just keep us updated on the discussion and if you want to propose how we should handle this in the future, I think we'd all be on board to make it happen :)

comment:14 Changed 3 months ago by ralph

On PDC, two things:

  • PDC stores the list of things that we actually did produce (by way of importing the metadata output from pungi).
  • When we first started looking at it, we were thinking about also storing the list of things we *should* produce in a compose as well (which would meet the criteria for this ticket). While the PDC authors were originally receptive to that idea, it seems there is concern out there that the scope of PDC is growing too large and so pushing for an expansion to handle this case may meet with opposition from upstream.

Another idea: since we keep our pungi config public here https://pagure.io/pungi-fedora/raw/master/f/fedora.conf and since it actually defines what we will try to produce, we could build separate tooling around it that reads it in and parses it to produce a list of expected outputs. (And we could go back in the git history to figure out what we intended to produce at some date in the past.)

comment:15 Changed 3 months ago by ausil

pungi does not actually have everything. it does not define where the content ships, or what is to be shipped as a torrent etc. we need to have both the what we build and the where and how we ship it.

comment:16 Changed 3 months ago by acarter

  • Cc acarter added

comment:17 Changed 3 months ago by jkurik

The list of Release blocking deliverables [1] has been updated to my best knowledge what deliverables are requested for F24. Tickets [2][3] were created or emails sent[4][5][6] to WGs requesting review of the list.

I would appreciate if anyone from Release Engineering can do review of the page as well and let me know in case there will be any discrepancies.

[1]https://fedoraproject.org/wiki/Fedora_Program_Management/ReleaseBlocking/Fedora24
[2] Cloud WG
[3] Workstation WG
[4] Server WG
[5] Spins SIG
[6] Base WG

Last edited 3 months ago by jkurik (previous) (diff)
Note: See TracTickets for help on using tickets.