Ticket #277 (new task)

Opened 2 years ago

Last modified 23 months ago

Add ARM to release criteria / validation process somehow

Reported by: adamwill Owned by: adamwill
Priority: critical Milestone: Fedora 18
Component: Wiki Version:
Keywords: arm Cc: tflink, pbrobinson, jcm, jdulaney, wmmartin@…
Blocked By: Blocking:

Description

phenomenon

ARM is becoming increasingly important to releases. Currently we have no kind of formal validation process for ARM images. Let's work with the ARM group to improve this, so we can test whatever is most important to them.

background analysis

ARM deployment is significantly different from x86: the standard way to do things is to provide a pre-built image to dump onto the system, not to 'install' via anaconda. So a lot of our test suite, which is based around anaconda, may not be entirely appropriate to ARM. Similarly, the desktop validation stuff isn't likely to be super-interesting to them, as desktop isn't currently a priority for ARM and things at that high a level ought to pretty much 'just work'. We should figure out with the ARM team what it would be most useful to test, and how to work testing into their release process.

implementation recommendation

We may need to write a few new tests, then I think we should design an ARM-specific validation matrix, which may be much smaller than the main ones. To start with we may also want to start separate ARM release criteria pages, to codify the things which are most important to ARM releases.

Change History

comment:1 Changed 2 years ago by wmmartin

  • Cc wmmartin@… added

comment:2 Changed 2 years ago by jwboyer

Do we have similar criteria for s390x and PowerPC? If not, I would suggest that either:

1) You wait until a proposal for promotion TO a primary architecture is worked out and incorporate whatever comes from that into the criteria

or

2) Draft a general set of "Secondary Architecture" criteria with a smaller (possibly optional) subset of things for architecture specific stuff (e.g. U-Boot in ARM vs. Yaboot in PowerPC).

comment:3 follow-up: ↓ 4 Changed 2 years ago by adamwill

"Do we have similar criteria for s390x and PowerPC?"

No.

"1) You wait until a proposal for promotion TO a primary architecture is worked out and incorporate whatever comes from that into the criteria"

That's obviously possible, but I thought it would be nicer to try and ramp up *some* kind of testing for ARM before we get to that point, given that ARM is obviously a key area already.

"2) Draft a general set of "Secondary Architecture" criteria with a smaller (possibly optional) subset of things for architecture specific stuff (e.g. U-Boot in ARM vs. Yaboot in PowerPC). "

James Laska started such an effort prior to leaving the QA team, but he never completed it. Honestly, I consider ARM substantially more important than PPC or s390x, and am trying to keep things simple by proposing we just deal with ARM rather than complicate matters with the other, less important secondary arches.

comment:4 in reply to: ↑ 3 Changed 2 years ago by jwboyer

Replying to adamwill:

James Laska started such an effort prior to leaving the QA team, but he never completed it. Honestly, I consider ARM substantially more important than PPC or s390x, and am trying to keep things simple by proposing we just deal with ARM rather than complicate matters with the other, less important secondary arches.

Which is kind of a slap in the face, given that s390x and PPC are the only two secondary architectures that have 1) actually done a release in the past 2 or 3 years, 2) are the only two secondary architectures that are actually building for _current_ releases, and 3) are certainly no less relevant than ARM depending on your perspective.

ARM is great, I'm sure ARM64 will be awesome, but I do think you're doing a disservice by focusing on something that doesn't exist instead of working with the teams that have already put forth great effort to participate in Fedora.

comment:5 Changed 2 years ago by adamwill

I'm trying to do _something_ constructive here. I'm focused more on 'let's get something going on ARM', I am not thinking 'boy, PPC / s390 suck, let's insult them'. I'm sorry if it looks that way, but I'd just like to follow through on the work already proposed with a bunch of ARM folks. We are not making the situation wrt any other arch any worse than it currently is.

Perspective is always a tricky thing, but ARM is a clearly growing arch with a huge amount of impetus behind it and there seems to be a broad consensus it will become a primary arch soon. This seems to me clearly not to be the case for PPC or s390, neither of which seem likely to grow explosively and more or less take over the end user device market any time soon.

comment:6 Changed 2 years ago by jdulaney

  1. Do we define criteria for successful installs considering how much variation there is out there?
  1. We can probably go through the criteria and get a good core.
  1. We could specify x86-only and ARM-only and keep all the criteria together.
  1. Do we actually *want* to ignore graphical desktop criteria? Some systems do support graphics rather well.

comment:7 Changed 2 years ago by adamwill

we don't need to *ignore* the desktop, but it shouldn't be the primary testing that gets done ahead of all other testing, i don't think.

comment:8 Changed 23 months ago by adamwill

  • Milestone changed from Fedora 17 to Fedora 18

Not done yet, so push out to F18.

Note: See TracTickets for help on using tickets.