Ticket #201 (closed task: fixed)

Opened 4 years ago

Last modified 3 years ago

Add mash support to depcheck

Reported by: wwoods Owned by: wwoods
Priority: major Milestone: 0.4.4
Component: tests Keywords:
Cc: lmacken, jskladan, jkeating Blocked By:
Blocking:

Description

To properly handle possible multilib problems (e.g. the nss-softokn problem in June and the recent gfortran problem) we need to run mash on the set of proposed packages, to generate a proper multilibified repo to test against.

Change History

comment:1 Changed 4 years ago by wwoods

  • Component changed from docs/wiki to tests

comment:2 Changed 4 years ago by lmacken

  • Cc lmacken added

comment:3 Changed 4 years ago by jskladan

  • Cc jskladan added

comment:4 Changed 4 years ago by jskladan

  • Owner set to jskladan
  • Status changed from new to assigned

comment:5 Changed 4 years ago by jskladan

So, I briefly went through the source code of mash (god damn the lack of documentation), and I must say, I'm not really enlightened or so :)

Correct me, if i'm wrong, but mash seems to be an utility, which is able to create a repo, based on kojitag.

From what i know, this is probably not _the thing_ we really need, or is it? Because my impression was, that we want to somehow check a small subset of packages.

BTW: maybe it would also be helpful for me to know, what exactly is multilib, and what is the aim of this test.

So, what do we expect mash to do for us? I've seen some "solve_multilib" routine inside the sources, so i guess that might be it, but it also seems to be working over the currently created repo.

comment:6 Changed 4 years ago by wwoods

So, what do we expect mash to do for us? I've seen some "solve_multilib" routine inside the sources, so i guess that might be it, but it also seems to be working over the currently created repo.

Right. The way I understand it, mash works kind of like createrepo except it takes a tag as input and - more importantly - does the multilib-solving magic.

In short:

createrepo: packages -> repodata
mash:       tag -> package list -> multilib solving -> repodata

Originally I talked to the rel-eng team about having a koji tag for "packages in bodhi that have requested push to {stable,testing}". Then we could run mash against that. And I'm pretty sure jkeating created some tags for this purpose.

But they also said we would need to run mash ourselves (i.e. on the autoqa systems) since it was too much strain to make koji/bodhi run mash for us. Furthermore, bodhi is not maintaining any "proposed-update" tags for us to use, so either:

  1. we figure out a way to run mash without a tag (e.g. using a list of build names instead),
  2. we update bodhi to maintain some special koji tags for us,
  3. we maintain the tags ourselves (e.g. in the watcher), or
  4. we roll our own code to generate repodata from a list of packages, and use Mash's 'solve_multilib' to make sure we ge get the full (multilib) set of packages.

comment:7 follow-up: ↓ 9 Changed 4 years ago by wwoods

  1. we update bodhi to maintain some special koji tags for us

lmacken has updated bodhi to handle the '-pending' tags for autoqa:

https://fedorahosted.org/bodhi/changeset/678384d0dae8228a28d2a3d17e03a091eb2d4d15

So, yay: once this goes live (later this week, I'm told) we can use mash as-is to set up a proper multilib repo for testing.

We'll still need some other method to handle the unittests though - either a set of special koji tags used only for testing, or some way of fooling mash into dealing with a bunch of packages that aren't in koji.

comment:8 Changed 4 years ago by jkeating

  • Cc jkeating added

http://fpaste.org/kGMu/ here is the mash config used to make f14 updates. What you want will be close to that.

comment:9 in reply to: ↑ 7 Changed 4 years ago by jskladan

Replying to wwoods:

We'll still need some other method to handle the unittests though - either a set of special koji tags used only for testing, or some way of fooling mash into dealing with a bunch of packages that aren't in koji.

Well, from what i've seen, i'm almost sure, that there is no _simple_ way to run mash out of koji. But I'll try to contact either Jesse Keating or Bill Nottingham (they seem to commit stuff to mash git a lot), and ask them.

comment:10 Changed 4 years ago by wwoods

As it turns out, the specific failure mentioned in the original ticket summary (the nss-softokn problem) can't be prevented by depcheck. The nss-softokn update was (technically) valid - at least as far as its dependencies were concerned - but any following (technically valid!) update that required nss-softokn.i686 would fail.

To prevent this specific problem we'd need something closer to rpmguard - I've filed ticket #222 to track that.

comment:11 Changed 4 years ago by jskladan

Finally made MASH run on a set of packages (yay for me :)) - if was quite simple at the end - just create fake repository with createrepo, and then force mash to run doMultilib() on it.

I'll provide more detailed information on the autoqa-devel list, once i polish my testing scripts a bit :)

comment:12 Changed 4 years ago by wwoods

Thanks to jskladan's mash code, we now have mash running in the unittests for depcheck. Yay!

I need to integrate this mash code and use it replace the createrepo code in depcheck, and then we should be done with this ticket.

comment:13 Changed 4 years ago by jlaska

  • Owner changed from jskladan to wwoods
  • Status changed from assigned to new

Based on previous comment, it looks like the remaining work is to integrate jskladan's mash script. Reassigning to wwoods for now.

comment:14 Changed 3 years ago by kparal

  • Milestone changed from Package Update Acceptance Test Plan - depcheck to 0.4.4

comment:15 Changed 3 years ago by wwoods

  • Resolution set to fixed
  • Status changed from new to closed

This code is believed to be complete in the 'depcheck' branch as of Nov. 18 (commit 61ac0276). Hooray!

Note: See TracTickets for help on using tickets.