Ticket #456 (closed defect: fixed)

Opened 4 years ago

Last modified 3 years ago

Proven tester metrics request

Reported by: adamwill Owned by: lmacken
Priority: major Milestone: Fixed, but not yet deployed
Component: bodhi-server Version:
Keywords: Cc: till
Blocked By: Blocking:

Description

Luke asked me to put this in trac, so here it is.

If possible I'd like to have some proven tester stats generated from Bodhi data and mailed to test list regularly. I think the two most important are:

  • top proven testers (by feedback left on critpath packages) * a list of critpath updates still requiring proventesters feedback (ordered perhaps by how long since they were submitted)

Thanks!

Luke gave a couple of alternatives for how to present this; my preference would be to have it as a separate mail, not part of the -devel bodhi report or the updates-testing mails, but honestly I haven't totally thought this through yet. Maybe the list of packages requiring testing belongs in the updates-testing mails, actually.

Change History

comment:1 Changed 4 years ago by till

  • Cc till added

For the list of untested critpath updates you could set up a cronjob that runs bodhi --untested -r F13 -s testing -l 1000, the only downside is that it is ordered with the oldest shown last.

comment:2 follow-up: ↓ 3 Changed 4 years ago by adamwill

That gives all updates, not just critpath. It doesn't seem like you can stack --critpath and --untested.

comment:3 in reply to: ↑ 2 ; follow-up: ↓ 4 Changed 4 years ago by lmacken

Replying to adamwill:

That gives all updates, not just critpath. It doesn't seem like you can stack --critpath and --untested.

The next release of bodhi will allow for this.

comment:4 in reply to: ↑ 3 Changed 4 years ago by till

Replying to lmacken:

Replying to adamwill:

That gives all updates, not just critpath. It doesn't seem like you can stack --critpath and --untested.

The next release of bodhi will allow for this.

The help text for "--untested" is wrong then: "Display a list of untested critical path updates"

comment:5 Changed 4 years ago by jdulaney

Maybe add a metric that shows amount of feedback a package has that doesn't have enough karma to pass, ie how many responses has that package got? If a package has twenty comments, and 0 or +1 Karma, maybe the package needs to be re-evaluated by the maintainer?

comment:6 Changed 4 years ago by lmacken

  • Owner set to lmacken
  • Status changed from new to assigned
  • Milestone changed from Wishlist to Fixed, but not yet deployed

I added a list of unapproved critical path updates to the top of the updates-testing digest mail in d478e974a06beca89112f056b998b5fd3674615b

As for proventester metrics, that is something that takes longer to generate at the moment. We'll tackle that in Bodhi v2.0.

comment:7 Changed 3 years ago by lmacken

  • Resolution set to fixed
  • Status changed from assigned to closed

Pushed to production

Note: See TracTickets for help on using tickets.