Ticket #49 (closed enhancement: fixed)

Opened 7 years ago

Last modified 5 years ago

RFE: Allow hiding of download links for packages (Allow for EPEL builds)

Reported by: kevin Owned by: jkeating
Priority: major Milestone:
Component: misc Version: 1.2.2
Keywords: Cc: ausil, thl, rmyers, mmcgrath, mikeb, timj, quaid, tuju
Blocked By: Blocking:

Description

Currently koji must have all packages it uses imported and tagged. All these packages are available for downloading.

In order to build EPEL packages we need to import all the RHEL packages, which should not be available for general download, or be accessable except from a build.

koji should have some way of marking some packages (or tags?) as 'private' or 'embargoed', so there are no download links, but yet packages can build from them privately.

Other suggestions on how to allow EPEL builds would koji welcomed.

Attachments

upstream.diff (5.7 KB) - added by rmyers 6 years ago.
This patch attempts to provide a mechanism to add an "upstream" yum repository to the mock configuration used during builds. The upstream repository added to kojid.conf depends on the tag and the build architecture. The format of this option is: <TAG>-<ARCH>-upstream-repos. See the samples in kojid.conf for details.

Change History

comment:1 Changed 7 years ago by ausil

  • Cc ausil added

comment:2 Changed 7 years ago by thl

  • Cc thl added

comment:3 Changed 7 years ago by jkeating

I'm pretty sure that this is going to require Koji to grow the ability to use "upstream" repos as part of populating buildroots. These repos can be http repos that are accessible only to koji builders and not the rest of the world. This way we can continuously point to a RHEL repo set of packages that updates nightly as well as the built EPEL packages for building.

This is just one suggestion for handling this, one that would be useful in other situations as well.

comment:4 Changed 6 years ago by rmyers

  • Cc rmyers added

i think an "upstream" repo capability for Koji could be a great feature. and this would probably be the easiest way to allow EPEL to migrate to Koji.

moreover, i would love to see a Koji interface for all the RHEL packages. compared to Koji, searching through RHN for a particular package or build is horribly painful. does the idea of a Koji full of RHEL rpms (where standard subscription requirements still apply) interest anyone else?

comment:5 Changed 6 years ago by rmyers

upstream.diff may be a simple, if ugly, mechanism to add "upstream" repositories for certain tags. any thoughts about the approach?

Changed 6 years ago by rmyers

This patch attempts to provide a mechanism to add an "upstream" yum repository to the mock configuration used during builds. The upstream repository added to kojid.conf depends on the tag and the build architecture. The format of this option is: <TAG>-<ARCH>-upstream-repos. See the samples in kojid.conf for details.

comment:6 Changed 6 years ago by mmcgrath

  • Cc mmcgrath added

comment:7 follow-up: ↓ 8 Changed 6 years ago by jkeating

This patch is a good start in my opinion. However I think that putting per tag configuration in the conf file is the wrong place to have it. Ideally it would be extra configuration in the database along with the other configs for a tag, at least in my opinion.

comment:8 in reply to: ↑ 7 Changed 6 years ago by rmyers

Replying to jkeating:

This patch is a good start in my opinion.

cool.

Ideally it would be extra configuration in the database along with the other configs for a tag

i think that would be a cleaner implementation, too. when i get some time i'll see what i can come up with.

comment:9 follow-up: ↓ 10 Changed 6 years ago by mikeb

  • Cc mikeb added

I was asked my opinion about this patch, and so I am noting my opposition to it. This patch breaks the guarantees that Koji makes about tracking, auditability, and reproduceability of packages it builds. The list of buildroot contents will now be incomplete, with any number of unknown, untracked packages being silently included, and the web UI will essentially be lying about the build environment. There isn't even an indication in the database that an external repository was used. I think this significantly reduces the value that Koji brings to build system management. There are other solutions to the problem this ticket is trying to solve (hiding download links for certain packages). I strongly feel that this is the wrong solution.

comment:10 in reply to: ↑ 9 Changed 6 years ago by rmyers

Replying to mikeb:

This patch breaks the guarantees that Koji makes about tracking, auditability, and reproduceability of packages it builds.

Yes, but only for build targets that use this feature.

The list of buildroot contents will now be incomplete, with any number of unknown, untracked packages being silently included, and the web UI will essentially be lying about the build environment.

Yes, but only for build targets that use this feature.

There isn't even an indication in the database that an external repository was used.

That could be added.

There are other solutions to the problem this ticket is trying to solve (hiding download links for certain packages).

This may be a better approach, but it may also be more difficult to implement.

Should I try to refine the existing patch to address the concerns that have been identified, or should we skip the idea of building against non-koji repositories entirely?

IOW, is there consensus that building against non-koji repositories is bad, and that the correct approach is to prevent downloading of certain packages?

comment:11 Changed 6 years ago by rmyers

after talking with mikeb for a bit, i think he's right that a better solution to the RHEL binaries issue might be a separate koji/package hierarchy for non-distributable rpms.

jkeating: did you need the ability to build from upstream repos to solve another problem? are you ok with dropping the upstream repos concept and switching to a separate koji/package hierarchy, or are there two distinct problems to solve here?

comment:12 Changed 6 years ago by jkeating

I think there are two problems. There are still some use cases where we want to build from an upstream repo, with less tracking of what's in the buildroots.

comment:13 Changed 6 years ago by timj

  • Cc timj added

comment:14 Changed 5 years ago by quaid

  • Cc quaid added

comment:15 Changed 5 years ago by tuju

  • Cc tuju added

comment:16 Changed 5 years ago by ausil

  • Status changed from new to closed
  • Resolution set to fixed

EPEL is now fully building in koji. we implemented an external repo system that tracks what repo the buildroot contents comes from.

Note: See TracTickets for help on using tickets.