"Option #3 is to use a new versioning scheme (ticket #1210) which will increment the version number for any source changes. This will also make Fedora and RHEL releases more consistent."

I do not believe that this is true for RHEL releases since they utilize a "patch" mechanism, and need to maintain the same "version" unless a "re-base" is approved; it is my understanding that this includes the entirety of the "version" number (not just the major:minor portions of it).

The new versioning scheme should be flexible enough to accommodate both Fedora and RHEL while at the same time forcing them to be more consistent. For example, here are the versions of the source code (not the versions of Fedora or RHEL packages):
- 1.0.0 - new major version
- 1.0.1 - fixes bug #1 with patch #1
- 1.0.2 - fixes bug #2 with patch #2
- 1.1.0 - new minor version introducing new features
Note that the source code doesn't have a concept of a release (4th) number.

On Fedora, suppose the package was initially created with source code version 1.0.0, the package will be called 1.0.0-1. When bug #1 is fixed, the package can be rebased with 1.0.1 source code and it will be called 1.0.1-1. When bug #2 is fixed, it can be rebased with 1.0.2 source code and it will be called 1.0.2-1. The Fedora package can be updated with patch mechanism too like on RHEL, but there is more flexibility to rebase so patches are optional.

On RHEL, suppose the package was initially based on Fedora 1.0.1-1, the RHEL package will be called 1.0.1-1 too. When bug #2 is fixed, if rebase is not allowed the patch #2 will be added and package will become 1.0.1-2 and it will be identical to Fedora 1.0.2-1. If rebase is allowed, RHEL package can be updated to 1.0.2-1.

Assuming the new versioning scheme is strictly followed, the patches will only contain bug fixes, not new features, so the difference between the above versions should be small, and there's probably no new upgrade scripts needed. New features should be introduced as a new major/minor version, not as a patch. RHEL will be able to pickup the new features in the next rebase opportunity.

Replying to [comment:2 edewata]:

The new versioning scheme should be flexible enough to accommodate both Fedora and RHEL while at the same time forcing them to be more consistent. For example, here are the versions of the source code (not the versions of Fedora or RHEL packages):
* 1.0.0 - new major version
* 1.0.1 - fixes bug #1 with patch #1
* 1.0.2 - fixes bug #2 with patch #2
* 1.1.0 - new minor version introducing new features
Note that the source code doesn't have a concept of a release (4th) number.

On Fedora, suppose the package was initially created with source code version 1.0.0, the package will be called 1.0.0-1. When bug #1 is fixed, the package can be rebased with 1.0.1 source code and it will be called 1.0.1-1. When bug #2 is fixed, it can be rebased with 1.0.2 source code and it will be called 1.0.2-1. The Fedora package can be updated with patch mechanism too like on RHEL, but there is more flexibility to rebase so patches are optional.

Question for clarity -- does your proposal presume that every single bug must be a new patch, or can a new patch address multiple bugs (this is the norm on other projects that I have worked on)?

Additionally, in your nomenclature above, does "bug" refer to a Bugzilla Bug, a PKI TRAC Ticket, or either?

On RHEL, suppose the package was initially based on Fedora 1.0.1-1, the RHEL package will be called 1.0.1-1 too. When bug #2 is fixed, if rebase is not allowed the patch #2 will be added and package will become 1.0.1-2 and it will be identical to Fedora 1.0.2-1. If rebase is allowed, RHEL package can be updated to 1.0.2-1.

This assumes a one-to-one correspondence between Fedora and RHEL. Not all Fedora bugs will necessarily go to RHEL (and vice-versa). Therefore, for this proposed versioning to work, are you proposing skipping numbers in RHEL? For example:

    Fedora:                     RHEL:
    1.0.1-1                     1.0.1-1
    1.0.2-1                     N/A
    1.0.3-1                     N/A
    1.0.4-1                     1.0.1-4

I have no issues with this, but it may cause confusion for support (especially if they have to create hot fixes - although a potential solution for this could be associated with the %{dist} tag) and some consternation with some customers.

Assuming the new versioning scheme is strictly followed, the patches will only contain bug fixes, not new features, so the difference between the above versions should be small, and there's probably no new upgrade scripts needed. New features should be introduced as a new major/minor version, not as a patch. RHEL will be able to pickup the new features in the next rebase opportunity.

Whatever the final decisions are on the versioning scheme, a well defined procedure of its usage and maintenance must be published to the Dogtag wiki.

A bug is an issue found in the code or small improvement needs to be made to the code. It can be tracked in Bugzilla, TRAC, or any other means. A single patch can fix multiple bugs. A bug can be fixed in multiple patches too. They can be split/combined in any way to make it easier to reapply the fix in other packages if needed. The maintenance number (3rd digit in the version number) corresponds to a collection of patches that fix a set of bugs, so it should be flexible enough.

It's true that not all Fedora fixes will go into RHEL immediately, but when a RHEL package is rebased, it will automatically pickup all Fedora fixes. RHEL fixes may not go into Fedora, but the RHEL release number doesn't need to be skipped. For example:

   Source:               Fedora: (source)          RHEL:   (source)
a) 1.0.0                 1.0.0-1 (1.0.0)           1.0.0-1 (1.0.0)
b)                       1.0.0-2 (1.0.0 + p1)
c)                                                 1.0.0-2 (1.0.0 + p2)
d) 1.0.1 (1.0.0 + p1)    1.0.1-1 (1.0.1)           1.0.0-3 (1.0.0 + p1 + p2)
e) 1.0.2 (1.0.1 + p3)    1.0.2-1 (1.0.2)           1.0.0-4 (1.0.0 + p1 + p2 + p3)
f)                                                 1.0.2-1 (1.0.2 + p2)

In (a) all start with 1.0.0 source code. In (b) a bug is found on Fedora and it's fixed with a patch. In (c) a different bug is found on RHEL and it's also patched. In (d) the Fedora patch is merged upstream, Fedora is rebased, RHEL is patched. In (d) a bug is found upstream, Fedora is rebased, and RHEL is patched. In (f) RHEL is rebased, but the RHEL patch is retained.

To reduce support issues the difference between Fedora and RHEL should be minimized. The new scheme should make it clearer which source a package is based on. And yes, it should be well-documented.

Per CS/DS Meeting of 11/24/2014 - 10.2.1

Additionally, per our discussions, I have an alternative approach to #1/#2 above which I believe will satisfy issues for both Fedora and RHEL as well as support both patches and re-bases.

My tweak suggests altering the "Source" URL paths since we control where these are published:

For example, option #1a would be to alter the "Source0" URL by adding both a %{version} and %{release} directory to the URL paths as follows:

Source0: http://pki.fedoraproject.org/pki/sources/%{name}/%{version}/%{release}/%{name}-%{version}%{?prerel}.tar.gz

This has the benefit of invisibly supporting a new tarball, or the same tarball (by merely providing the appropriate directories and a symlink inside the source repository to the previous tarball).

Additionally, this would require no code changes other than to our spec files (and minimal updates to the source repository whenever a release needs to be generated).

For the unofficial spec files utilized by developers, a slight variation (e. g. - #1b) on this theme could be utilized which distinguishes Fedora releases from RHEL releases (since RHEL releases will either always use patches or be version re-bases):

%if  0%{?rhel}
Source0: http://pki.fedoraproject.org/pki/sources/%{name}/%{version}/%{name}-%{version}%{?prerel}.tar.gz
%else
Source0: http://pki.fedoraproject.org/pki/sources/%{name}/%{version}/%{release}/%{name}-%{version}%{?prerel}.tar.gz
%endif

It should be noted that utilizing either of these approaches, we would continue to publish all patches to the 'http://pki.fedoraproject.org/pki/sources/%{name}/%{version}' directory since none of the build systems utilized actually pull their patches utilizing this URL, but rather store them within their own repositories.

I don't think the proposed alternative approach will satisfy the Fedora requirement because the tarball file name is still the same. When an SRPM is imported to Fedora, what matters is the file name of the tarball, the source URL of the tarball is irrelevant. As reported by mkosek, the recent 10.1.2-4 SRPM import replaced the old tarball and changed the checksum of the source.

-4c633f0636e3fa56bfbc5b6ab449bfac  pki-core-10.1.2.tar.gz    (from release 3)
+c4d28c12beb0095c66350a913484922e  pki-core-10.1.2.tar.gz    (from release 4)

So someone getting a pki-core-10.1.2.tar.gz will not know if it's coming from release 3 or 4. That distinction is important because the source code changes between releases (patches are merged into the source code).

The original option #1 proposed that each release will have its own tarball:

Source0: <source URL>/%{name}-%{version}-%{release}.tar.gz

The downside is it will consume more storage space.

The option #2 proposed to provide the download instruction instead of source URL. It needs to be corrected though, it needs to include the release number as well in the file name to avoid checksum issue.

# ... download instruction ...
Source0: %{name}-%{version}-%{release}.tar.gz

With option #3, Fedora will use the same patch mechanism as RHEL, except that Fedora has more flexibility to rebase.

Source0: <source URL>/%{name}-%{version}.tar.gz
Patch0: <patch>.patch

After numerous discussions, it was decided that we would utilize my suggestion in comment #5 above to address the immediate issue at hand (this may be altered at the point we begin utilizing a new versioning scheme):

Source0:          http://pki.fedoraproject.org/pki/sources/%{name}/%{name}-%{version}%{?prerel}.tar.gz

will be replaced by:

%if 0%{?rhel}
# NOTE:  In the future, as a part of its path, this URL will contain a release
#        directory which consists of the fixed number of the upstream release
#        upon which this tarball was originally based.
Source0:          http://pki.fedoraproject.org/pki/sources/%{name}/%{version}/%{
release}/rhel/%{name}-%{version}%{?prerel}.tar.gz
%else
Source0:          http://pki.fedoraproject.org/pki/sources/%{name}/%{version}/%{
release}/%{name}-%{version}%{?prerel}.tar.gz
%endif

This change will be applied to the next builds of the following branches:

• master
  • pki-core-10.2.1-0.2.fc22
  • dogtag-pki-theme-10.2.1-0.2.fc22
  • pki-console-10.2.1-0.2.fc22
  • dogtag-pki-10.2.1-0.2.fc22
• DOGTAG_10_2_RHEL_BRANCH
  • pki-core-10.2.0-5.el7
  • dogtag-pki-theme-10.2.0-5.el7
  • pki-console-10.2.0-5.el7
  • dogtag-pki-10.2.0-5.el7
• DOGTAG_10_2_0_BRANCH
  • pki-core-10.2.0-5.fc21
  • dogtag-pki-theme-10.2.0-5.fc21
  • pki-console-10.2.0-5.fc21
  • dogtag-pki-10.2.0-5.fc21
• DOGTAG_10_1_BRANCH
  • pki-core-10.1.2-7.fc20
  • pki-console-10.1.2-7.fc20
  • dogtag-pki-10.1.2-7.fc20

NOTE:  The RHEL Source URL needed to remain unchanged for
       the DOGTAG_10_1_BRANCH since it is currently in
       maintenance mode and is only altered by patches.

Ticket has been cloned to Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1169981

Checked into master:

• 516eebe9e0b3ced48103ffb6e29987bc49b3f2a5

Checked into DOGTAG_10_1_BRANCH:

• c83a91f611a5cd6610c5fbf010507fdc0e0dffae

Checked into DOGTAG_10_2_BRANCH:

• 2a489ce060cc3604cc8d8c5f77b736895df39517

Checked into DOGTAG_10_2_RHEL_BRANCH:

• 1a408dc023c7032b0bcb28fd4b04b232f43d4f9d

Update compose_functions development script to account for remote tarballs and patches

Checked into master:

• 69640a184ab10d78d57d5c3cd235eefc752bb859

Checked into DOGTAG_10_2_0_BRANCH:

• 220d18ac19d72769cc1ef5f6a08e3504a18bde4c

Checked into DOGTAG_10_2_1_BRANCH:

• e382040c4e170888abb92a6e4c31394cba45278f

Checked into Dogtag_10_2_RHEL_BRANCH:

• 9f9cbb75da1ee72fc8a23e7b3db57b19126489d0

Dogtag PKI is moving from Pagure issues to GitHub issues. This means that existing or new
issues will be reported and tracked through Dogtag PKI's GitHub Issue tracker.

This issue has been cloned to GitHub and is available here:
https://github.com/dogtagpki/pki/issues/1773

If you want to receive further updates on the issue, please navigate to the
GitHub issue and click on Subscribe button.

Thank you for understanding, and we apologize for any inconvenience.