#49089 Parallel build failures
Closed: wontfix None Opened 7 years ago by firstyear.

Using make -j8 install, the build fails:

libtool: install: /usr/bin/install -c ldif-bin /opt/dirsrv/bin/ldif-bin
libtool: install: /usr/bin/install -c .libs/ns-slapd /opt/dirsrv/sbin/ns-slapd
libtool: install: /usr/bin/install -c ldap-agent-bin /opt/dirsrv/sbin/ldap-agent-bin
libtool: install: /usr/bin/install -c .libs/migratecred-bin /opt/dirsrv/bin/migratecred-bin
libtool: install: /usr/bin/install -c .libs/mmldif-bin /opt/dirsrv/bin/mmldif-bin
libtool: install: /usr/bin/install -c .libs/pwdhash-bin /opt/dirsrv/bin/pwdhash-bin
libtool: install: /usr/bin/install -c rsearch-bin /opt/dirsrv/bin/rsearch-bin
/home/william/build/ds/.libs/libns-dshttpd.so: file not recognized: File truncated
collect2: error: ld returned 1 exit status
libtool: install: error: relink `libacl-plugin.la' with the above command before installing it
make[2]: *** [install-serverpluginLTLIBRARIES] Error 1
make[2]: *** Waiting for unfinished jobs....
libtool: install: /usr/bin/install -c .libs/libns-dshttpd.so.0.0.0T /opt/dirsrv/lib/dirsrv/libns-dshttpd.so.0.0.0
libtool: install: (cd /opt/dirsrv/lib/dirsrv && { ln -s -f libns-dshttpd.so.0.0.0 libns-dshttpd.so.0 || { rm -f libns-dshttpd.so.0 && ln -s libns-dshttpd.so.0.0.0 libns-dshttpd.so.0; }; })
libtool: install: (cd /opt/dirsrv/lib/dirsrv && { ln -s -f libns-dshttpd.so.0.0.0 libns-dshttpd.so || { rm -f libns-dshttpd.so && ln -s libns-dshttpd.so.0.0.0 libns-dshttpd.so; }; })
libtool: install: /usr/bin/install -c .libs/libns-dshttpd.lai /opt/dirsrv/lib/dirsrv/libns-dshttpd.la
libtool: finish: PATH="/sbin:/bin:/usr/sbin:/usr/bin:/sbin" ldconfig -n /opt/dirsrv/lib/dirsrv

I do not think that problem is related to CXXLINK.

the problem is that different library ''libns-dshttpd.so'' is empty at linking of ''libacl-plugin.la''. And ''libns-dshttpd.so'' is dependency of ''libacl-plugin.so''

BTW; the reason why the ''libns-dshttpd.so'' is empty is that it was created a little bit later than ''libacl-plugin.so''

{{{
/home/william/build/ds/.libs/libns-dshttpd.so: file not recognized: File truncated
collect2: error: ld returned 1 exit status
libtool: install: error: relink `libacl-plugin.la' with the above command before installing it
make[2]: [install-serverpluginLTLIBRARIES] Error 1
make[2]:
Waiting for unfinished jobs....
}}}

The only reasonable solution is to build package in parallel but install with single thread
{{{
make -j8
make -j8 check
make install -j1
}}}

Here is a related automake BZ
https://bugzilla.redhat.com/show_bug.cgi?id=1260190

IIRC, the install dependencies are properly handled if they are in the same directory. In another words in the same variable ''$name_LTLIBRARIES''. But ''libaddn-plugin.so'' is installed into ''serverplugindir'' and ''libns-dshttpd.so'' into ''serverdir''

Sorry about my basic question...

When I acked, I tried on my laptop:
make install -j8
as well as even
make install -j32
with the William's patch 0001-Ticket-49089-Fix-invalid-cxxlink-statement-from-hpux.patch​ to see both successful. (I did not try without the patch, though... Probably, it'd be successful, too?)

To duplicate the problem, do we need some specific build env such as a host with lots of CPUs?

Thanks.

It is a race condition and you never know when you hit it.

IMHO it's fine to remove/comment out lines related to HPUX linking issues. Especially, if this platform is not tested.

But It will not completely fix the problem; because race condition is still there. It might just for some reason decrease probability of race condition between linking ''libaddn-plugin.so'' and ''libaddn-plugin.so''.

BTW this is a reason why there is '''emake -j1 install''' in sssd portage[1]. We hit the same problem.
And we hit it more often due to our internal dependencies.

[1] https://gitweb.gentoo.org/repo/gentoo.git/tree/sys-auth/sssd/sssd-1.13.1.ebuild#n173

It's up to you whether you will push patch or no. Feel free to add ACK back. But it would be good to update the commit message because the failure is not caused by '''CXXLINK'''

So I've updated the comment, but I think I'll also update the ebuild like you say. If it's a known issue, not much we can really do about it :(

commit a694a4c
Writing objects: 100% (13/13), 5.46 KiB | 0 bytes/s, done.
Total 13 (delta 10), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
f0fdc4d..28e5e15 master -> master

commit 9b702ed
Writing objects: 100% (3/3), 1.12 KiB | 0 bytes/s, done.
Total 3 (delta 2), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
9fcfb6c..9b702ed master -> master

This patch helps a lot, but it's still not perfect: The fault is still in autotools, so there is a gentoo ebuild patch that goes with this.

Metadata Update from @firstyear:
- Issue assigned to firstyear
- Issue set to the milestone: 1.3.6 backlog

7 years ago

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/389ds/389-ds-base/issues/2148

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Metadata Update from @spichugi:
- Issue close_status updated to: wontfix (was: Fixed)

3 years ago

Login to comment on this ticket.

Metadata