Ticket #1514 (closed defect: fixed)

Opened 20 months ago

Last modified 18 months ago

[abrt] sssd-1.8.4-13.fc16: __GI_exit: Process /usr/libexec/sssd/sssd_pam was killed by signal 6 (SIGABRT)

Reported by: jhrozek Owned by: pbrezina
Priority: major Milestone: SSSD 1.9.2
Component: SSSD Version:
Keywords: Cc:
Blocked By: Blocking:
Tests Updated: no Coverity Bug:
Patch Submitted: yes Red Hat Bugzilla: 851334, 857047
Design link:
Feature Milestone:
Design review: Fedora test page:
Chosen: Candidate to push out:
Release Notes:

Description

https://bugzilla.redhat.com/show_bug.cgi?id=851334 (Fedora)

libreport version: 2.0.10
abrt_version:   2.0.7
backtrace_rating: 4
cmdline:        /usr/libexec/sssd/sssd_pam --debug-to-files
crash_function: __GI_exit
executable:     /usr/libexec/sssd/sssd_pam
kernel:         3.4.9-1.fc16.x86_64
pid:            1341
pwd:            /
time:           Thu 23 Aug 2012 10:06:50 AM EDT
uid:            0
username:       root
var_log_messages: Aug 23 10:06:51 common1-nyc2 abrt[21652]: Saved core dump of
pid 1341 (/usr/libexec/sssd/sssd_pam) to
/var/spool/abrt/ccpp-2012-08-23-10:06:50-1341 (1335296 bytes)

backtrace:      Text file, 15362 bytes
maps:           Text file, 17952 bytes

build_ids:
:ad9b0193632c8cf2055f4d2a76c752360bfb06e9
:9efd824bf587def50099e2ae657a5cce012fa658
:c8696756e2d4d1455aeb54be1dc21af9525f4d78
:bb2eda4e80e107a3874f5cf7612166a1d81cd08b
:6cded60dd119fb8b8aa41b264e12931cb2c7b67f
:3a2c00058f6bc91272c8846d553d3834fab0f3c2
:3c551a8eb76f313e60299cad1e9a0f429968033a
:9181b91e3dd3b2786bf09199d2f89ec2a27d3652
:98710ef6382aabb845e5d732c12dfb9d7a26a9f7
:77ca13b377dc2850e7bde1c91e160c0f19421fb1
:a16442bf244219ec9db7918311806c2db8b8128f
:2b09b8bb44cdbcce3b7e6a786adb0c0ffb4a27b9
:d8d5e14ffd0e602ec48f16b257f574af44d08a9d
:b1532c8fcd1888fb7dc99186b3b8ec6875c72fa4
:3903219097bcd7fef6702a8fc24e50484083e23e
:c95e8094ddf00e9e03f98e4bd796b7c60a50d919
:b9746e6e639b060dcd2809a4c46655ee77838bf1
:8ea3a1b50dfbd33351111dac86cb8d6b6a73976f
:3ed9e0f5bbcd8d9d5b77f7dd4b562ec83d7ea767
:aa49b34d7e62383e4dc7195307c32462688d934f
:dd8106edd02ab9cb84bd426e3e1c24b2e96d7e20
:b52ecaab000fad35642f76aaf4396f4e7ae01c45
:6ba9409fb9f58785e255d94fc533fbf1d1fe35fd
:186430a109712d3f99c968fdb8d897a7aca2eb77
:f8f4d8e260ee8f3b262936920270ace01b2da537
:58803db420709341d8e6a713b18e54a6e1824272
:6a89975b8fa3aacafc6230189ead7d149806b520
:b069cf603f5827d380aeac72472ca9c661db1b33
:be507c791e34415e8f42f0e6030c889b2895cf9a
:4fc5fba2757ff2e60f581ecb4cc8bd9af5b83eff
:c2692dcb73a5877a23c9b3943bb4603ab19168de
:66d651684c3f89335cd989490c28489cc9da1fe5
:e814ef8432b7dec42a0b8ec12b9abd9fc7f57b40
:4415a32837ba7f07891f1a4c4e2f3b0ba2ee9271
:fc684ef0fc723dcb68a88799b8fb98e68e5f7441
:caeeab6968f72c5f76b50b0d94ee93db61784288
:82df68f406427a9efe3b2e41cec4374c1faf8dab
:6cfd35e0ead3e8ae8b000a42efe467c28762a08a
:5cb5f8da286abd58aeab3bd6676e661d52ec2b5d
:ee2f04900ae1f07517d91eba300ef385fccab1b8
:91b09f79dbe82783e58336f21fd4bae86c9f7389
:96be920f9f506a5bdcb7c4d7a8565b4f0791fc61

comment:
:Crash on restart, about 30 minutes after cron failed:
:
:Aug 23 09:25:01 common1-nyc2 /usr/sbin/crond[13334]: (TARGET-ACCOUNT) PAM
ERROR (Authentication service cannot retrieve authentication info)
:Aug 23 09:25:01 common1-nyc2 /usr/sbin/crond[13334]: (TARGET-ACCOUNT) FAILED
to authorize user with PAM (Authentication service cannot retrieve
authentication info)
:
:Aug 23 09:25:01 common1-nyc2 /usr/sbin/crond[13334]: pam_sss(crond:account):
Request to sssd failed. Timer expired

dso_list:
:/usr/lib64/liblber-2.4.so.2.7.1 openldap-2.4.26-8.fc16.x86_64 (Fedora Project)
1342557098
:/usr/lib64/libnssutil3.so nss-util-3.13.5-1.fc16.x86_64 (Fedora Project)
1345584126
:/usr/lib64/ldb/modules/ldb/paged_results.so libldb-1.1.0-1.fc16.x86_64 (Fedora
Project) 1337637200
:/lib64/libfreebl3.so nss-softokn-freebl-3.13.5-1.fc16.x86_64 (Fedora Project)
1345584111
:/usr/libexec/sssd/sssd_pam sssd-1.8.4-13.fc16.x86_64 (Fedora Project)
1341502371
:/usr/lib64/libsasl2.so.2.0.23 cyrus-sasl-lib-2.1.23-27.fc16.x86_64 (Fedora
Project) 1337637167
:/usr/lib64/libdhash.so.1.0.1 libdhash-0.4.3-5.fc16.x86_64 (Fedora Project)
1337637664
:/lib64/libresolv-2.14.90.so glibc-2.14.90-24.fc16.7.x86_64 (Fedora Project)
1345584124
:/usr/lib64/ldb/modules/ldb/server_sort.so libldb-1.1.0-1.fc16.x86_64 (Fedora
Project) 1337637200
:/usr/lib64/libtevent.so.0.9.15 libtevent-0.9.15-1.fc16.x86_64 (Fedora Project)
1337637188
:/usr/lib64/libssl3.so nss-3.13.5-1.fc16.x86_64 (Fedora Project) 1345584190
:/usr/lib64/libini_config.so.2.0.0 libini_config-0.6.2-5.fc16.x86_64 (Fedora
Project) 1337637490
:/lib64/libnspr4.so nspr-4.9.1-2.fc16.x86_64 (Fedora Project) 1345584125
:/lib64/libpthread-2.14.90.so glibc-2.14.90-24.fc16.7.x86_64 (Fedora Project)
1345584124
:/usr/lib64/libnss3.so nss-3.13.5-1.fc16.x86_64 (Fedora Project) 1345584190
:/lib64/libplc4.so nspr-4.9.1-2.fc16.x86_64 (Fedora Project) 1345584125
:/usr/lib64/libsmime3.so nss-3.13.5-1.fc16.x86_64 (Fedora Project) 1345584190
:/lib64/libcrypt-2.14.90.so glibc-2.14.90-24.fc16.7.x86_64 (Fedora Project)
1345584124
:/usr/lib64/libldap-2.4.so.2.7.1 openldap-2.4.26-8.fc16.x86_64 (Fedora Project)
1342557098
:/usr/lib64/libunistring.so.0.1.2 libunistring-0.9.3-5.fc16.x86_64 (Fedora
Project) 1345584142
:/usr/lib64/libtdb.so.1.2.9 libtdb-1.2.9-10.fc16.x86_64 (Fedora Project)
1337637128
:/lib64/libdbus-1.so.3.5.6 dbus-libs-1:1.4.10-3.fc16.x86_64 (Fedora Project)
1337637117
:/lib64/libplds4.so nspr-4.9.1-2.fc16.x86_64 (Fedora Project) 1345584125
:/lib64/libdl-2.14.90.so glibc-2.14.90-24.fc16.7.x86_64 (Fedora Project)
1345584124
:/usr/lib64/ldb/modules/ldb/skel.so libldb-1.1.0-1.fc16.x86_64 (Fedora Project)
1337637200
:/usr/lib64/ldb/modules/ldb/asq.so libldb-1.1.0-1.fc16.x86_64 (Fedora Project)
1337637200
:/usr/lib64/ldb/modules/ldb/tdb.so libldb-1.1.0-1.fc16.x86_64 (Fedora Project)
1337637200
:/lib64/libc-2.14.90.so glibc-2.14.90-24.fc16.7.x86_64 (Fedora Project)
1345584124
:/usr/lib64/libcollection.so.2.0.0 libcollection-0.6.1-5.fc16.x86_64 (Fedora
Project) 1337637197
:/lib64/librt-2.14.90.so glibc-2.14.90-24.fc16.7.x86_64 (Fedora Project)
1345584124
:/lib64/libpopt.so.0.0.0 popt-1.13-9.fc16.x86_64 (Fedora Project) 1337637118
:/usr/lib64/ldb/modules/ldb/memberof.so sssd-1.8.4-13.fc16.x86_64 (Fedora
Project) 1341502371
:/lib64/libz.so.1.2.5 zlib-1.2.5-6.fc16.x86_64 (Fedora Project) 1337637115
:/lib64/ld-2.14.90.so glibc-2.14.90-24.fc16.7.x86_64 (Fedora Project)
1345584124
:/usr/lib64/libtalloc.so.2.0.7 libtalloc-2.0.7-4.fc16.x86_64 (Fedora Project)
1337637131
:/usr/lib64/ldb/modules/ldb/rdn_name.so libldb-1.1.0-1.fc16.x86_64 (Fedora
Project) 1337637200
:/usr/lib/locale/locale-archive glibc-common-2.14.90-24.fc16.7.x86_64 (Fedora
Project) 1345584119
:/usr/lib64/libref_array.so.1.0.0 libref_array-0.1.2-5.fc16.x86_64 (Fedora
Project) 1337637487
:/usr/lib64/libldb.so.1.1.0 libldb-1.1.0-1.fc16.x86_64 (Fedora Project)
1337637200
:/usr/lib64/ldb/modules/ldb/sample.so libldb-1.1.0-1.fc16.x86_64 (Fedora
Project) 1337637200
:/usr/lib64/libpath_utils.so.1.0.0 libpath_utils-0.2.1-5.fc16.x86_64 (Fedora
Project) 1337637490
:/lib64/libpcre.so.0.0.1 pcre-8.12-9.fc16.x86_64 (Fedora Project) 1337637125
:/usr/lib64/ldb/modules/ldb/paged_searches.so libldb-1.1.0-1.fc16.x86_64
(Fedora Project) 1337637200

environ:
:SYSFONT=latarcyrheb-sun16
:PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
:PWD=/
:LANG=en_US.UTF-8
:KEYTABLE=us
:SHLVL=0
:BOOT_IMAGE=/vmlinuz-3.4.9-1.fc16.x86_64
:_SSS_LOOPS=NO
:KRB5RCACHEDIR=/var/cache/krb5rcache

Change History

comment:1 Changed 20 months ago by jhrozek

  • Tests Updated unset
  • tests set to 0
  • upgrade set to 0

Judging by the backtrace this looks like a memory hierarchy bug on responder shutdown, probably something similar to #1226.

comment:2 Changed 19 months ago by dpal

  • Milestone changed from NEEDS_TRIAGE to SSSD 1.9.1

comment:3 Changed 19 months ago by dpal

  • Red Hat Bugzilla changed from [https://bugzilla.redhat.com/show_bug.cgi?id=851334 851334] to [https://bugzilla.redhat.com/show_bug.cgi?id=851334 851334], [https://bugzilla.redhat.com/show_bug.cgi?id=857047 857047]

Ticket has been cloned to Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=857047

comment:4 Changed 19 months ago by pbrezina

  • Owner changed from somebody to pbrezina
  • Status changed from new to assigned

comment:5 Changed 19 months ago by pbrezina

Yes, it is happening during responder shutdown. This bug will be probably in all responders, not just pam.

The problem happens, when a responder is shutting down but there are still some unresolved dp request. Responder context and all subsequent context (like pam ctx, sudo ctx, ...) are freed along with dp request hash table. When the hash table is freed, it calls sss_dp_req_destructor(), which sets state to DP_ERR_FATAL and EIO, runs tevent_req_error(req, EIO) which calls the callback.

The problem is, that it doesn't indicate to the callback whether the error came from the provider or the responder. The callback just prints the error to log and tries to continue to serve data from sysdb. But the pam ctx is already freed which causes the crash.

comment:6 Changed 19 months ago by pbrezina

  • Patch Submitted set

comment:7 Changed 19 months ago by jhrozek

  • Milestone changed from SSSD 1.9.1 to SSSD 1.9.2

comment:8 Changed 18 months ago by jhrozek

  • Resolution set to fixed
  • Status changed from assigned to closed
Note: See TracTickets for help on using tickets.