#2134 Password Policy causes segfault (ApacheDS & SSSD, CentOS 6.4)
Closed: Invalid None Opened 10 years ago by acidrainfall.

ApacheDS v2.0.0_M15

SSSD v1.9.2

CentOS 6.4

When I have a password policy enabled (which currently has no other options
enabled), SSSD dies at bind. Debug output gives segfault...

(Wed Oct 30 14:47:03 2013) [sssd[be[default]]] [sdap_set_config_options_with_rootdse] (0x0020): get_naming_context failed.
sssd_be: ../../../libraries/liblber/io.c:108: ber_write: Assertion `buf != ((void *)0)' failed.
(Wed Oct 30 14:47:03 2013) [sssd] [mt_svc_exit_handler] (0x0010): Process [default], definitely stopped!

Traceback reveals password expiration policy to be the problem, though the account and password were just created yesterday for testing.

#0  0x00007f07cec988a5 in raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
        resultvar = 0
        pid = <value optimized out>
        selftid = 3771
#1  0x00007f07cec9a085 in abort () at abort.c:92
        save_stage = 2
        act = {__sigaction_handler = {sa_handler = 0x7f07d0b75975, sa_sigaction = 0x7f07d0b75975}, sa_mask = {__val = {139671512009081, 140734663391424,
              140734663392116, 140734663391664, 139671511024790, 206158430232, 140734663391680, 140734663391456, 139671510930888, 206158430256,
              140734663391712, 11236112, 946176, 0, 139671543175541, 140734663399655}}, sa_flags = -824462739, sa_restorer = 0x7f07d0b76048}
        sigs = {__val = {32, 0 <repeats 15 times>}}
#2  0x00007f07cec91a1e in __assert_fail_base (fmt=<value optimized out>, assertion=0x7f07d0b75975 "buf != ((void *)0)",
    file=0x7f07d0b76048 "../../../libraries/liblber/io.c", line=<value optimized out>, function=<value optimized out>) at assert.c:96
        str = 0xab7310 ""
        total = 4096
#3  0x00007f07cec91ae0 in __assert_fail (assertion=0x7f07d0b75975 "buf != ((void *)0)", file=0x7f07d0b76048 "../../../libraries/liblber/io.c", line=108,
    function=0x7f07d0b76258 "ber_write") at assert.c:105
No locals.
#4  0x00007f07d0b724d5 in ber_write (ber=<value optimized out>, buf=0x0, len=0, zero=<value optimized out>) at ../../../libraries/liblber/io.c:108
        p = <value optimized out>
        __PRETTY_FUNCTION__ = "ber_write"
#5  0x00007f07d0b72556 in ber_init (bv=0xaa70a8) at ../../../libraries/liblber/io.c:362
        ber = 0xab8240
        __PRETTY_FUNCTION__ = "ber_init"
#6  0x00007f07d095748c in ldap_parse_passwordpolicy_control (ld=0xa0e500, ctrl=<value optimized out>, expirep=0x7fff579e7b78, gracep=0x7fff579e7b7c,
    errorp=0x7fff579e7b74) at ../../../libraries/libldap/ppolicy.c:138
        ber = <value optimized out>
        exp = -1
        grace = -1
        tag = <value optimized out>
        berLen = 140734663391944
        last = <value optimized out>
        err = 65535
        __PRETTY_FUNCTION__ = "ldap_parse_passwordpolicy_control"
#7  0x00007f07c66a8676 in simple_bind_done (op=<value optimized out>, reply=<value optimized out>, error=<value optimized out>, pvt=<value optimized out>)
    at src/providers/ldap/sdap_async_connection.c:582
        req = 0xab6fd0
        state = 0xab7150
        errmsg = 0x0
        nval = <value optimized out>
        ret = <value optimized out>
        lret = <value optimized out>
        response_controls = 0xab8bd0
        c = <value optimized out>
        pp_grace = 0
        pp_expire = 16
        pp_error = PP_passwordExpired
        __FUNCTION__ = "simple_bind_done"
#8  0x00007f07c66805b1 in sdap_process_message (ev=<value optimized out>, pvt=<value optimized out>) at src/providers/ldap/sdap_async.c:366
        msgtype = <value optimized out>
        ret = 0
        reply = 0xa0e4c0
        op = 0xab86c0
        msgid = <value optimized out>
#9  sdap_process_result (ev=<value optimized out>, pvt=<value optimized out>) at src/providers/ldap/sdap_async.c:209
        sh = <value optimized out>
        no_timeout = <error reading variable no_timeout (Cannot access memory at address 0x7fff579e7c10)>
        te = <value optimized out>
        msg = <error reading variable msg (Cannot access memory at address 0x7fff579e7c20)>
        ret = <error reading variable ret (Cannot access memory at address 0x7fff579e7c2c)>
        __FUNCTION__ = <error reading variable __FUNCTION__ (Cannot access memory at address 0x7f07c66f6580)>
Cannot access memory at address 0x7fff579e7c88}}}

Fields changed

milestone: NEEDS_TRIAGE => SSSD 1.11.3

If it turns to be an SSSD bug it might require cloning.

rhbz: => 0

Could you also upload core file?

I would like to know the exact version of some packages (sssd, openldap).

rpm -q openldap sssd

Packages:

openldap-2.4.23-32.el6_4.1.x86_64

sssd-1.9.2-82.10.el6_4.x86_64

As for core file - I don't know how to get you that. Do I use gdb?

Replying to [comment:4 acidrainfall]:

Packages:

openldap-2.4.23-32.el6_4.1.x86_64

sssd-1.9.2-82.10.el6_4.x86_64

As for core file - I don't know how to get you that. Do I use gdb?

Sorry for the late reply, I didn't realize Lukas was out of the office today and couldn't follow up.

I would say that the simplest way to generate a core file would be to install and enable abrt on RHEL/Centos. If that doesn't work for whatever reason, you can install sssd debuginfo packages (debuginfo-install sssd as root), attach to a running sssd_be process (gdb program pidof sssd_be), hit continue to resume execution and when SSSD crashes, save core file with generate-core-file at the gdb prompt.

Feel free to ping us for assistance.

Replying to [comment:5 jhrozek]:

Sorry for the late reply, I didn't realize Lukas was out of the office today and couldn't follow up.

No problemo.

I would say that the simplest way to generate a core file would be to install and enable abrt on RHEL/Centos. If that doesn't work for whatever reason, you can install sssd debuginfo packages (debuginfo-install sssd as root), attach to a running sssd_be process (gdb program pidof sssd_be), hit continue to resume execution and when SSSD crashes, save core file with generate-core-file at the gdb prompt.

I couldn't get ABRT to see the segfault, but gdb still did - I pulled the core dump that way.

File is too big for TRAC - linked: http://tinyurl.com/lmt7d44

Fields changed

cc: => lslebodn@redhat.com

It seems to be bug in openldap, I will try to describe it.

src/providers/ldap/sdap_async_connection.c
562         lret = ldap_parse_result(state->sh->ldap, state->reply->msg,
563                                 &state->result, NULL, &errmsg, NULL,
564                                 &response_controls, 0);
565         if (lret != LDAP_SUCCESS) {
566             DEBUG(SSSDBG_MINOR_FAILURE,
567                   ("ldap_parse_result failed (%d)\n", state->op->msgid));
568             ret = EIO;
569             goto done;
570         }
571
572         if (response_controls == NULL) {
573             DEBUG(SSSDBG_TRACE_LIBS, ("Server returned no controls.\n"));
574             state->ppolicy = NULL;
575         } else {
576             for (c = 0; response_controls[c] != NULL; c++) {
577                 DEBUG(SSSDBG_TRACE_INTERNAL,
578                       ("Server returned control [%s].\n",
579                        response_controls[c]->ldctl_oid));
580                 if (strcmp(response_controls[c]->ldctl_oid,
581                            LDAP_CONTROL_PASSWORDPOLICYRESPONSE) == 0) {
582                     lret = ldap_parse_passwordpolicy_control(state->sh->ldap,
583                                                             response_controls[c],
584                                                             &pp_expire, &pp_grace,
585                                                             &pp_error);

sssd calls openldap function ldap_parse_result to parse response from openldap (line 562). Variable response_controls contains just one response.

(gdb) p response_controls[0]
$5 = (LDAPControl *) 0x1377df0
(gdb) p *response_controls[0]
$6 = {ldctl_oid = 0x132bcd0 "1.3.6.1.4.1.42.2.27.8.5.1", ldctl_value = {bv_len = 0, bv_val = 0x0}, 
  ldctl_iscritical = 0 '\000'}
(gdb) p response_controls[1]
$7 = (LDAPControl *) 0x0

Constant LDAP_CONTROL_PASSWORDPOLICYRESPONSE has value "1.3.6.1.4.1.42.2.27.8.5.1", the same like in response_controls[0]->ldctl_oid. sssd passes this reponse directly to another openldap function ldap_parse_passwordpolicy_control without any change (line 582)(frame:6 from back trace).

The crash is caused by assert in the function ber_write, because function expects non NULL value of argument buf "buf != NULL" (frame:3 from back trace) and parameter buf is actually "response_controls[0]->ldctl_value.bv_val"

ldap_parse_passwordpolicy_control could not handle output from another openldap function ldap_parse_result. Newer version of openldap (2.4.35 or 2.4.37) has the same code, so it seems to be a new issue.

Fields changed

owner: somebody => lslebodn

Moving tickets that didn't make 1.11.3 to 1.11.4

milestone: SSSD 1.11.3 => SSSD 1.11.4

We're talking to the OpenLDAP maintainers about including the fix for the real issue. While this report is still being worked on, I don't think it should block the 1.11.4 releae, so I'm moving the ticket to 1.11.5

milestone: SSSD 1.11.4 => SSSD 1.11.5

I would like to mention there is a workaround how to prevent crash in sssd: to disable password policy on apacheds.

There is no problem in sssd. Problem was in the openldap and apacheds. ApacheDS sent wrong response with password policy and openldap could not parse malformed response.

Both problems are fixed in the upstream projects.

openldap ticket: http://www.openldap.org/its/index.cgi/Software%20Bugs?id=7759;page%3D5

ApacheDS ticket: https://issues.apache.org/jira/browse/DIRSERVER-1955

Fields changed

resolution: => invalid
status: new => closed

Metadata Update from @acidrainfall:
- Issue assigned to lslebodn
- Issue set to the milestone: SSSD 1.11.5

7 years ago

SSSD is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in SSSD's github repository.

This issue has been cloned to Github and is available here:
- https://github.com/SSSD/sssd/issues/3176

If you want to receive further updates on the issue, please navigate to the github issue
and click on subscribe button.

Thank you for understanding. We apologize for all inconvenience.

Login to comment on this ticket.

Metadata