#2982 ipa-client-install Failed to obtain host TGT
Closed: Fixed None Opened 11 years ago by dpal.

https://bugzilla.redhat.com/show_bug.cgi?id=845691 (Red Hat Enterprise Linux 6)

Description of problem:

Occassionally ipa-client-install fails KRB pre-auth and rolls back like this:

# ipa-client-install  -U --domain=testrelm.com --realm=TESTRELM.COM -p admin -w
PASSWORD --server=qe-blade-13.testrelm.com

Failed to obtain host TGT.
Discovery was successful!
Hostname: qe-blade-04.testrelm.com
Realm: TESTRELM.COM
DNS Domain: testrelm.com
IPA Server: qe-blade-13.testrelm.com
BaseDN: dc=testrelm,dc=com


Synchronizing time with KDC...

Enrolled in IPA realm TESTRELM.COM
Created /etc/ipa/default.conf
New SSSD config will be created.
Configured /etc/sssd/sssd.conf
Configured /etc/krb5.conf for IPA realm TESTRELM.COM
Installation failed. Rolling back changes.

This is primarily being seen when trying to install clients (in automated test)
when the server topology is a branched tree.

Topology:

            M
           / \
          R1  R2
         /    /\
        R3   R4-R5

C1 - C6 are balances across M, R1 - R5 where:
C1 -> M
C2 -> R1
...
C6 -> R5

In this particular case, it was C4 -> R3 but, I've also seen C1 -> M fail the
same way.


Version-Release number of selected component (if applicable):
RHEL6.3 server
ipa-client-2.2.0-16.el6.x86_64
certmonger-0.56-1.el6.x86_64
krb5-workstation-1.9-33.el6.x86_64

How reproducible:
Unknown but is occurring noticably on automated tests.

Steps to Reproduce:
1.  <May need to build a similar server topology>
2.  ipa-client-install  -U --domain=$DOMAIN --realm=$RELM -p admin -w $ADMINPW
--server=$SERVER

Actual results:
error as above.

Expected results:
no error and client now joined to IPA domain.


Additional info:

/var/log/ipaclient-install.log:
...
2012-08-03T15:45:59Z DEBUG args=/usr/bin/kinit -k -t /etc/krb5.keytab
host/qe-blade-04.testrelm.com@TESTRELM.COM
2012-08-03T15:45:59Z DEBUG stdout=
2012-08-03T15:45:59Z DEBUG stderr=kinit: Preauthentication failed while getting
initial credentials

2012-08-03T15:45:59Z DEBUG Loading Index file from
'/var/lib/ipa/sysrestore/sysrestore.index'
2012-08-03T15:45:59Z DEBUG args=/usr/bin/certutil -L -d /etc/pki/nssdb -n IPA
CA
2012-08-03T15:45:59Z DEBUG stdout=Certificate:

 ...cert info here....let me know if needed.

2012-08-03T15:45:59Z DEBUG stderr=
2012-08-03T15:45:59Z DEBUG args=/usr/bin/certutil -D -d /etc/pki/nssdb -n IPA
CA
2012-08-03T15:45:59Z DEBUG stdout=
2012-08-03T15:45:59Z DEBUG stderr=
2012-08-03T15:45:59Z DEBUG args=/sbin/service messagebus start
2012-08-03T15:45:59Z DEBUG stdout=Starting system message bus:

2012-08-03T15:45:59Z DEBUG stderr=
2012-08-03T15:45:59Z DEBUG args=/sbin/service certmonger start
2012-08-03T15:45:59Z DEBUG stdout=
2012-08-03T15:45:59Z DEBUG stderr=
2012-08-03T15:45:59Z DEBUG args=/usr/bin/certutil -L -d /etc/pki/nssdb -n IPA
Machine Certificate - qe-blade-04.testrelm.com
2012-08-03T15:45:59Z DEBUG stdout=
2012-08-03T15:45:59Z DEBUG stderr=certutil: Could not find cert: IPA Machine
Certificate - qe-blade-04.testrelm.com
: File not found
...

Now it shows log entries for undoing config.

IPA SERVER httpd/access_log:

[root@qe-blade-13 log]# grep <IPADDRESS> httpd/access_log |cut -f2- -d' '
- - [03/Aug/2012:11:45:54 -0400] "GET /ipa/config/ca.crt HTTP/1.0" 200 1321 "-"
"Wget/1.12 (linux-gnu)"
- - [03/Aug/2012:11:45:54 -0400] "GET /ipa/config/ca.crt HTTP/1.0" 200 1321 "-"
"Wget/1.12 (linux-gnu)"
- - [03/Aug/2012:11:45:55 -0400] "POST /ipa/xml HTTP/1.1" 401 1856
- admin@TESTRELM.COM [03/Aug/2012:11:45:55 -0400] "POST /ipa/xml HTTP/1.1" 200
1925
- - [03/Aug/2012:11:46:01 -0400] "POST /ipa/xml HTTP/1.1" 401 1856
- host/qe-blade-04.testrelm.com@TESTRELM.COM [03/Aug/2012:11:46:01 -0400] "POST
/ipa/xml HTTP/1.1" 200 427

IPA SERVER httpd/error_log:

[Fri Aug 03 11:45:56 2012] [error] ipa: INFO: admin@TESTRELM.COM:
join(u'qe-blade-04.testrelm.com', nshardwareplatform=u'x86_64',
nsosversion=u'2.6.32-279.el6.x86_64'): SUCCESS
[Fri Aug 03 11:46:02 2012] [error] ipa: INFO:
host/qe-blade-04.testrelm.com@TESTRELM.COM:
host_disable(u'qe-blade-04.testrelm.com'): SUCCESS

Can you take a network trace while this happens ?
It would allow me to rule in/out an hypothesis I have about why this may be happening.

The cause seems to be that we don't always contact the same KDC, because it's looked up in DNS several times.

From Simo's mail:

We should probably use a fixed overriding krb5.conf during setup that
disables KJDC and Realm DNS resolution and has only the specific server
we are working against in the [realm] section.

Once the machine account is setup and sssd has the locator plugin
working then we can revert top the normal krb5.conf file.

It's due to the client using DNS to find a KDC that hasn't yet replicated info about the new client, and then asking it for a TGT. How to reproduce:

- Install a "master" and "replica"
- Change the Kerberos DNS entries to only point to the replica:
    for REC_NAME in '_kerberos-master._tcp' '_kerberos-master._udp' '_kerberos._tcp' '_kerberos._udp' '_kpasswd._tcp' '_kpasswd._udp'; do
        ipa dnsrecord-mod $DOMAIN $REC_NAME --srv-rec="0 100 88 $REPLICA_HOSTNAME"
    done
    ipa dnsrecord-mod $DOMAIN _ldap._tcp --srv-rec="0 100 389 $MASTER_HOSTNAME"
    ipa dnsrecord-find $DOMAIN  # check
- Sever communication between the hosts, disabling replication:
    on master:
    # iptables -A INPUT -j DROP -p all --source $REPLICA_IP
- On client machine, put master as nameserver in /etc/resolv.conf & install client

Additional change to fix new installs:

master: c87ac6b

ipa-3-0: a649d27

Additional fix to make ipa-client-install use private Kerberos CCACHE:

master: 79b90d1[[BR]]
ipa-3-0: 29a5d16

Metadata Update from @dpal:
- Issue assigned to pviktori
- Issue set to the milestone: FreeIPA 3.0 RC1

7 years ago

Login to comment on this ticket.

Metadata