https://bugzilla.redhat.com/show_bug.cgi?id=812692 (Red Hat Enterprise Linux 6)
Description of problem: Due to checks in the installer related to IP addressing, IPA will not install on Amazon EC2. On Amazon EC2 virtual machines are provisioned with an IP address effectively situated behind network address translation. This leads to a situation where the public facing IP will never match up with the address of the interface, even to other machines which will be accessing IPA. The specific issue even occurs when trying to force the IP address that the server will use. How reproducible: 100% Steps to Reproduce: 1. Provision RHEL machine on Amazon EC2 2. yum -y install ipa-server 3. ipa-server-install Actual results: [root@ipa ~]# ipa-server-install ... Unexpected error - see ipaserver-install.log for details: No network interface matches the provided IP address and netmask [root@ipa ~]# ipa-server-install --ip-address=50.19.212.236 Usage: ipa-server-install [options] ipa-server-install: error: option --ip-address: invalid IP address 50.19.212.236: No network interface matches the provided IP address and netmask [root@ipa ~]# ifconfig eth0 eth0 Link encap:Ethernet HWaddr 12:31:3B:02:5C:3D inet addr:10.243.95.203 Bcast:10.243.95.255 Mask:255.255.254.0 inet6 addr: fe80::1031:3bff:fe02:5c3d/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:44356 errors:0 dropped:0 overruns:0 frame:0 TX packets:12170 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:61443561 (58.5 MiB) TX bytes:1553781 (1.4 MiB) Interrupt:8 Expected results: Working IPA install
Investigate how to do it in the NATed environment.
Testing environment can be set up with something like that:
Commands for "router" machine:
iptables -A POSTROUTING -t nat -s 10.0.0.3 -j SNAT --to 69.40.155.3 iptables -A PREROUTING -t nat -d 69.40.155.3 -j DNAT --to 10.0.0.3
Network arhitecture ASCII art:
internet ==== (ext) "router" machine (in) ==== server
<dpal> simo, I am installing on EC2 now. It works if you say --ip-address=internal-ip but then it uses the internal hostname. I do not think it is right. <dpal> I think it should use the external name <dpal> but the internal IP <dpal> And I suspect this is where the install would not work <simo> dpal: ok the point is that you can set whatever hostname you want <simo> the IP itslef is not a real issue, unless you try to set an arbitrary IP like the guy did try to <simo> dpal: just add in /etc/hosts private-ip desired-hostname <simo> and it will magically work <simo> of course then you may have issues with kerberos if DNS name -> ip does not match <dpal> But do you think this name will be usable form the outside? <simo> in general though I do not think you want to install freeipa in ec2 for "public" use ? <simo> so I would think actually just using private IPs only would be desirable but that's not up to me <simo> and I guess people may have distributed resources in multiple location so may really need to route stuff around <simo> dpal: the name itself can be usable from the outside but you may have to set up your own DNS server and play with views so that clients on your private LAN see the private IP while clients outside it see the public IP we do not support views in IPA though I think I've seen a ticket open about planning for that <dpal> if client not inside EC2 will it work? <simo> dpal: depends on how you set up the DNS <simo> but in general I think it would work <simo> if the client outside can resolve name -> public-ip it should just work of course I am not putting my hand on fire wrt SRV records but in general the problem would be mostly confined to DNS views the only exception I think may be password cahnges I think MIT fixed them over NAT only in 1.11 and I am not sure if the fix is needed only on the server side or also on the client side so pwd changes from the outside netowkr using kpasswd may fail however you can always use ldappasswd to perform password changes directly against ldap in that case <dpal> or use the UI when it is ready <simo> if we need to support that we may want a ticket in sssd to try to fallback to ldappasswd if kpasswd changes fail <simo> dpal: yes that would also be an appropriate fallback
I tested IPA server on Amazon EC2 VM, and it worked for me even when I used a custom hostname.
I set the custom hostname to /etc/hosts (IPA can do it too, but only if you pass IP address interactively and not via --ip-address - this is fixed in 6.3):
/etc/hosts
# cat /etc/hosts 127.0.0.1 localhost.localdomain localhost ::1 localhost6.localdomain6 localhost6 10.111.66.13 ipa.example.com ipa
And then run the installer where I passed the custom hostname and the internal IP address:
# ipa-server-install -p secret123 -a secret123 --hostname ipa.example.com --setup-dns --ip-address=10.111.66.13 The log file for this installation can be found in /var/log/ipaserver-install.log ============================================================================== This program will set up the IPA Server. This includes: * Configure a stand-alone CA (dogtag) for certificate management * Configure the Network Time Daemon (ntpd) * Create and configure an instance of Directory Server * Create and configure a Kerberos Key Distribution Center (KDC) * Configure Apache (httpd) * Configure DNS (bind) To accept the default shown in brackets, press the Enter key. Existing BIND configuration detected, overwrite? [no]: y Enter the fully qualified domain name of the computer on which you're setting up server software. Using the form <hostname>.<domainname> Example: master.example.com. Server host name [ipa.example.com]: Warning: skipping DNS resolution of host ipa.example.com The domain name has been calculated based on the host name. Please confirm the domain name [example.com]: The IPA Master Server will be configured with Hostname: ipa.example.com IP address: 10.111.66.13 Domain name: example.com The kerberos protocol requires a Realm name to be defined. This is typically the domain name converted to uppercase. Please provide a realm name [EXAMPLE.COM]: Do you want to configure DNS forwarders? [yes]: Enter the IP address of DNS forwarder to use, or press Enter to finish. Enter IP address for a DNS forwarder: 172.16.0.23 DNS forwarder 172.16.0.23 added Enter IP address for a DNS forwarder: Do you want to configure the reverse zone? [yes]: Please specify the reverse zone name [66.111.10.in-addr.arpa.]: Using reverse zone 66.111.10.in-addr.arpa. The following operations may take some minutes to complete. Please wait until the prompt is returned. Configuring ntpd [1/4]: stopping ntpd [2/4]: writing configuration [3/4]: configuring ntpd to start on boot [4/4]: starting ntpd done configuring ntpd. Configuring directory server for the CA: Estimated time 30 seconds [1/3]: creating directory server user [2/3]: creating directory server instance [3/3]: restarting directory server done configuring pkids. Configuring certificate server: Estimated time 3 minutes 30 seconds [1/17]: creating certificate server user [2/17]: creating pki-ca instance [3/17]: configuring certificate server instance [4/17]: disabling nonces [5/17]: creating CA agent PKCS#12 file in /root [6/17]: creating RA agent certificate database [7/17]: importing CA chain to RA certificate database [8/17]: fixing RA database permissions [9/17]: setting up signing cert profile [10/17]: set up CRL publishing [11/17]: set certificate subject base [12/17]: configuring certificate server to start on boot [13/17]: restarting certificate server [14/17]: requesting RA certificate from CA [15/17]: issuing RA agent certificate [16/17]: adding RA agent as a trusted user [17/17]: Configure HTTP to proxy connections done configuring pki-cad. Configuring directory server: Estimated time 1 minute [1/35]: creating directory server user [2/35]: creating directory server instance [3/35]: adding default schema [4/35]: enabling memberof plugin [5/35]: enabling referential integrity plugin [6/35]: enabling winsync plugin [7/35]: configuring replication version plugin [8/35]: enabling IPA enrollment plugin [9/35]: enabling ldapi [10/35]: configuring uniqueness plugin [11/35]: configuring uuid plugin [12/35]: configuring modrdn plugin [13/35]: enabling entryUSN plugin [14/35]: configuring lockout plugin [15/35]: creating indices [16/35]: configuring ssl for ds instance [17/35]: configuring certmap.conf [18/35]: configure autobind for root [19/35]: configure new location for managed entries [20/35]: restarting directory server [21/35]: adding default layout [22/35]: adding delegation layout [23/35]: adding replication acis [24/35]: creating container for managed entries [25/35]: configuring user private groups [26/35]: configuring netgroups from hostgroups [27/35]: creating default Sudo bind user [28/35]: creating default Auto Member layout [29/35]: creating default HBAC rule allow_all [30/35]: initializing group membership [31/35]: adding master entry [32/35]: configuring Posix uid/gid generation [33/35]: enabling compatibility plugin Restarting IPA to initialize updates before performing deletes: [1/2]: stopping directory server [2/2]: starting directory server done configuring dirsrv. [34/35]: tuning directory server [35/35]: configuring directory to start on boot done configuring dirsrv. Configuring Kerberos KDC: Estimated time 30 seconds [1/14]: setting KDC account password [2/14]: adding sasl mappings to the directory [3/14]: adding kerberos entries to the DS [4/14]: adding default ACIs [5/14]: configuring KDC [6/14]: adding default keytypes [7/14]: adding default password policy [8/14]: creating a keytab for the directory [9/14]: creating a keytab for the machine [10/14]: exporting the kadmin keytab [11/14]: adding the password extension to the directory [12/14]: adding the kerberos master key to the directory [13/14]: starting the KDC [14/14]: configuring KDC to start on boot done configuring krb5kdc. Configuring ipa_kpasswd [1/2]: starting ipa_kpasswd [2/2]: configuring ipa_kpasswd to start on boot done configuring ipa_kpasswd. Configuring the web interface: Estimated time 1 minute [1/13]: disabling mod_ssl in httpd [2/13]: setting mod_nss port to 443 [3/13]: setting mod_nss password file [4/13]: enabling mod_nss renegotiate [5/13]: adding URL rewriting rules [6/13]: configuring httpd [7/13]: setting up ssl [8/13]: setting up browser autoconfig [9/13]: publish CA cert [10/13]: creating a keytab for httpd [11/13]: configuring SELinux for httpd [12/13]: restarting httpd [13/13]: configuring httpd to start on boot done configuring httpd. Applying LDAP updates WARNING:root:remove: '(target = "ldap:///ipaentitlementid=*,cn=entitlements,cn=etc,dc=example,dc=com")(version 3.0;acl "permission:Register Entitlements";allow (add) groupdn = "ldap:///cn=Register Entitlements,cn=permissions,cn=pbac,dc=example,dc=com";)' not in aci WARNING:root:remove: '(targetattr = "usercertificate")(target = "ldap:///ipaentitlement=*,cn=entitlements,cn=etc,dc=example,dc=com")(version 3.0;acl "permission:Write Entitlements";allow (write) groupdn = "ldap:///cn=Write Entitlements,cn=permissions,cn=pbac,dc=example,dc=com";)' not in aci WARNING:root:remove: '(targetattr = "userpkcs12")(target = "ldap:///ipaentitlementid=*,cn=entitlements,cn=etc,dc=example,dc=com")(version 3.0;acl "permission:Read Entitlements";allow (read) groupdn = "ldap:///cn=Read Entitlements,cn=permissions,cn=pbac,dc=example,dc=com";)' not in aci WARNING:root:remove: '60' not in nsslapd-pluginPrecedence Restarting IPA to initialize updates before performing deletes: [1/2]: stopping directory server [2/2]: starting directory server done configuring dirsrv. Restarting the directory server Restarting the KDC Restarting the web server Configuring named: [1/9]: adding DNS container [2/9]: setting up our zone [3/9]: setting up reverse zone [4/9]: setting up our own record [5/9]: setting up kerberos principal [6/9]: setting up named.conf [7/9]: restarting named named service failed to start [8/9]: configuring named to start on boot [9/9]: changing resolv.conf to point to ourselves done configuring named. ============================================================================== Setup complete Next steps: 1. You must make sure these network ports are open: TCP Ports: * 80, 443: HTTP/HTTPS * 389, 636: LDAP/LDAPS * 88, 464: kerberos * 53: bind UDP Ports: * 88, 464: kerberos * 53: bind * 123: ntp 2. You can now obtain a kerberos ticket using the command: 'kinit admin' This ticket will allow you to use the IPA tools (e.g., ipa user-add) and the web user interface. Be sure to back up the CA certificate stored in /root/cacert.p12 This file is required to create replicas. The password for this file is the Directory Manager password
I was able to access at least WebUI from my machine outside of EC2 internal network using its public IP address.
Unfortunately, I could not test Kerberos login as it seems that only ports 80 and 443 are routed through:
# ipa-replica-conncheck -m ipa.example.com Check connection from replica to remote master 'ipa.example.com': Directory Service: Unsecure port (389): FAILED Directory Service: Secure port (636): FAILED Kerberos KDC: TCP (88): FAILED Kerberos Kpasswd: TCP (464): FAILED HTTP Server: Unsecure port (80): OK HTTP Server: Secure port (443): OK Port check failed! Inaccessible port(s): 389 (TCP), 636 (TCP), 88 (TCP), 464 (TCP)
I will close the upstream ticket as I did not find any bug during EC2 investigation.
Replying to [comment:4 mkosek]:
You probably need / want to define new security group and put your instance there -- in that group, open whatever ports you need to.
Metadata Update from @rcritten: - Issue assigned to mkosek - Issue set to the milestone: FreeIPA 3.0 Core Effort - 2012/04
I have made a implementation that works publicly and privately in Amazon EC2. I was using Red Hat 7.3 and the ipa-server 4.4 from the Red Hat repos. The trick was to have any given hosts' DNS name (such as ipa.example.com) resolve to a different IP depending on where the request was made. External requests should get the public IP, requests within the VPC (to include the IPA server itself) should get the private IP (allowing the install since the private IP is attached to an interface). As long as the same systems appear to have the same name every time Kerberos is happy and doesn't care about how many IP addresses it has.
I had to go to the AWS Route53 service (which is a managed DNS), I set up three hosted zones. 1. example.com (public hosted zone), this is your internet facing DNS zone (this doesn't need to be hosted on Route53, but can't be hosted on the IPA to my knowledge). Add an A record for your hosts using their publicly accessible IPs. 2. example.com (type: pivate hosted zone for Amazon VPC), this should have A records for your hosts using their private IPs (which defaults to the 172.31.0.0/18 range, but can be whatever you specify when you create the VPC). 3. reverse lookup for VPC (type: pivate hosted zone for Amazon VPC), if you are using the default VPC, this should be 31.172.in-addr.arpa. Add PTR records for your hosts (i.e. if your ipa server is 172.31.66.41 then create a PTR record for 41.66.31.172.in-addr.arpa to point to ipa.example.com).
This makes it so forward and reverse look-ups within the VPC resolve from/to the private IPs, and allows kerberos to work inside the VPC. All lookups outside of the VPC will resolve the same names to same hosts, but they'll be using the public IPs.
You'll still need to set the hostname on the server before the install. And edit the cloud-init so it doesn't change your hostname on reboot: printf "preserve-hostname: 1\n" | sudo tee /etc/cloud/cloud.cfg
You can then run ipa-server-install without an integrated DNS. Afterward, add the SRV records to your public and private DNS zones in Route 53, such as creating a record for "_kerberos._tcp.example.com" with the value "0 100 88 ipa.example.com" and whatever else is specified in the /tmp/ipa.system.records.blah.db file.
Login to comment on this ticket.