Wednesday, April 8, 2015

Open AM Realms and Authentication Chains

Wow. Two posts in one day. I've had this one ready to publish for a while and wanted to get it out there before I forget everything.

As I refine the RADIUS support being added to Open AM I've run across a few items that I want to preserve so that I don't loose them. I hope they are of benefit to others.

Realms

Open AM provides the concept of Realms. In a newly installed instance of Open AM there is one realm named '/' known as the root realm. Any number of realms can be defined and are conceptually presented in the admin console as being children of the root realm as shown below.


Authentication Chains

In OpenAM Users can only authenticate through another OpenAM construct called an authentication chain. An authentication chain is defined in a realm on the Authentication tab for that realm. A chain has a name and from one to many Authentication Module instances placed within it. These module instances must be created before they can be embedded in any chain. A single module instance can be embedded within multiple chains if desired since these instances are actually instances of configuration. When actually used for authentication, the underlying Pluggable Authentication Module class for the authentication module is instantiated, given a copy of this instance configuration, and is allowed to contribute its authentication behavior to the authentication experience provided to the user by the chain.

Modules in the chain have order meaning users must pass the requirements for an earlier module in the chain before incurring the requirements for modules later in the chain. This order is presented vertically in the admin console as shown below. Authentication by a user proceeds through the modules in the chain from top to bottom.



If any authentication chains exist in the top level realm when a new realm is created, those chains and correspondingly their module configuration instances will be copied into the new realm. If they are deleted from that new realm or any configuration changes are made to the configuration instances in that other realm it has no impact on those in the root realm.

LDAP Constructs

Interestingly, from an ldap perspective, the OpenAM DN at which OpenAM is installed (such as dc=openam,dc=forgerock,dc=org), has an ou=services child node that holds a number of child nodes mostly for the root realm. For example, in OpenAM version 11.0.0 the authentication chains defined for that realm show up in ou=services, ou=iPlanetAMAuthConfiguration, ou=1.0, ou=OrganizationConfig, ou=default, ou=Configurations with nodes therein each representing a given chain by name:


Any new realms show up as child nodes of the base ou=services node as well. And within each of those there is an ou=services node that has a subset of the top level ou=services nodes children copied within. Note the entries for the "again", "another", and "yet-another" in the image below corresponding to the realms listed in the first image above. And if you look in the corresponding path noted above for authentication chains you'll find the chains for that realm. Hence why changing config for a chain in a newly created realm does not affect the configuration that was copied from the root realm.



Programmatically Listing Realms and Their Chains

The set of defined realms that are currently defined in OpenAM can be acquired by the following code:

   SSOToken admTk = (SSOToken) AccessController.doPrivileged(
     AdminTokenAction.getInstance());

   // see what realms we have available to us
   OrganizationConfigManager ocm = null;
   try {
    ocm = new OrganizationConfigManager(admTk, "/");
   } catch (SMSException e) {
    cLog.log(Level.SEVERE, "----- Unable to get realms", e);
   }
   Set<String> realms = null;
   if (ocm != null) {
    realms = CollectionUtils.asOrderedSet("/");
    try {
     realms.addAll(ocm.getSubOrganizationNames("*", true));
     cLog.log(Level.INFO, "---->>> realms: " + realms);
    } catch (SMSException e) {
     cLog.log(Level.SEVERE, "----- Unable to get sub-realms", e);
    }
   }

Once I have the set of realm names I can loop through each and acquire the set of defined chains by name with the following code:

if (realms != null) {
 ConfiguredAuthServices cas = new ConfiguredAuthServices();

 for(String realm : realms) {
  Map parms = new HashMap();
  parms.put(Constants.ORGANIZATION_NAME, realm);
  parms.put(Constants.SSO_TOKEN, admTk);
  Map vals = cas.getChoiceValues(parms);
  cLog.log(Level.INFO, "--->>> '" + realm + "' chains: " + vals);
 }
}

The output of these lines is as shown below with highlighting added for clarity and the timestamp and class name and other cruft trimmed from the front of the lines. I can see the realms as shown in the first image above. I can see the chains of the root realm reflected in some of the other realms. Where they are not found is where I went into that realm and removed that chain. And you can see where I've added new chains to a given realm. Those Empty chains are a concern.

realms: [/, another, again, yet-another, more]
chains for realm /: {smsotp=smsotp, [Empty]=[Empty], ldapService=ldapService, local-radius-1815-server=local-radius-1815-server}
chains for realm another: {smsotp=smsotp, [Empty]=[Empty], ldapService=ldapService, local-radius-1815-server=local-radius-1815-server}
chains for realm again: {smsotp=smsotp, [Empty]=[Empty], againChain2=againChain2, ldapService=ldapService, againChain1=againChain1, againChain3=againChain3, local-radius-1815-server=local-radius-1815-server}
chains for realm yet-another: {smsotp=smsotp, [Empty]=[Empty], ldapService=ldapService, yetAnotherChain=yetAnotherChain, local-radius-1815-server=local-radius-1815-server}
chains for realm more: {smsotp=smsotp, testOne=testOne, [Empty]=[Empty], ldapService=ldapService, local-radius-1815-server=local-radius-1815-server}

At some point I hope to modify the current RADIUS Server config page to provide a dropdown of al l realm/chain combinations from which to select a value rather than require users to enter the realm and chain names as outlined in a previous blog.  These will be very useful notes to have whenever I get time to polish off that RADIUS support.

Enjoy.





Fixing Docker VPN Incompatibilities on OSX

I currently use a Macbook Pro as my dev box. And I have Cisco's AnyConnect for VPN access. From time to time I've noticed that I get into a state where I can't access my docker container running in boot2docker. For example I'll get the following behavior:

$ docker images
FATA[0032] An error occurred trying to connect: Get https://192.168.59.103:2376/v1.17/images/json: dial tcp 192.168.59.103:2376: i/o timeout


When in this state boot2docker works just fine. I can start it, stop it, and check its status just fine:

$ boot2docker status
running

But docker is nowhere to be found including contacting the virtual box host IP address which gives the biggest clue as to what is causing this problem:

$ boot2docker ip
192.168.59.103

$ ping 192.168.59.103
PING 192.168.59.103 (192.168.59.103): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2

Turns out that one VPN profile that I use, significantly alters the routing table entries. To see this, I captured my routing table contents after a fresh hard restart of the Mac. Here is the output with things that might be related showing:

$ netstat -rn
Routing tables

Internet:
Destination  Gateway       Flags   Refs    Use     Netif Expire
default      10.109.16.1   UGSc      90      0     en8
default      10.109.128.1  UGScI      2      0     en0
...
127          127.0.0.1     UCS        0      0     lo0
127.0.0.1    127.0.0.1     UH       135   7682     lo0
...

By capturing this output to a text file then starting boot2docker and capturing it again in a separate text file and perform a diff between them the only changes after factoring out changes in Refs, Use, and Expire is this line that has been added to the bottom of the table:

Destination  Gateway       Flags   Refs    Use   Netif Expire
...
192.168.59   link#14       UC         1      0 vboxnet

Depending on if you have run any docker commands after running boot2docker up or boot2docker start you may have a "host route" in there as well looking something like this:

Destination    Gateway         Flags  Refs   Use   Netif Expire
...
192.168.59.103 8:0:27:ac:51:8b UHLWIi    1    40 vboxnet   1197

If you haven't done much with networking issues, like me, pull up the man page for the netstat -r output to grasp what is going on here. It indicates:

The routing table display indicates the available routes and their status.  Each route consists of a destination host or network and a gateway to use in forwarding packets.  The flags field shows a collection of information about the route stored as binary choices.  The individual flags are discussed in more detail in the route(8) and route(4) manual pages.  The mapping between letters and flags is:

1  RTF_PROTO1     Protocol specific routing flag #1
2  RTF_PROTO2     Protocol specific routing flag #2
3  RTF_PROTO3     Protocol specific routing flag #3
B  RTF_BLACKHOLE  Just discard packets (during updates)
b  RTF_BROADCAST  The route represents a broadcast address
C  RTF_CLONING    Generate new routes on use
c  RTF_PRCLONING  Protocol-specified generate new routes on use
D  RTF_DYNAMIC    Created dynamically (by redirect)
G  RTF_GATEWAY    Destination requires forwarding by intermediary
H  RTF_HOST       Host entry (net otherwise)
I  RTF_IFSCOPE    Route is associated with an interface scope
i  RTF_IFREF      Route is holding a reference to the interface
L  RTF_LLINFO     Valid protocol to link address translation
M  RTF_MODIFIED   Modified dynamically (by redirect)
m  RTF_MULTICAST  The route represents a multicast address
R  RTF_REJECT     Host or net unreachable
r  RTF_ROUTER     Host is a default router
S  RTF_STATIC     Manually added
U  RTF_UP         Route usable
W  RTF_WASCLONED  Route was generated as a result of cloning
X  RTF_XRESOLVE   External daemon translates proto to link address
Y  RTF_PROXY      Proxying; cloned routes will not be scoped

     Direct routes are created for each interface attached to the local host; the gateway field for such entries shows the address of the outgoing interface.  The refcnt field gives the current number of active uses of the route.  Connection oriented protocols normally hold on to a single route for the duration of a connection while connectionless protocols obtain a route while sending to the same destination.  The use field provides a count of the number of packets sent using that route.  The interface entry indicates the network interface utilized for the route.  A route which is marked with the RTF_IFSCOPE flag is instantiated for the corresponding interface.  A cloning route which is marked with the RTF_PROXY flag will not generate new routes that are associated with its interface scope.


So the first new line is for a network itself and having the UC flags indicates that the route is useable and will generate (clone) a new route upon being used; namely a host route which is that second line that gets added when first you run a docker command and access the VM. Its flags, UHLWIi, respectively indicate it is useable, is a host route, performs address translation, was generated as a result of cloning, is associated with an interface, and is holding a reference to that interface which is indicated in the Netif column as vboxnet. And docker is happy as can be seen with any of its command such as:

$ docker version
Client version: 1.5.0
Client API version: 1.17
Go version (client): go1.4.1
Git commit (client): a8a31ef
OS/Arch (client): darwin/amd64
Server version: 1.5.0
Server API version: 1.17
Go version (server): go1.4.1
Git commit (server): a8a31ef

Interestingly, if I put the Macbook into sleep and come back at this point, when I look at the route table the host route now has changed. Notably the destination, gateway, and flags. That new flag, b, indicates that it is a broadcast address. Once exercised it reverts back to the same destination, gateway, and flags that we had before.

Destination    Gateway           Flags  Refs  Use    Netif Expire
...
92.168.59      link#14           UC        2    0  vboxnet
192.168.59.255 ff:ff:ff:ff:ff:ff UHLWbI    0   15  vboxnet

So far so good.

VPN to the Rescue (NOT)

Now I start up that VPN profile and try using docker which is no longer happy after quite a TCP timeout delay:

$ docker version
Client version: 1.5.0
Client API version: 1.17
Go version (client): go1.4.1
Git commit (client): a8a31ef
OS/Arch (client): darwin/amd64
FATA[0032] An error occurred trying to connect: Get https://192.168.59.103:2376/v1.17/version: dial tcp 192.168.59.103:2376: i/o timeout

Looking at the routing table, the host route has been removed completely and the 192.168.59 network routing has now been changed to:

Destination    Gateway           Flags  Refs  Use  Netif Expire
...
192.168.59     link#13           UCS       0    0  utun0

That changed entry is the problem. With this entry in the routing table the network to which 192.168.59.X traffic gets routed is the VPN's network adaptor and it knows nothing about any VM running locally.

In this state with VPN running I've been unable to remove that route with any of the following:

sudo route -n delete -net 192.168.59.0/24 -interface utun0
sudo route -n delete -net 192.168.59.0 -interface utun0 

Even though they appear to work, looking at the route table afterward shows the same route still in there. However, once VPN is turned off there is hope.


Fixing Once VPN is Off

Once I shut down my VPN and look at the route table that entry has disappeared and neither boot2docker start nore sudo route -n flush followed by boot2docker start will add it back in although restarting the Mac will fix it. But it can be added back in manually with the following command:

sudo route -n add -net 192.168.59.0/24 -interface vboxnet0

If you are uncertain if your VM is using the vboxnet0 interface you can find out with this command which  is returning vboxnet0 on my machine as shown:

$VBoxManage showvminfo boot2docker-vm --machinereadable | grep hostonlyadapter | cut -d '"' -f 2
vboxnet0

Once re-added, take a look at the table and you'll see:

Destination    Gateway           Flags  Refs  Use    Netif Expire
...
92.168.59      link#14           UCSc      0    0  vboxnet

And docker is happy again. 

If anyone figures out how to fix that link in the route table while VPN is open please leave a post so the rest of us can docker while we work.

Enjoy.