Wednesday, June 3, 2015

IPv6 and Centos 6.6 -- SocketException: Permission denied

What I don't know about IPv6 would fill a very long web page.

When verifying that Enterprise Failover Manager (EFM) would work with IPv6 on Centos 6.6, my connections (using JGroups) would fail at the socket level with Permission denied.

A short Java app reproduces the problem:

[root@FOUR ~]}> cat
public class IPv6Test {
    public static void main(String[] args) {
        try {
            InetAddress ia = InetAddress.getByName("fe80::20c:29ff:feb0:ba66");
            System.err.println("Opening socket for: " + ia);
            Socket socket = new Socket(ia, 22);
            System.err.println("We have: " + socket);
        } catch (Exception e) {
[root@FOUR ~]}> javac && java IPv6Test
Opening socket for: /fe80::20c:29ff:feb0:ba66 Permission denied
    at Method)
    at [....]

I thought this was a problem at the OS level creating the socket in the first place, but now I understand that the error message could be coming back from the remote node and this is how it's displayed by Linux back to the client. In case others run into this, here are the configuration changes needed to get a proper IPv6 setup working. First, disable the usual suspects (if you know how to properly use NetworkManager and ip6tables, feel free to do so instead of killing them):

[root@FOUR ~]}> grep disabled /etc/selinux/config
#     disabled - No SELinux policy is loaded.
[root@FOUR ~]}> service NetworkManager stop
Stopping NetworkManager daemon:                            [  OK  ]
[root@FOUR ~]}> chkconfig NetworkManager off
[root@FOUR ~]}> service ip6tables stop
ip6tables: Setting chains to policy ACCEPT: filter         [  OK  ]
ip6tables: Flushing firewall rules:                        [  OK  ]
ip6tables: Unloading modules:                              [  OK  ]
[root@FOUR ~]}> chkconfig ip6tables off

At this point, the problem is my link-scoped IPv6 address on both nodes:

[root@THREE ~]}> ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0C:29:B0:BA:66  
          inet addr:  Bcast:  Mask:
          inet6 addr: fe80::20c:29ff:feb0:ba66/64 Scope:Link

What we want is a globally-scoped address for each node. After a little editing, this is my current eth0 config:

[root@FOUR ~]}> cat /etc/sysconfig/network-scripts/ifcfg-eth0

The IPv6 address was found using this link, and then using ::1, ::2, etc., for the various virtual machines. After the above changes and a reboot, I now have a proper global IPv6 address on my nodes:

[root@THREE ~]}> ifconfig
eth0      Link encap:Ethernet  HWaddr 00:0C:29:B0:BA:66  
          inet addr:  Bcast:  Mask:
          inet6 addr: fdd9:6fe6:6a5b:3835::3/64 Scope:Global
          inet6 addr: fe80::20c:29ff:feb0:ba66/64 Scope:Link

…and no more SocketException with the new address:

[root@FOUR ~]}> javac && java IPv6Test
Opening socket for: /fdd9:6fe6:6a5b:3835:0:0:0:3
We have: Socket[addr=/fdd9:6fe6:6a5b:3835:0:0:0:3,port=22,localport=49971]

My thanks to the JGroups users forum (and Bela), the OpenJDK list, and Dave Page for their help getting me back on track with IPv6.