- This topic has 10 replies, 6 voices, and was last updated 8 years, 11 months ago by
.
-
Topic
-
I am unsure what this is telling me. An unexpected exception occurred during the login process.
The exception shown below was caught.
Unable to establish a connection to the host server: 10.8.52.12; nested exception is:
com.hie.cloverleaf.securityserver.NoCloverleafSrvException: Unable to contact Server on server: 10.8.52.12; nested exception is:
java.rmi.ConnectIOException: Cannot connect to host: //10.8.52.12:13019/; nested exception is:
java.rmi.ConnectException: Connection refused to host: 10.8.52.12; nested exception is:
java.net.ConnectException: Connection refused: connect
-
Replies
-
-
-
James:
Did you have this problem prior to Cloverleaf 5.8?I admit I had some challenges to get the virtual service address to fail-over transparently and had to redo the NIC definitions with my fail-over startup scripts so that the virtual service address is the one associated with the gateway, then everything using the virtual service address has worked nicely.
We are not on cloverleaf 5.8 yet and that is why I asked.
I would expect the issues discussed to arise if no action like I described was taken to make the virtual servcie address fail-over in a transparent fashion.
With each level of the OS (AIX) and HACMP comes a new method to accomplish this for us.
Prior to our using ehter channel I would detach the NIC and reattach them in the fail-over startup to assure the virtual IP was on the NIC of the gateway and the persistant address was on the other NIC.
Now that we are using ether channel I alias the addresses to accomplish the same outcome as follows:
Code:
#————————————————————————–
# get netstat snapshot before doing any ehter channel alias reconfiguration
#————————————————————————–rm -f $SCRIPTDIR/os.start__netstat-audit.txt
echo “” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “—————————————————————-” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “netstat -rn before doing any ether channel alias reconfiguration (`date`)” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “—————————————————————-” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “” >> $SCRIPTDIR/os.start__netstat-audit.txtnetstat -rn >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “—————————————————————-” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “netstat -in before doing any ether channel alias reconfiguration (`date`)” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “—————————————————————-” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “” >> $SCRIPTDIR/os.start__netstat-audit.txtnetstat -in >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “” >> $SCRIPTDIR/os.start__netstat-audit.txt
#——————————————————————————————-
# remove the persitent address ether channel alias from en2 to assure all outbound traffic
# will use the service address ether channel alias on en2 as the default address
#
# Note:
# – this is so that all inbound and outbound trafic will failover transparently
# – the hostname is the name of the persistent address
# because it is no longer necessary to set the hostname to the name of the service address
#——————————————————————————————-ifconfig en2 $HACMP_ORIG_HOSTNAME -alias
#———————————————————————————————————
# add back the ehter channel alias for the persistent address on en2
# now that the ether channel alias for the service address is the default address for all outbound traffic
#———————————————————————————————————ifconfig en2 $HACMP_ORIG_HOSTNAME netmask 255.255.255.128 alias
#————————————————————————–
# get netstat snapshot after doing any ehter channel alias reconfiguration
#————————————————————————–echo “” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “================================================================” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “netstat -rn after doing any ether channel alias reconfiguration (`date`)” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “================================================================” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “” >> $SCRIPTDIR/os.start__netstat-audit.txtnetstat -rn >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “================================================================” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “netstat -in after doing any ether channel alias reconfiguration (`date`)” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “================================================================” >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “” >> $SCRIPTDIR/os.start__netstat-audit.txtnetstat -in >> $SCRIPTDIR/os.start__netstat-audit.txt
echo “” >> $SCRIPTDIR/os.start__netstat-audit.txt
This will also eliminate issues we used to have with firewalls before I figured out how to make the virtaul IP fail-over transparently.
I used hcitcptest with an tcp/ip trace to make sure the traffic coming and going from both nodes in the HACMP cluster indicate the virtual IP and nothing shows the persistent IP and once that was accomplished all my problems dissapeared so far.
I did observe that getting the outbound traffice to have the virtual IP was accomplished everytime when I had the virtual address associated with the gateway.
Here is the last HACMP log of os.start__netstat-audit.txt to help grasp the concept if you are able to follow along.
Yes weve been up longer than we like without a fail-over but just too busy, good thing we have it running so robustly and practially bullet proof.
Still not the worst we ever did once we went a little over 900 days of uptime.
Code:
—————————————————————-
netstat -rn before doing any ether channel alias reconfiguration (Sun Apr 4 03:18:26 CDT 2010)
—————————————————————-Routing tables
Destination Gateway Flags Refs Use If Exp GroupsRoute tree for Protocol Family 2 (Internet):
default 10.111.87.129 UG 3 857557 en2 – –
10.10.15.0 10.10.15.35 UHSb 0 0 en2 – – =>
10.10.15/25 10.10.15.35 U 3 2390738 en2 – –
10.10.15.35 127.0.0.1 UGHS 0 859307 lo0 – –
10.10.15.127 10.10.15.35 UHSb 0 4 en2 – –
10.111.87.128 10.111.87.143 UHSb 0 0 en2 – – =>
10.111.87.128/25 10.111.87.143 U 1 246239 en2 – –
10.111.87.143 127.0.0.1 UGHS 0 4931 lo0 – –
10.111.87.148 127.0.0.1 UGHS 0 0 lo0 – –
10.111.87.255 10.111.87.143 UHSb 0 4 en2 – –
127/8 127.0.0.1 U 7 1989966 lo0 – –Route tree for Protocol Family 24 (Internet v6):
::1 ::1 UH 0 3742 lo0 – –—————————————————————-
netstat -in before doing any ether channel alias reconfiguration (Sun Apr 4 03:18:26 CDT 2010)
—————————————————————-Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll
en2 1500 link#2 0.9.6b.dd.2d.72 7373974 0 3597908 4 0
en2 1500 10.10.15 10.10.15.35 7373974 0 3597908 4 0
en2 1500 10.111.87.1 10.111.87.143 7373974 0 3597908 4 0
en2 1500 10.111.87.1 10.111.87.148 7373974 0 3597908 4 0
lo0 16896 link#1 2860239 0 2849831 0 0
lo0 16896 127 127.0.0.1 2860239 0 2849831 0 0
lo0 16896 ::1 2860239 0 2849831 0 0================================================================
netstat -rn after doing any ether channel alias reconfiguration (Sun Apr 4 03:18:26 CDT 2010)
================================================================Routing tables
Destination Gateway Flags Refs Use If Exp GroupsRoute tree for Protocol Family 2 (Internet):
default 10.111.87.129 UG 0 857557 en2 – –
10.10.15.0 10.10.15.35 UHSb 0 0 en2 – – =>
10.10.15/25 10.10.15.35 U 0 2390738 en2 – –
10.10.15.35 127.0.0.1 UGHS 0 859307 lo0 – –
10.10.15.127 10.10.15.35 UHSb 0 4 en2 – –
10.111.87.128 10.111.87.148 UHSb 0 0 en2 – – =>
10.111.87.128/25 10.111.87.148 U 1 246239 en2 – –
10.111.87.143 127.0.0.1 UGHS 0 0 lo0 – –
10.111.87.148 127.0.0.1 UGHS 0 0 lo0 – –
10.111.87.255 10.111.87.148 UHSb 0 4 en2 – –
127/8 127.0.0.1 U 2 1989966 lo0 – –Route tree for Protocol Family 24 (Internet v6):
::1 ::1 UH 0 3742 lo0 – –================================================================
netstat -in after doing any ether channel alias reconfiguration (Sun Apr 4 03:18:26 CDT 2010)
================================================================Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll
en2 1500 link#2 0.9.6b.dd.2d.72 7373974 0 3597909 4 0
en2 1500 10.10.15 10.10.15.35 7373974 0 3597909 4 0
en2 1500 10.111.87.1 10.111.87.148 7373974 0 3597909 4 0
en2 1500 10.111.87.1 10.111.87.143 7373974 0 3597909 4 0
lo0 16896 link#1 2860239 0 2849832 0 0
lo0 16896 127 127.0.0.1 2860239 0 2849832 0 0
lo0 16896 ::1 2860239 0 2849832 0 0The default gatewat IP in this case is 10.111.87.129 and shows up on en2 wich is the combination of more than one NIC card since we are using ether channel.
The virtual IP in this case is 10.111.87.148
The persistant IP in this case is 10.111.87.143
Since I’m using ether channel wich combines 2 NICs into one in my case that means I simply need the virtual IP 10.111.87.148 to appear in the list ahead of the persistent address to have the desired outcome, which is what the audit file shows after redoing the NIC ether channel aliasing during fail-over startup.
Personally I think it is a sad event that IBM does not automatically handle transparent fail-over of the virtual service address.
I’ve seen many places struggle with this and without this transparent IP fail-over I argue you haven’t got satisfactory fail-over.
For the work we do the transparent fail-over of the virtual service IP is one of the most important outcomes to achieve.
Russ Ross
RussRoss318@gmail.com -
Has anyone seen this behavior. Primary box 10.5.19.82 Virtual 10.5.10.105. Logon with box IP OK. Once logged on can change sites and enter Virtual IP and works OK. Exit and log back in get the following exception
“Unknown host: 10.5.19.82; nested exception is:
java.net.UnknownHostException: No entry in proxy socket for host: 10.5.19.82
Now if if I click OK and then reenter 10.5.19.82 them it works.
-
I just installed CIS 5.8.4 on our Primary server as a separate version. Current prod version CIS 5.7 works fine. When starting the 5.7 IDE, I can use the Virtual IP DNS name. But, when I open the CIS 5.8.4 GUI, I cannot initially use the Virtual IP DNS. I can enter the actual server DNS, and select a site. From that point on, I can use the Virtual IP if I choose. But once I log off, then I would have to start over. I was not involved when 5.7 was installed. Not sure if there is some entry somewhere that needs updated. Think I may try and contact the Lawsom person that was involved, but that was two years ago.
-
I don’t remember if my specifics were exactly the same symptoms as you described but I looked at my $HCIROOT/server/server.ini file and see I added the following at the very bottom of the file to be able to use my service address name in the Cloverleaf IDE/GUI
Code:[firewall]
rmi_exported_server_port=mdahub8snamdahub8sna is the name of my virtual machine name used for HACMP fail-over between my active and passive nodes, so I want to always use it instead of the persistant hostnames of the individual nodes.
I’m curious if you were to add these lines to your server.ini file for your case and stop/start the hostserver do you end up with a better outcome.
Russ Ross
RussRoss318@gmail.com -
Thanks for the reply. It dawned on me several hours later to look at the CIS 5.7 server.ini I saw the rmi_exported_server_port= our vip
We have a section in our server.ini marked firewall. We have entries because of Global Monitor and there was a statement for host server default port.
I again thank you for the feedback. It would have been a life saver
-
Highland Dave:
What is the server.ini entry to change the port number that the host server listens on by default?
I have never changed the default port # that the host server listens on.
I knew there was a server.ini entry to handle that but either can’t recall what that looks like or have yet to see it.
Russ Ross
RussRoss318@gmail.com -
-
So far we have only used basic security and not advanced security.
Even though we purchased advanced security we have been too overwhelmed with work demands to take it past basic security.
As long as basic security suffices as it has thus far, I don’t see use fiddling with advanced security anytime soon.
I don’t know if advanced security will need any special additional server.ini entries.
Russ Ross
RussRoss318@gmail.com -
Thought I’d follow up with the missing piece and our final resolution. In addition to the “[firewall]rmi_exported_server_port=clovprod” entry in server.ini, we had to add our AIX cluster’s LPAR hostnames (e.g., clusternode1 and clusternode2) to the ACL entries in hciaclrolemgr for the GUI to connect and authenticate successfully with the advanced security server. We left our existing service names defined in the ACL config of the Security Server (e.g., clovprod and clovtest). Support was very helpful in confirming that we could simply add the entries in hciaclrolemgr.
Evidently the security server stopped recognizing the “service name” (or “virtual” server’s hostname) when we stopped assigning it as hostname in our startup scripts. We are still able to use our service name/ip address in the GUI (our “virtual” servers’ hostnames) though.
-
- The forum ‘Cloverleaf’ is closed to new topics and replies.
