Recovering lost connections from firewall failover

[Anonymous]

We are now hosted by cloverleaf with our firewall going through Aviatrix. We experienced a firewall failover a while back, and had some interface connections that did not re-establish without manual intervention from the downstream applications. Management has tasked our team to research and discover any possibilities to automate the process of recycling the connections so that they re-establish. Has anyone else experienced this issue, and what process/tasks do you have when a failover occurs for connections that go through a firewall/VPN?

Topic

[Steve Herber]

Some firewalls preserve existing VPN connections during a failover.  Others do not.  In your case, it seems your VPN connection went away when the failover occurred.  This puts the two hl7 servers in a bad state where they think the connection is still up and without the VPN there is no way to successfully close the connection.  When either end sends data, there is no reply because the firewall only creates a new VPN tunnel during opening state.  Both sides have to be manually stopped and restarted.  There is no way for cloverleaf to know that the VPN has gone away so there is no way cloverleaf would know it needs to do anything and there is nothing cloverleaf could do anyway.  Your company should get a firewall that migrates the connection during a failover.  Other than that you could try to create some sort of monitor script and cycle the thread.  And all the remote sites would have to do the same monitoring.

You need to involve your network, firewall, and VPN people to help you with this problem.

At my site we have 5 or 10 VPN connections, some inbound and some outbound.  When the firewall/VPN people want to do an upgrade, we shutdown our side, which forces the remote sites to start trying to reconnect.  Once the VPN is back, we restart all our threads. Both inbound and outbound interfaces then start new connections through the firewall/VPN and all is good.

I haven’t used aviatrix, but his page might be useful:

How does High Availability work with Aviatrix?

Steve Herber
University of Washington

[Author]

Replies