fin_wait tcpip

[Julie Kimmel]

Hello…I just need some enlightenment.

I had a situation yesterday where everything just slowed to a complete stop.

With the much appreciated help of Quovadx helpdesk, we found that if we bounced the process, it would run for a little while..then halt again.

Then they found a suspect outbound port which had a tcpip status (through looking at netstat) of

fin_wait….a sign of an ugly disconnect.

In stopping and restarting the application that that port was sending to, everything seemed to start to work OK.

My question, that I failed to ask before leaving help desk is..I understand that that port is having problems…but how did it consume all of the CPU cycles

Topic

[Traci Zee]

Julie,

I don’t have a good answer for your 100% CPU cycles used…but thought you’d like to hear a likely explanation to ‘fin_wait’ issues.

It’s been a while, but here’s my experience:

If a firewall times out due to inactivity it breaks the connection w/o telling either application its tcp/ip connection is gone and causes it to go into fin_wait.  

Whenever I have an interface that has been ‘working well for awhile’ and all of a sudden I get ‘fin_wait’ disconnects the cause has been that the timeout on a network appliance is shorter than the tcp/ip keep alive timeout on the OS on EITHER side of the connection.  

Once Ops set the OS Keepalive timeout below the network firewall, my odd fin_wait issues stopped occurring.

Hope this is helpful!

Traci Zee

Emdeon Business Services, LLC

770.844.9242

[Julie Kimmel]

Thank you so much for your reply Traci,

In this situation, though…it is an interface with in house application… and I was stopping/starting the interface all over the place…so I don’t know if a timeout would have been an issue in this case…but I could imagine how stops and starts piled up on top of eachother might cause such a situation….however I definatly will apply your advice to other problems of mine that I have been encountering with routers and VPNs, etc etc.

But I am still wondering very much about what could consume all of that CPU time…so if anyone has any thoughts out there….

Thanks!

Julie

[John Mercogliano]

Take a look at your err and log file sizes.  Being you are having connection issues, your error log might be getting to large.  I have found that if you don’t cycle often enough, not only does your process size grow large, but the whole process can lock up. Being when you cycle a process this automagically cycles your log files that could explain why it clears up for a while.  I have encountered similiar issues and I’ve just increased the time that I cycle the logs until we can resolve it.

I’ve notices I start having the problem when the log reached the low 8 digits in bytes.

This problem can also lock up your hostserver.  We have encountered this on our test system when we add a process or connection and forget to add cycling.  When you attempt to switch to a site in the client gui with a large log or smat file the hostserver will lock up.

On HP-UX I use gpm to find the open files associated with a process to find the offender.

Hope this helps,

John Mercogliano
Sentara Healthcare
Hampton Roads, VA

[Julie Kimmel]

very interesting…I will have to keep an eye on that.

I do have threads that continuously send out ‘harmless’ errors that do fill up the logs quickly.

Is there a command to cycle the logs without downing the process? I seem to remember trying something like that a couple of years ago

[Michael Hertel]

You can set the process up to automatically cycle logs.

It’s under the edit process info.

[Author]

Replies