I can not speak to windows, but I will address your question on Linux.
I have been running on Linux since 2000, and have been relatively happy.
My main (only) reason for switching from AIX to Linux was cost. Our AIX system was stable, and our Linux box is stable as well.
(We do have a minor semaphore problem, that Quovadx tells me has been fixed in 5.4) ..
If you move off of Windows, I suggest that you contract some time with Quovadx, as I understand that there is some non-intuitive differences, and modifications to some scripts.
Little on sizing ..
We have about 100 threads in our production site, and we handle approximately 600,000 messages outbound from the engine each day.
I suggest that you get fast spinning disks. We have two disks that are mirrored, and an identical fail-over system that has its mirrored disks, which are mirrored with production using DRDB.
We have 2GB of memory and 2 Xeon (hyperthreading) 3 GHz processors in each box..
Dennis