We ran into this problem on 5.7 rev2 on linux red had enterprise. While I can’t definitively say which fix really was the solution here are some of the things we did:
1. Increased max file size: fs.file-max=262144 in /etc/sysctl.conf
2. Increased semaphoresL kernel.sem=1000 128000 400 1024 in /etc/sysctl.conf
3. We have a multi-server configuration with global monitor. One of the upgrades (adding rev 2 onto 5.7) was corrupted. We reinstalled. This was more likely associated to our Java problems but could have been associated to the too many open files issue too.
