10-25-2013 07:13 AM
We had an issue last night where our web service became completely unresponsive behind our ADX 1000. There seems to be an issue with the clock on the system? The logs repeatedly showed
Oct 24 00:10:17 Boyd-ADX, L4 TCP SYN limits 262143 reached
with the time never changing. We manually set the clock to try and see if that fixed it, but it still ended up with the wrong time and the time never increased after that. After a reboot, the box seems to be stabilized and working, but we're concerned about this happening again. Has anybody seen this and been able to resolve it? We do use NTP in our network, so that should have corrected the time issue. Any thoughts are appreciated.
10-25-2013 10:00 AM
Definitely open a ticket on that, support should be able to help guide you to the root cause. Given the limited info, it looks like your session table was full, this could be due to a DDoS attack, heavy traffic, or other issues. Best bet is support though.
10-26-2013 06:41 AM
Found the solution (or rather the problem). It was a bug in the code we run 12.3.01d that makes the ADX clock lock up after either 620 days, or the TCP SYN number of 262143 is reached. The syslog revealed that by having the frozen clock, and the only solution is a reload. The bug is fixed in code 12.3.01f, but since we're probably going to decommission this box anyway, we're not really looking to do any upgrades. Just FYI for anybody in the future.