FWIW, I'm still trying to analyze whats up...it feels like there aren't enough connections to serve the content. Unfortunately I know at 3am the system isn't taxed one bit and should be at it's fastest, but it isn't.
I've even added memcache/memcached with 512MB so everything serves from ram - so it's not disk access issues which still makes me think there aren't enough system (read operating system) connections or way too many http calls per page. I'm going to play with caching compiled php to see if that helps any.
I can help you with your hacks, I just don't know what you've done. Let me know.