Unlock POST while logged out causes infinite loop

[brandond]

I'm not sure exactly how to debug this, but it appears that under some circumstances we can get an unlock POST sent just after the user logs out. This seems to throw PHP and/or Apache into a loop that eats 100% CPU time. This doesn't happen all the time, but out of about 30 thousand unique users per day it will happen once or twice.

I've been trying to track it down, but all I can see via strace is that Apache is constantly allocating memory with mmap. If I leave it running long enough it will actually run the system out of RAM.

The request logs show some normal client action, and then a logout. Then I get a request that hangs Apache, which never makes it to the log because logging is done AFTER request processing is complete.

128.223.129.51 - ldixon [25/Oct/2010:15:53:29 -0700] "GET /?_task=logout HTTP/1.1" 200 2907 "https://webmail.uoregon.edu/?_task=mail&_id=13294649134cc607910fab6&_action=compose" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.11) Gecko/20101012 Firefox/3.6.11"
128.223.129.51 - - [25/Oct/2010:15:53:29 -0700] "GET /plugins/account_link/skins/default/account_link.css?s=1282548336 HTTP/1.1" 200 128 "https://webmail.uoregon.edu/?_task=logout" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.11) Gecko/20101012 Firefox/3.6.11"
128.223.129.51 - - [25/Oct/2010:15:59:18 -0700] "POST / HTTP/1.1" 302 20 "https://webmail.uoregon.edu/?_task=logout" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.11) Gecko/20101012 Firefox/3.6.11"

server-status on the hung Apache worker shows that the request it's processing is:

POST /?_unlock=loading1288047052649 HTTP/1.1

I'll see if I can find some commonality in what they're doing just before logout, on the off chance that has anything to do with it.

[alec]

Any news on this? I'm unable to reproduce.

[brandond]

I haven't seen it in a while, and haven't been able to get any additional insight into the cause. Might as well go ahead and close the ticket for now, I'll reopen it if I have more data.