| Time |
Nick |
Message |
| 08:48 |
|
sleary joined #evergreen |
| 10:28 |
|
mantis1 joined #evergreen |
| 11:31 |
|
mantis1 left #evergreen |
| 13:06 |
|
sleary joined #evergreen |
| 13:53 |
Bmagic |
For those of you following from home: my issue turned out to be a legit DDOS from Singapore! Once I took a close look at the search queries coming in, they were all the same! I was using standard bash tools to figure out which IP's were hitting the server, and I settled on one, but it turned out that wasn't it. memcached revealed what I needed to know |
| 13:53 |
Bmagic |
memcdump --servers 172.31.86.26|grep "47\.128"|wc |
| 13:57 |
Bmagic |
resulted in: 11,756. So, we have over 11k unique IP's asking our server to search. The search was pretty simple. "where the crawdads sing" x1000. The search query was just different enough each time "copy_limit=10;" and/or "copy_offset=10;" thrown in there to make the Evergreen logic result in a different key for memcached. Which I believe breaks "opac.max_concurrent_search.query" tracking |
