One idiot was ripping the site with enormous speed and connections per second. Something I haven’t tuned the site for, have no money, have no only intentions to support such behaviour and will not tolerate it.
Yes, the site contains gigabytes of pictures, yet it’s free. Want more and right now? Donate. Want to have all the pictures on your disk and immediately? Donate more and I will upgrade the hardware. Do not want to pay? No problem. But behave.
The recent couple of weeks I was busy tuning the performance. Pages load almost 5 times faster now. But I had no time for updates.
Apparently this DoS attack (effectively it was a DoS attack, and obviously not the first time) will keep me busy protecting the site from rippers for a couple of more weeks. Yes more fun…
I’m sooo pissed off…
See Site performance, Server errors, outages and tunings forum thread.
Forgive me… But what is a Ripper ?
I love your website and I do love all the information
and your posts. Please keep up the good work.
And thank you for your postings on Pantyhoes.
D.V.
“D.V. ” wrote:
A “ripper” is a person who downloads entire content of websites using a “spider” program.
“Spiders” go through all links and according to the rules, filters and options download either each and every file or, for example, only images.
Rippers can be configured to download one file by one, with a pause in between or simultaneously using several threads.
The program which brought the site down was establishing several dozens of connections per second. Considering that the site has only 1GB of RAM and shares a VERY BUSY disk array with other virtual systems, it was lethal.
Is there anything one can do to stop “ripping” ?
Like a time limit or a download limit ?
I recall an old site that would limit the number of items one can download in a 24 hour time limit.
D.V.
Hi,
yes you can do something like that.
You can slow down answers for a specific IP Adress if its using more than XX connections in SSS seconds…
or if an IP adress downloads more than xx kBytes in SSS seconds….
mod_qos? Never used it. Currently reading about.
# /usr/local/apache/bin/apachectl -t -D DUMP_MODULES | fgrep -i qos
#
# yum list | fgrep -i qos
#
OK, let’s compile one.
I knew that there could be limits placed somewhere on a site. Do I get any credit for the thought ?
Like a good self bondage ideia ? (yes, My spelling is not the greatest)
D.V.
“D.V. ” wrote:
I’ve already had lots of them. Everywhere. Unfortunately the providers set their limits too 😀 For example – on the memory and disk.
“D.V. ” wrote:
“D.V. ” wrote:
The idea is obvious, but the implementation is tricky, when your freedom (CPU, disk, RAM) is severely restrained. And this is more of bondage than self-bondage 😉 See also this post
mod_qos is compiled (along with Apache) and configured. Let’s see what the optimal parameters are.
Something’s definitely working. See the forum thread.