DDOS Suspension

I’m going to take a break from Disney posts to make note of something that happened early last week.  On Monday night, April 30th, I opened up Windows Live Writer to quickly post from Disney.  As I hit Publish, Live Writer told me that there was a problem publishing my post.  At first, I figured I did something wrong.  I loaded up my website and saw a terrifying message: Account Suspended.

My mind began to race.  My sites were down.  All of them.  TechyDad.com, TheAngelForever.com, FollowerHQ.com.  All down.  And not just down, but proclaiming to the world that we were suspended.  A quick call to my host later and our accounts were restored to full access with the explanation that a page on our sites had caused undue strain on the server.

Let’s put aside for a moment the suspension reaction… I’m dealing with them on how better to react.  In fact, a similar problem a couple of days ago resulted in my sites being taken down with an error message for a few minutes – a much better solution.

Back to the strain, though.  When I got back home, I loaded up the log files.  Now, a typical log file, for a single view of one page, will contain multiple entries.  After all, as you’re reading this, you’re likely seeing images, there are colors and text styling via CSS files, JavaScript files and more.  This is part of what I saw in B’s log file: - - [30/Apr/2012:13:21:03 -0400] "GET /2012/04/a-floral-splash-of-color/ HTTP/1.1" - - [30/Apr/2012:13:21:03 -0400] "GET /2012/04/dole-whip-cravings/ HTTP/1.1" - - [30/Apr/2012:13:21:03 -0400] "GET /2012/04/the-cupcake-club-book-giveaway/ HTTP/1.1" - - [30/Apr/2012:13:21:05 -0400] "GET /2012/02/falling-in-love-with-disney/ HTTP/1.1" - - [30/Apr/2012:13:21:03 -0400] "GET /2012/04/selecting-phone-sounds/ HTTP/1.1" - - [30/Apr/2012:13:21:04 -0400] "GET /2012/04/views-from-a-friends-birthday-party/ HTTP/1.1" - - [30/Apr/2012:13:21:06 -0400] "GET /2012/04/time-for-ubp-2012/ HTTP/1.1" - - [30/Apr/2012:13:21:06 -0400] "GET /2012/04/photo-gifts-for-mothers-day/ HTTP/1.1" - - [30/Apr/2012:13:21:05 -0400] "GET /2012/04/disney-natures-chimpanzee/ HTTP/1.1"

As you can see, these were requests for only the HTML code of the pages. No CSS, images, or JavaScript downloaded. Just repeated calls to pages on TheAngelForever.com.  Different IP addresses were used, but they all seemed to originate from Amazon Web Services in Dublin, Ireland.

So what happened?  I have two theories.  The first is that a content scraper was trying to pull all of B’s content via an Amazon Web Services hosted script.  Something went wrong and it made so many requests so quickly that it bombed the site out.  The second is that the person behind the script didn’t care about content and the rapid requests/site bombing was by design.  In this scenario, our mystery script runner isn’t a scraper, but instead launched a DDOS attack on TheAngelForever.com.

(Quick explanation for those who don’t know what a DDOS attack is.  A DDOS, or Distributed Denial of Service, attack is when a group of computers request so many services from a server that valid visitors can’t get through.  To use an analogy, imagine a restaurant that thrives on delivery orders.  Now suppose a group of people (thus "distributed") repeatedly call the restaurant’s phone and hang up.  If they do this often enough, valid would-be customers can’t get through (i.e. "denial of service").

Stopping a DDOS attack is tricky.  We can’t block the attacker because they are coming from so many IP addresses.  You can block a group of IP addresses (e.g. 46.137.*), but then you might be blocking a lot of valid users.

So how can we keep this from happening again?  There are some network tools that our hosting provider can employ, but not much B and I can do.  It might happen again tomorrow or it might never happen again at all.  So if you ever see that message again, just be patient (and perhaps tweet me) because the site will be back soon.