Site stability fixed (I think)

Druchii.net Announcements! Whenever something is IMPORTANT, it is in here - PLEASE READ!!!

Moderator: The Dreadlords

Post Reply
User avatar
Daeron
Malekith's Best Friend
Posts: 3982
Joined: Fri Oct 03, 2003 7:36 pm
Location: Belgium, Brussels
Contact:

Site stability fixed (I think)

Post by Daeron »

Greetings everyone,

For the past weeks, both Ulthuan and Druchii sites have been suffering from downtime. I want to apologize for the inconvenience.
The source of the problem was that AI bots were scraping our forums, to the point that they completely shut down the normal working of the sites. I've blocked their access, and it seems the site is acting normally once more.

Unfortunately, our forum is longer being used to train AI LLM's. As a result, AI bots might become less versed in Warhammer and may start talking like normal people again.

If there are any other issues, please feel free to point them out and I'll have a look at it.
Cheers,
Daeron

--- For those interested in more technical details

Our site was struggling to communicate with the outside world (sftp and ssh access were still fine). This lead me to believe there was a congestion for HTTP. The Host support indicated the site was functioning as expected, and that there may have been "too much" access. The site had over 700K visits yesterday. It took me a while to figure it out, but the access logs showed a lot of bot activity. Some googling showed it's AI LLM training related.
Over 600K (88%) were from a single bot source.

I tried a different strategy for Ulthuan and Druchii, to see which one would be more effective. For Druchii I tried the nice way, by blocking the AI bot through the robots.txt file. Bots usually check this to see what they are and are not permitted to do. It seems this one, Claude, wasn't interested in playing nice. For Ulthuan I tried a Firewall rule to block Claude. Unfortunately, this isn't supported by our Host without paying a premium. It is possible to secure this at the DNS level though. So I set up the CloudFlare account to provide our DNS services, and set up the blocking rule there.
The traffic dropped almost immediately, and the firewall rule was triggered roughly 2000 times in just a few minutes.
Since the firewall is up, forum behaviour is back to normal.

The downside is that we're now dependent on an external party for the DNS service, but it gives us free (but limited) Web Application Firewall. This shouldn't be an issue at technical level. I just prefer to keeps things centered, but this was getting out of hand.
I love me a bowl of numbers to crunch for breakfast. If you need anything theoryhammered, I gladly take requests.

Furnace of Arcana, a warhammer blog with delusional grandeur.

"I move unseen. I hide in light and shadow. I move faster than a bird. No plate of armour ever stopped me. I strike recruits and veterans with equal ease. And all shiver at my coldest of whispers."
- The stiff breeze
User avatar
Calisson
Corsair
Corsair
Posts: 8821
Joined: Fri Mar 14, 2008 10:00 pm
Location: Hag Graef

Re: Site stability fixed (I think)

Post by Calisson »

Thanks for taking care of the sites. :D
Winds never stop blowing, Oceans are borderless. Get a ship and a crew, so the World will be ours! Today the World, tomorrow Nagg! {--|oBrotherhood of the Coast!o|--}
Post Reply