Improving Druchii.net's performance

Throw your ideas for Druchii.net around in here...

Moderator: The Dread Knights

User avatar
Tarbo
Morathi's Best Friend
Morathi's Best Friend
Posts: 1203
Joined: Tue Oct 04, 2005 5:06 pm
Location: Flanders, Belgium

Improving Druchii.net's performance

Post by Tarbo »

It's hardly a secret: Druchii.net has been having serious performance issues lately. Whether related to the server or the software it's running on, the truth is that we're getting 500's time after time. Sometimes, we complain about it, but maybe it's time we find out what's wrong.

Now, I wouldn't start a thread like this without having done a little research, first. There are two things that could be done to seriously improve responsiveness from the server, one of which has been briefly mentioned before, but let's go over the pros and cons of each.

Turn off compression? Druchii.net supports GZIP compression. I, myself, am a big proponent of servers supporting this because it saves bandwidth for every router that the packages pass. However, it's also a fact that compression guzzles CPU. Since no caching is done, each request is being compressed (iff the browser reveals it accepts compression). Perhaps we could experiment with turning it off for a week or so, see what the results are?

Pros -- less CPU needed
Cons -- increases bandwidth


Separate script & style from content? This was touched on before: Druchii.net loses a good deal of bandwidth with inline scripting and styling, and moving these to separate, cacheable objects would do us all a lot of good. Also, the less bytes on the wire, the less the server needs to work!

Pros -- less bandwidth
Cons -- requires some redesign


For a moment, I believed I had a third way, but it has slipped me. It probably wasn't worth much (oh yes, moving to another server--not sure how easy/expensive that is).

Turning off compression sounds like the safest bet. I'm not sure where the bottleneck is (database, CPU, bandwidth) but it's worth a shot, in my humble opinion.
User avatar
Langmann
Malekith's Tastetester & Physician
Malekith's Tastetester & Physician
Posts: 5170
Joined: Thu Oct 10, 2002 9:41 pm
Location: Putting needles into people.

Post by Langmann »

Yeah, it is a source of annoyance I have with the server company. Most of the time the server runs fine, but when it runs slow, I complain, they fix it, and I am pacified for a while.

Then it does it again.

The problem is the bottleneck between the server and the mysql server. I actually have half a mind to switch servers but then I am worried about going from the frying pan to the fire.

How often are you getting 500 errors, certain times during the month, every day? Everyone else?
While running a million dollar company, singing at weddings, and his frequent jetting to Spain Elton Jon style, Dark Alliance found the time to stand on the doorstep of Games Workshop like Moses and the Pharoah and calmly state, "Let my people go."
User avatar
Tarbo
Morathi's Best Friend
Morathi's Best Friend
Posts: 1203
Joined: Tue Oct 04, 2005 5:06 pm
Location: Flanders, Belgium

Post by Tarbo »

langmann wrote:How often are you getting 500 errors, certain times during the month, every day? Everyone else?
As befits the irony of complaining in IT, most problems stopped the instant I had posted this. Performance has been outright commendable ever since.

However, I've had repeated problems with it over most of August; sometimes, this was announced by the company providing the server, but often it was not. It would seem that holiday seasons are the times when some kind of maintenance is done on the server, or a time of DoS attacks.

If the bottleneck is the database, then there is little we can do. I doubt we have the access rights to upgrade the database, so that's not an option. We could limit the queries, but that'd kind of limit the point of the database-driven forum, eh? ;)
User avatar
Balthamael
Cold One Knight
Posts: 221
Joined: Thu Nov 02, 2006 5:48 pm
Location: Norway

Post by Balthamael »

langmann wrote:How often are you getting 500 errors, certain times during the month, every day? Everyone else?

About 5-6 times since the beginning of September. I've tried to refresh the page regularly, but it takes about 10 minutes of refreshing attempts until the forum actually loads. Most of the 500 errors occured around 01.00 - 04.00 CET.
User avatar
Linda lobsta defenda
Witch Elf
Witch Elf
Posts: 8090
Joined: Sun Jun 09, 2002 11:31 pm
Location: Toms River, NJ, USA

Post by Linda lobsta defenda »

If you keep getting these errors write down the date and time and post them so we can get a clear picture of when this happens (oh and dont forget to add your timezone)
User avatar
Tarbo
Morathi's Best Friend
Morathi's Best Friend
Posts: 1203
Joined: Tue Oct 04, 2005 5:06 pm
Location: Flanders, Belgium

Post by Tarbo »

Linda Lobsta Defenda wrote:If you keep getting these errors write down the date and time and post them so we can get a clear picture of when this happens (oh and dont forget to add your timezone)
Saturday, 22h30 GMT. "Only" ten minutes that I know of.

Tuesday, Sept 11, 17h30-17h50 GMT+2 (1+DST)
Last edited by Tarbo on Tue Sep 11, 2007 4:04 pm, edited 1 time in total.
User avatar
Langmann
Malekith's Tastetester & Physician
Malekith's Tastetester & Physician
Posts: 5170
Joined: Thu Oct 10, 2002 9:41 pm
Location: Putting needles into people.

Post by Langmann »

We also do have a problem with DoS attacks. The host actually does a fair job of fighting them off for us and getting things back up and running which is something I would have difficulty doing with my home servers since I don't have the time.

Apparently D.net has enemies.

Yeah keep a log as best we can so I have something to report to the company. They actually do go looking for problems for me and it helps if I have times to tell them about so they can sort out answers.
While running a million dollar company, singing at weddings, and his frequent jetting to Spain Elton Jon style, Dark Alliance found the time to stand on the doorstep of Games Workshop like Moses and the Pharoah and calmly state, "Let my people go."
User avatar
Loki
Brolock
Brolock
Posts: 2296
Joined: Tue Aug 10, 2004 3:41 am
Location: Keeping an eye on Rork and Calisson
Contact:

Post by Loki »

Internal Server Error. 15:40 East coast of the US (GMT -4, I think)
2 more 500 errors at 18:00 GMT -5 (Thanks Rork).

Rork edit: GMT -5 ;)
Last edited by Loki on Sun Sep 09, 2007 10:04 pm, edited 2 times in total.
+++ Team Mulligans +++

Image

FAQ
User avatar
Balthamael
Cold One Knight
Posts: 221
Joined: Thu Nov 02, 2006 5:48 pm
Location: Norway

Post by Balthamael »

9th of September 23.58 CET (GMT +1)
10th of September 14.51 Took atleast 20 minutes before I could load the page.
Last edited by Balthamael on Mon Sep 10, 2007 1:18 pm, edited 1 time in total.
User avatar
Eldacar
Loremaster
Posts: 2806
Joined: Tue Oct 12, 2004 8:22 am
Location: Australia
Contact:

Post by Eldacar »

GMT +10, 11:30AM, Monday 10th September 2007. That search table was taking five minutes or so to load up.
"I live in hope and fear. Hope that once more the Lords of Caledor will ride on the backs of Dragons. Fear that if we do, when we do, it will be our last ride." ~Imrik, High Prince

Initiative Leader - Editors
User avatar
Draknir
Dark Artist
Posts: 1091
Joined: Wed Jan 03, 2007 4:16 pm
Location: Where you'd least expect

Post by Draknir »

500 Internal Server Error, Monday 10th September 2007 16:43 GMT +2, I managed to re-enter the site 16:47.
User avatar
1991 denver broncos
Trainee Warrior
Posts: 35
Joined: Thu Aug 30, 2007 9:10 am
Location: Staffs, England

Post by 1991 denver broncos »

I just got a 500 at around 14:30 BST.

I suspect that the problems could be easily resolved by changing servers. There are much busier forums out there (rllmuk for example), and they never suffer from lag. Though shutting donw one site, moving it, and re opening is a daunting task, and certainly not for the feint hearted.
User avatar
Langmann
Malekith's Tastetester & Physician
Malekith's Tastetester & Physician
Posts: 5170
Joined: Thu Oct 10, 2002 9:41 pm
Location: Putting needles into people.

Post by Langmann »

How long has this been going on for most people now?
While running a million dollar company, singing at weddings, and his frequent jetting to Spain Elton Jon style, Dark Alliance found the time to stand on the doorstep of Games Workshop like Moses and the Pharoah and calmly state, "Let my people go."
User avatar
Loki
Brolock
Brolock
Posts: 2296
Joined: Tue Aug 10, 2004 3:41 am
Location: Keeping an eye on Rork and Calisson
Contact:

Post by Loki »

Well, at least since that time I've emailed you, but probably for most of the summer.
+++ Team Mulligans +++

Image

FAQ
User avatar
Eldacar
Loremaster
Posts: 2806
Joined: Tue Oct 12, 2004 8:22 am
Location: Australia
Contact:

Post by Eldacar »

langmann wrote:How long has this been going on for most people now?

A fair chunk of the year, I'd say. Don't really know the exact dates, but it seems to come and go.
"I live in hope and fear. Hope that once more the Lords of Caledor will ride on the backs of Dragons. Fear that if we do, when we do, it will be our last ride." ~Imrik, High Prince

Initiative Leader - Editors
User avatar
Dggrj
Malekith's Personal Guard
Posts: 813
Joined: Mon Oct 09, 2006 3:20 am
Location: Reporting live from the battlefield!

Post by Dggrj »

I know this isn't useful per se, but I've had really good performance with the site. And while I don't post too often, I'm here reading the page every day.
User avatar
Elfik
Assassin
Posts: 531
Joined: Mon Aug 06, 2007 12:24 am
Location: Victoria, BC, on the Western shores of the New World

Post by Elfik »

Internal server error, 10:37pm, gmt - 7, Monday Sept.10.
People say "insert problem" but it's better to say "insert improvement"
User avatar
1991 denver broncos
Trainee Warrior
Posts: 35
Joined: Thu Aug 30, 2007 9:10 am
Location: Staffs, England

Post by 1991 denver broncos »

Happened again today. I noticed from around 16:25 till 16:45 BST.
All your base are belong to us.
User avatar
Rork
Lord of Khorne
Lord of Khorne
Posts: 8432
Joined: Fri Nov 22, 2002 1:29 pm
Location: Leading the revolution (and in the chat).

Post by Rork »

Happened just now, 1800 BST (GMT+1).
Image

"Rork.. a wonderful guy :)" - Linda Lobsta Defenda

+++ Team Mulligans +++
User avatar
Balthamael
Cold One Knight
Posts: 221
Joined: Thu Nov 02, 2006 5:48 pm
Location: Norway

Post by Balthamael »

11th of September 18.43 - 19.10 CET
GMT +1 (UCT+2 due to daylights savings time).
User avatar
1991 denver broncos
Trainee Warrior
Posts: 35
Joined: Thu Aug 30, 2007 9:10 am
Location: Staffs, England

Post by 1991 denver broncos »

And again until 20:27 BST
All your base are belong to us.
User avatar
Ranieth
Malekith's Best Friend
Posts: 1221
Joined: Thu Jul 31, 2003 4:06 pm
Location: Linköping, Sweden.

Post by Ranieth »

550 at 18:00 GMT+1
User avatar
Draknir
Dark Artist
Posts: 1091
Joined: Wed Jan 03, 2007 4:16 pm
Location: Where you'd least expect

Post by Draknir »

Ranieth wrote:550 at 18:00 GMT+1


Me too.
User avatar
Loki
Brolock
Brolock
Posts: 2296
Joined: Tue Aug 10, 2004 3:41 am
Location: Keeping an eye on Rork and Calisson
Contact:

Post by Loki »

500 at Noon GMT -5
+++ Team Mulligans +++

Image

FAQ
User avatar
Arquinsiel
Shadowdancer
Posts: 4987
Joined: Tue Oct 21, 2003 1:16 pm
Location: The deepest pits in a hell of my own making
Contact:

Post by Arquinsiel »

Warning: mysql_connect() [function.mysql-connect]: Lost connection to MySQL server during query in /home/.catnochio/kitrik/druchii.net/db/mysql4.php on line 48

Warning: mysql_error(): supplied argument is not a valid MySQL-Link resource in /home/.catnochio/kitrik/druchii.net/db/mysql4.php on line 330

Warning: mysql_errno(): supplied argument is not a valid MySQL-Link resource in /home/.catnochio/kitrik/druchii.net/db/mysql4.php on line 331
phpBB : Critical Error

Could not connect to the database


Just now.
ImageImageImageImage
faerthurir wrote:Arq kicked me in the gyros.
Post Reply