first: no it wasn't an attack...
seems the last reboot was a little too long ago so the system became a little buggy, a process had a hang and caused a loop which occupied all memory.
this time I wasn't in the shower, I was sleeping...that's why the downtime was that long....a simple reboot fixed it...
downtime oct. 16
- bad_brain
- Site Owner
- Posts: 11636
- Joined: 06 Apr 2005, 16:00
- 19
- Location: In your eye floaters.
- Contact:
k, further analysis points to this:
at 0:23 local time the database is backed up, and accidentally the mail server needed a lot of resources at the same time....but because the backup script was running the mail server had not enough resources left, this caused some weird loop where the mail server opened new internal connections on every retry until there were so many processed that the server crashed.
the chances that the backup script AND the mail server need resources at the same time are very low because the backup script runs only once a day and usually needs only 2 minutes.....but to avoid such problems in the future I will do the backups on the database mirror server instead of the main server. on the mirror server there are a lot less processes running, and traffic (mail traffic especially) is almost zero. and even if such a problem appears again it will only crash the mirror server, not the main one.
at 0:23 local time the database is backed up, and accidentally the mail server needed a lot of resources at the same time....but because the backup script was running the mail server had not enough resources left, this caused some weird loop where the mail server opened new internal connections on every retry until there were so many processed that the server crashed.
the chances that the backup script AND the mail server need resources at the same time are very low because the backup script runs only once a day and usually needs only 2 minutes.....but to avoid such problems in the future I will do the backups on the database mirror server instead of the main server. on the mirror server there are a lot less processes running, and traffic (mail traffic especially) is almost zero. and even if such a problem appears again it will only crash the mirror server, not the main one.
Re: downtime oct. 16
[quote="p4inl0v3r"][quote="bad_brain"]this time I wasn't in the shower, I was sleeping...that's why the downtime was that long....a simple reboot fixed it... [/quote]
*note : gift b_b an alram clock which goes off when server crashes on his birthday [/quote]
Love that idea!! lol *hands 5 dollars* theres some money for it haha
*note : gift b_b an alram clock which goes off when server crashes on his birthday [/quote]
Love that idea!! lol *hands 5 dollars* theres some money for it haha
me thinks it depends on the exploit, HTTP is a separate service on its own while its on the same machine that also handles FTP service. One may not necessarily effect the entire server. It can also depend on the server OS, I am more familiar with IIS and Novell servers.
reedit:
Check out this link:
DIY:Defending against DDoS - some nice tactics
http://www.darkreading.com/security/att ... =220600886" onclick="window.open(this.href);return false;
No viagra needed here, I get hardons all the time, can't you tell?
DNR
reedit:
that should explain my question, which I sort of answered anyways..Network DoS vs. Web App DoS
Whereas network level DoS attacks aim to flood your pipe with lower-level OSI traffic (SYN packets, etc...), web application layer DoS attacks can often be achieved with much less traffic. Just take a look at Rsnake's Slowloris app if you want to see a perfect example of the fragility of web server availability. The point here is that the amount of traffic which can often cause an HTTP DoS condition is often much less than what a network level device would identify as anomalous and therefore would not report on it as they would with traditional network level botnet DDoS attacks.
Check out this link:
DIY:Defending against DDoS - some nice tactics
http://www.darkreading.com/security/att ... =220600886" onclick="window.open(this.href);return false;
No viagra needed here, I get hardons all the time, can't you tell?
DNR
-
He gives wisdom to the wise and knowledge to the discerning. He reveals deep and hidden things; he knows what lies in Darkness, and Light dwells with him.
He gives wisdom to the wise and knowledge to the discerning. He reveals deep and hidden things; he knows what lies in Darkness, and Light dwells with him.
- bad_brain
- Site Owner
- Posts: 11636
- Joined: 06 Apr 2005, 16:00
- 19
- Location: In your eye floaters.
- Contact:
I think you confuse the servers DNR, on the suck-o.com server there is no ftp at all, your account is on another one....
but in general (not in context with this incident) a server system usually doesn't crash completely at once, usually the services die one by one....first the ones with the most RAM usage, and last the ones with a low RAM usage, so it can happen http, mysql and mail are already crashed but ftp or irc are still available (for a short time).
but in general (not in context with this incident) a server system usually doesn't crash completely at once, usually the services die one by one....first the ones with the most RAM usage, and last the ones with a low RAM usage, so it can happen http, mysql and mail are already crashed but ftp or irc are still available (for a short time).