Cron jobs and server load - Help

All about creating websites!
Post Reply
User avatar
z3r0aCc3Ss
Fame ! Where are the chicks?!
Fame ! Where are the chicks?!
Posts: 700
Joined: 23 Jun 2009, 16:00
14
Contact:

Cron jobs and server load - Help

Post by z3r0aCc3Ss »

What's the relation between running several cron jobs and server load?
Can running multiple cron job crash the server?
I have a dedicated server, running 15-20 websites, and 10 cron jobs.
Beta tester for major RATs, all kinds of stealers and keyloggers.
Learning NMAP

User avatar
bad_brain
Site Owner
Site Owner
Posts: 11636
Joined: 06 Apr 2005, 16:00
19
Location: In your eye floaters.
Contact:

Re: Cron jobs and server load - Help

Post by bad_brain »

well, depends of course on the kind of cronjobs and when they run. it's a bad idea in general to run multiple cronjobs at the same time, especially when the cause multiple write access (especially again on a non-RAID system). the worst what you can do for example is to run multiple processes with database access (like backups/dumps) because that locks up the databases and when having generic database activity through website visitors it can cause a high i/o-wait which causes again a high CPU load.

so, to stay on the safe side:
- always run only one cronjob at a time
- if the cronjob causes a high HDD activity it's a good idea to nice the process to a lower priority ( http://linux.die.net/man/1/nice" onclick="window.open(this.href);return false; )
- if the cronjob causes a high database activity and/or locks the database it's better to let it run as quick as possible, so either no nice level or even a negative one (but use that option with common sense, an insane nice level of -19 would even prioritize it over kernel processes)
- keep an eye on swap usage, if there is any caused the cronjobs tune your system by adjusting swappiness to speed things up (also with common sense, I have made a post about that on the Linux boards not too long ago.

if you need more detailed advice feel free to post your crontab. :wink:
Image

User avatar
z3r0aCc3Ss
Fame ! Where are the chicks?!
Fame ! Where are the chicks?!
Posts: 700
Joined: 23 Jun 2009, 16:00
14
Contact:

Re: Cron jobs and server load - Help

Post by z3r0aCc3Ss »

@b_b, by reading your reply... I feel like I'm raping my server... :P :D
I have 6 cron jobs running every minute, dumping data into 6 different databases from live twitter and Facebook feed.
One cron job is backing up the database every 12 hours.
One cron job is backing up server files every day once.

Does this looks like a lot of traffic or read/writes?
Beta tester for major RATs, all kinds of stealers and keyloggers.
Learning NMAP

User avatar
bad_brain
Site Owner
Site Owner
Posts: 11636
Joined: 06 Apr 2005, 16:00
19
Location: In your eye floaters.
Contact:

Re: Cron jobs and server load - Help

Post by bad_brain »

sounds a bit of an overkill to me, but well....as long as your server performs well without swapping a lot it'll be no problem.
you should definitely keep an eye on the database server performance though, I recommend to use this script frequently for analysis and then adjusting the settings accordingly:
https://launchpad.net/mysql-tuning-primer" onclick="window.open(this.href);return false;

once you have more and/or high traffic sites you'll not be able to go that way anymore, especially with the backups....you should consider getting a backup server then for setting up a mysql master/slave replication and incremental file backups (the latter one would be already an option now in case you do full backups).
Image

Post Reply