Results 1 to 17 of 17
  1. #1

    Hunting for a cron server - help please!

    Hi there, I had the urge/idea to create a service where you can setup crons on our server, instead of your own, either because your host is too restrictive or you don't have the ability to.

    It started when I realised my Mediatemple package didn't allow 1min crons (fair enough), so figured a specialised service offering it (or any time interval) would be handy. Searched for others but wasn't too satisfied and thought this would be a fun project regardless.

    No idea what server I need apart from a good one. I don't expect that many hits or crons running since it'll most likely be a paid service, but the logic would be as follows:

    - Every minute, run a cron for us, fetch data from database
    - Find out which ones I'm meant to run, and request those pages (maybe using cURL?)
    - Bam, that's about it

    Simplified that'll be the most intensive part, the site itself shouldn't be too much to handle (a few updates by a few users a day).

    What do I need to look for in a host for this? I know 1min crons can be a burden but in this case it's necessary. Sadly I don't know anything about servers or hosting or what I'd need.

    Like I said, the most intensive thing would be the cron running. I can clarify anything if needed, and I'd like a figure I'd be looking at for costs, managed preferably, but also unmanaged in case I find a partner to help out with it. Both would be able to factor into my equations. I'm anticipating costs of thousands a month, don't know if that's accurate.

    Edit; Also something to consider is a culmination of many on the hour or so mark. If I had 100 people set for a 1hr interval, and 20 for 1min, I don't want the site to be hammered at each hour and cause 1-2 minute delays to get through and request each one.

    Thanks for any help
    Last edited by MrHappyPants; 06-19-2009 at 08:44 AM.

  2. #2
    Join Date
    Mar 2009
    Location
    Miami, Florida
    Posts
    19,185
    Well. There are several "Remote Cron" services out there. You really dont need a high end server at all, a VPS will actually work for this. You can easilly create a PHP Script to do this and call the remote URL with GET, cURL or even Lynx.
    Keith I Myers
    KMyers.me The rantings of a lunatic
    Geek Survival Guide - Reviews and Advice for Geeks

  3. #3
    Really? I was under the (apparently incorrect) impression it'd take a more powerful server. I guess that's good news then

  4. #4
    Join Date
    Dec 2007
    Location
    Indiana, USA
    Posts
    16,087
    I don't know... if I really needed a 1 minute cron rather than relying on a third-party service that I'd likely have to pay for in addition to my normal hosting plan I'd probably just move to another provider that would satisfy my needs.

    That's just my thoughts on the subject, it may work out to be very successful for you.
    Michael Denney - MDDHosting LLC
    New shared plans for 2016! Check them out!
    Highly Available Shared, Premium, Reseller, and VPS
    http://www.mddhosting.com/

  5. #5
    Join Date
    Dec 2001
    Location
    Above The Clouds
    Posts
    6,999
    A cron itself doesn't cause any load. It's what teh cron does that can cause load. If it's pulling something from a database it would depend on what it is pulling and how large the table was. If it's something light then I don't see how it would be an issue even on shared hosting Loading a page on a webite might cause more load for example as it could have 10-20 or more queries.

    So basically a one minute cron is meaningless without context.
    Laurence Flynn @ atOmicVPS LTD
    Linux & Windows Cloud Hosting Solutions Powered by OnApp
    Fully Managed [Shared][Reseller][Cloud VPS] [Dedicated]
    Featuring the atOmicSTACK ● Speed ● Performance ● Reliability

  6. #6
    Join Date
    Jun 2008
    Location
    London, UK
    Posts
    928
    From where the OP is coming from though, it only matters about quantity of crons, as the server is actually handling the process.

    I think.
    The Simple Host Ltd - Premium Quality European Web Hosting - Uptime Stats
    Shared, Reseller, VPS & Dedicated Hosting
    15-Day Money Back Guarantee, Nightly Backups, Installatron, Softaculous, RVSiteBuilder PRO, Instant Setup - All as standard!
    Call (+44) 0845 300 9066 for more information.

  7. #7
    Join Date
    Dec 2001
    Location
    Above The Clouds
    Posts
    6,999
    Even better then. Pretty sure even all you cPanel guys don't limit how often a cron can run.
    Laurence Flynn @ atOmicVPS LTD
    Linux & Windows Cloud Hosting Solutions Powered by OnApp
    Fully Managed [Shared][Reseller][Cloud VPS] [Dedicated]
    Featuring the atOmicSTACK ● Speed ● Performance ● Reliability

  8. #8
    Join Date
    Feb 2005
    Location
    Australia
    Posts
    5,842
    Quote Originally Posted by TSH-Nathan View Post
    From where the OP is coming from though, it only matters about quantity of crons, as the server is actually handling the process.
    If the cron job is just requesting a web page (presumably PHP/MySQL-generated) from another server then yes, most of the work will be done by the other server. But if 1-minute cron jobs are forbidden on the other server, running them like this at 1-minute intervals may well cause resource-usage problems also.
    Chris

    "Some problems are so complex that you have to be highly intelligent and well informed just to be undecided about them." - Laurence J. Peter

  9. #9
    Join Date
    Dec 2007
    Location
    Indiana, USA
    Posts
    16,087
    Quote Originally Posted by NexDog View Post
    Even better then. Pretty sure even all you cPanel guys don't limit how often a cron can run.
    There is a limit in the terms of service however I can tell you that if you set a 1 minute cron to write one line of text to a file or something ridiculously simple and resource-non-intensive you're not going to hear from us. It's when your job being run by the cron every minute causes issues that it would be investigated.

    Most providers don't go hunting for reasons to cite people and suspend/and/or/terminate them but will take action when one individual's use of the service affects the performance for everybody else on the server.

    As stated - it's not the cron itself but the task that is fired by the cron that causes the load - so having an external cron set to 1 minute is still going to cause suspension if you're not careful.
    Michael Denney - MDDHosting LLC
    New shared plans for 2016! Check them out!
    Highly Available Shared, Premium, Reseller, and VPS
    http://www.mddhosting.com/

  10. #10
    Join Date
    Dec 2005
    Location
    Loma Linda, CA
    Posts
    41
    Quote Originally Posted by MikeDVB View Post
    I don't know... if I really needed a 1 minute cron rather than relying on a third-party service that I'd likely have to pay for in addition to my normal hosting plan I'd probably just move to another provider that would satisfy my needs.

    That's just my thoughts on the subject, it may work out to be very successful for you.
    I would just recommend a VPS or VDS many shared hosting companies would not allow a 1 minute cron job as it would use up more resources than they allow.

  11. #11
    Join Date
    Dec 2007
    Location
    Indiana, USA
    Posts
    16,087
    Quote Originally Posted by wbacky View Post
    I would just recommend a VPS or VDS many shared hosting companies would not allow a 1 minute cron job as it would use up more resources than they allow.
    It's not really the cron, but what job the cron is firing minutely that is causing the issue.
    Michael Denney - MDDHosting LLC
    New shared plans for 2016! Check them out!
    Highly Available Shared, Premium, Reseller, and VPS
    http://www.mddhosting.com/

  12. #12
    Join Date
    Feb 2008
    Location
    Houston, Texas, USA
    Posts
    2,955
    Most cron jobs are scripts that need local server / filesystem access. I don't see how a remote cron server can be useful in that regard.

    Best
    UNIXy - Fully Managed Servers and Clusters - Established in 2006
    [ cPanel Varnish Nginx Plugin ] - Enhance LiteSpeed and Apache Performance
    www.unixy.net - Los Angeles | Houston | Atlanta | Rotterdam
    Love to help pro bono (time permitting). joe > unixy.net

  13. #13
    Join Date
    Dec 2007
    Location
    Indiana, USA
    Posts
    16,087
    Quote Originally Posted by UNIXy View Post
    Most cron jobs are scripts that need local server / filesystem access. I don't see how a remote cron server can be useful in that regard.

    Best
    This is true as well, most cron jobs you don't want to be publicly accessible otherwise somebody could become aware of it and send hundreds of simultaneous requests on the cron to either bring the server down or have your account suspended in a hurry.
    Michael Denney - MDDHosting LLC
    New shared plans for 2016! Check them out!
    Highly Available Shared, Premium, Reseller, and VPS
    http://www.mddhosting.com/

  14. #14
    Quote Originally Posted by KDisk View Post
    Well. There are several "Remote Cron" services out there.


    Yes, a detailed list here (onlinecronservices.com)

    My one is ArtCava WebSchedule

    I have some experience in remote cron .
    You really dont need a high end server at all, yes that's true that loading a page on a website might cause timed-out your service. I solved this item making a new task for any job my service have to play... If someone get time-out only that task fault, or better report the error.

    I allow maximum 5 minutes for a cron...

    available for further questions
    Last edited by artcava; 06-27-2009 at 06:03 PM. Reason: apply some smilies

  15. #15
    Join Date
    Feb 2004
    Posts
    741
    Here is another one, http://www.onlinecronjobs.com . I had a client tell me about this site today; never knew they existed.

  16. #16
    Join Date
    Jun 2006
    Location
    NYC
    Posts
    1,446
    Quote Originally Posted by atbnet View Post
    Here is another one, http://www.onlinecronjobs.com . I had a client tell me about this site today; never knew they existed.
    Hah.. Never realized it either... We have a VPS that we have on the same private subnet as our backup interfaces. We use it for this purpose.

    Free... can't go wrong with that!
    FiberPeer.Com | | REAL DDoS Protection | Cloud Hosting | VPS | Dedicated Servers | High Bandwidth Hosting | 1Gbps-10Gbps Unmetered
    FiberPeer DDoS Mitigation | ethProxy Upgraded! | 14-Years Experience | Emergency 24/7 Support
    Visit us @ www.fiberpeer.com

  17. #17
    Thanks for all the responses! I'll do my best to reply.

    Quote Originally Posted by MikeDVB View Post
    I don't know... if I really needed a 1 minute cron rather than relying on a third-party service that I'd likely have to pay for in addition to my normal hosting plan I'd probably just move to another provider that would satisfy my needs.

    That's just my thoughts on the subject, it may work out to be very successful for you.
    That's a fair point. I originally thought of this however when adding a very minor feature to a site of mine. Without it, the site could still run fine, but with it, it'd definitely set it apart from competition. Not important enough to warrant a major host change though, especially since apart from that, the host is fine. I reluctantly ended up going with a 5-min cron.

    Quote Originally Posted by NexDog View Post
    A cron itself doesn't cause any load. It's what teh cron does that can cause load. If it's pulling something from a database it would depend on what it is pulling and how large the table was. If it's something light then I don't see how it would be an issue even on shared hosting Loading a page on a webite might cause more load for example as it could have 10-20 or more queries.

    So basically a one minute cron is meaningless without context.
    Quote Originally Posted by TSH-Nathan View Post
    From where the OP is coming from though, it only matters about quantity of crons, as the server is actually handling the process.

    I think.
    Pretty much, it'd be up to the account owner to make sure their script worked well. I guess I can set a lower timeout value when connecting to make sure. I'd have to think that one out a bit more, but glad to know at least on my end it won't be so intensive.

    Quote Originally Posted by UNIXy View Post
    Most cron jobs are scripts that need local server / filesystem access. I don't see how a remote cron server can be useful in that regard.

    Best
    You have a point there - that didn't spring to mind. I guess that'd be more of an overall business problem rather than one with the server, though, but thanks for bringing it up.

    Quote Originally Posted by MikeDVB View Post
    This is true as well, most cron jobs you don't want to be publicly accessible otherwise somebody could become aware of it and send hundreds of simultaneous requests on the cron to either bring the server down or have your account suspended in a hurry.
    As above it's something I'd have to consider. I think a fairly strong authentication could be built though using even basic techniques (HTTP auth with us having the password for example).

    Quote Originally Posted by artcava View Post


    Yes, a detailed list here (onlinecronservices.com)

    My one is ArtCava WebSchedule

    I have some experience in remote cron .
    You really dont need a high end server at all, yes that's true that loading a page on a website might cause timed-out your service. I solved this item making a new task for any job my service have to play... If someone get time-out only that task fault, or better report the error.

    I allow maximum 5 minutes for a cron...

    available for further questions
    I've been thinking of doing it like that too, so that if one users' request fails it doesn't just stop the entire lot of them.

    Thanks again for all the replies.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •