Results 1 to 6 of 6
  1. #1
    Join Date
    Oct 2004
    Location
    Celebration, Florida
    Posts
    209

    Help with Bash Monitoring Script...

    Good morning all.

    While I know PHP, HTML, Java programming and can handle most linux tasks, I'm stuck on a bash script, and am hopeful for a little help.

    Currently, we have two linked services (service1 and service2) running on one of our servers. Together, those provide access to an online conferencing system. Randomly and very infrequently, one of the services stop--usually service2--making the service completely unavailable. Since the services are linked, stopping and starting service1 (see below) restarts both.

    Therefore, we set up a cron that runs a bash script every ten minutes to make sure that the service is running and, if it's not, attempts a restart. The script checks two things--the results of the "service status" command for each service and the availability and response of port 9000--the port that service1 uses to provide the conferencing access.

    However, despite these checks, the service is still occasionally failing without being automatically restarted. So, my question to you is, based on the information above and the bash script below, can you think of anything that we can add to the bash script that would do a better job of monitoring the service and restarting it if necessary? Is there a third check that we could add that would further help catch outages? Finally, with the organization of the script, is the port check being run correctly?

    The script is below... and thanks!


    ------
    SERVICE1=`/sbin/service service1 status | grep -c "service1.*is running"`
    SERVICE2=`/sbin/service service2 status | grep -c "service2.*is
    running"`

    (echo >/dev/tcp/localhost/9000) &>/dev/null
    if [ $? -eq 0 ]; then
    if [[ "$SERVICE2" == "0" || "$SERVICE2" == "0" ]]; then
    /sbin/service service1 stop
    sleep 10
    /sbin/service service1 start
    TO="XXX@xxxx.com"
    echo "SERVICE `date`" | /bin/mail -s
    "SERVER ALERT: RESTART" "$TO" -c "$CC"
    fi
    else
    /sbin/service service1 stop
    sleep 10
    /sbin/service service1 start
    TO="XXX@xxxx.com"
    echo "PORT `date`" | /bin/mail -s "SERVER ALERT:
    RESTART" "$TO" -c "$CC"
    Richard C. Hay - Celebration, Florida
    Web Application Development, Graphic Design & Organization Management Services

  2. #2
    Join Date
    Feb 2005
    Location
    Australia
    Posts
    5,849
    The logic seems a bit... strange, TBH. Indentation makes things a bit clearer:
    Code:
    SERVICE1=`/sbin/service service1 status | grep -c "service1.*is running"`
    SERVICE2=`/sbin/service service2 status | grep -c "service2.*is running"`
    
    (echo >/dev/tcp/localhost/9000) &>/dev/null
    if [ $? -eq 0 ]; then
        if [[ "$SERVICE2" == "0" || "$SERVICE2" == "0" ]]; then
            /sbin/service service1 stop
            sleep 10
            /sbin/service service1 start
            TO="XXX@xxxx.com"
            echo "SERVICE `date`" | /bin/mail -s
            "SERVER ALERT: RESTART" "$TO" -c "$CC"
        fi
    else
        /sbin/service service1 stop
        sleep 10
        /sbin/service service1 start
        TO="XXX@xxxx.com"
        echo "PORT `date`" | /bin/mail -s "SERVER ALERT:
        RESTART" "$TO" -c "$CC"
    Looks like you're missing a "fi", and you only test service2 (or service2 ) if the echo test shows no error... What's the real aim here; to run all 3 tests and if any of them fails, run the restart procedure?
    Chris

    "Some problems are so complex that you have to be highly intelligent and well informed just to be undecided about them." - Laurence J. Peter

  3. #3
    Join Date
    Oct 2004
    Location
    Celebration, Florida
    Posts
    209
    Hi Chris.

    Thanks so much for writing. The end goal is to basically test three things and, if any one of them fails, to stop the service, pause, and restart the service, and then send an email.

    Would I be better off simplifying and setting it up like:

    -----
    SERVICE1=`/sbin/service service1 status | grep -c "service1.*is running"`
    SERVICE2=`/sbin/service service2 status | grep -c "service2.*is running"`

    (echo >/dev/tcp/localhost/9000) &>/dev/null

    if [[ "$SERVICE2" == "0" || "$SERVICE2" == "0" || $? -eq 0 ]]; then
    /sbin/service service1 stop
    sleep 10
    /sbin/service service1 start
    TO="XXX@xxxx.com"
    echo "PORT `date`" | /bin/mail -s "SERVER ALERT:
    RESTART" "$TO" -c "$CC"
    fi
    -----

    (By the way, the script had indentation, but I lost it in the paste. Sorry about that!)

    Richard
    Richard C. Hay - Celebration, Florida
    Web Application Development, Graphic Design & Organization Management Services

  4. #4
    Join Date
    Feb 2005
    Location
    Australia
    Posts
    5,849
    That certainly looks better, but in bash the conditional operators are a bit weird. Not sure but I think you may need "-o" instead of "||", and watch out for string vs numeric comparisons. Personally I prefer the KISS principle, particularly for little scripts like this, eg.

    Code:
    RESULT="FAILED"
    FAIL=0
    
    SERVICE1=`/sbin/service service1 status | grep -c "service1.*is running"`
    if [ $SERVICE1 -eq 0 ]; then
        RESULT="$RESULT 1 "
        FAIL=1
    fi
    
    SERVICE2=`/sbin/service service2 status | grep -c "service2.*is running"`
    if [ $SERVICE2 -eq 0 ]; then
        RESULT="$RESULT 2 "
        FAIL=1
    fi
    
    (echo >/dev/tcp/localhost/9000) &>/dev/null
    if [ $? -eq 0 ]; then
        RESULT="$RESULT 3 "
        FAIL=1
    fi
    
    if [ $FAIL -eq 1 ]; then
        /sbin/service service1 stop
        sleep 10
        /sbin/service service1 start
        TO="XXX@xxxx.com"
        echo "PORT `date`" | /bin/mail -s "SERVER ALERT: RESTART. $RESULT" "$TO" -c "$CC"
    fi
    Chris

    "Some problems are so complex that you have to be highly intelligent and well informed just to be undecided about them." - Laurence J. Peter

  5. #5
    Join Date
    Oct 2004
    Location
    Celebration, Florida
    Posts
    209
    Wow Chris... thank you so much. I'm off to try this now.

    Richard
    Richard C. Hay - Celebration, Florida
    Web Application Development, Graphic Design & Organization Management Services

  6. #6
    Join Date
    Oct 2011
    Location
    Redding, CA
    Posts
    62
    Just wanted to toss in a word for "monit" here. It might be a little overkill for your precise need in this case, but it does exactly what you're trying to achieve with a shell script. I found it to be one of those things that I find many other uses for once I have it in place.
    <<< Please see Forum Guidelines for signature setup. >>>

Similar Threads

  1. Replies: 2
    Last Post: 07-17-2011, 09:10 PM
  2. Bash/perl/python etc. script for Adaptec raid card monitoring ?
    By WebHostDog in forum Hosting Security and Technology
    Replies: 0
    Last Post: 12-07-2010, 09:31 PM
  3. How to write bash script for round-trip email monitoring
    By sallyanne in forum Hosting Security and Technology
    Replies: 0
    Last Post: 02-20-2010, 02:28 PM
  4. Can .htaccess call a script (e.g. a bash .sh script)?
    By Kadence in forum Hosting Security and Technology
    Replies: 7
    Last Post: 03-26-2008, 03:59 PM
  5. bash comman/bash script
    By getbusy in forum Hosting Security and Technology
    Replies: 2
    Last Post: 12-25-2004, 06:50 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •