Monitoring a Web Page for Changes

Bash Script

Today, I found myself needing a way to monitor a page. I didn’t need anything fancy, just something that would alert me if a page changed in any way. So, I set up a simple bash script and cron job to monitor the page. For me, this was a perfect solution. Since I’ve got a server running 24/7, it’s always able to monitor the page. This wouldn’t work quite as well from, say, a laptop, but a server or always-on desktop work perfectly. But in truth, all you really need is a system capable of running cron jobs. So, without further ado, whip open your favorite text editor and plug this in there:

#!/bin/bash
pageAddress="http://example.com/index.html"
pageHashFile=/path/to/pageHash.txt

newhash=$(curl "${pageAddress}" | md5sum | awk '{ print $1 }')
oldhash=$(cat $pageHashFile)

# Check the hashes, send an email if it's changed
if [ $newhash != $oldhash ]; then
    echo "${pageAddress}" | mail -s "Page changed!" [email protected]

    # Only update the hash if the email was successfully sent.
    returnCode=$?
    if [[ $returnCode == 0 ]] ; then
        echo "${newhash}" > $pageHashFile
    fi
fi

Of course, you’ll need to change the page address, the path to where you want the hash put, and the email so that they meet your situation. Finally, just add the script to your crontab, and you’re good to go! I’ve got mine set to run every 10 minutes. To put it in your crontab, run crontab -e, and insert the following (adapt it as needed):

*/10 * * * * bash /path/to/script.sh

It could be adapted to be more versatile and enable monitoring multiple pages, but since I just needed one (at least for now), this does the trick nicely.

Comments

3 responses to “Monitoring a Web Page for Changes”

  1. Taylor McGann Avatar

    Yay for bash!

    This is very elegant. Might I ask (instead of Google), what’s the difference between putting variables in ${} and $().

    ${} evaluates an expression?
    $() holds a variable with pipes and multiple commands?

    My bash knowledge is slowly slipping away from me….:(

    Also why did you pass the md5sum into awk and not cat or something? Does md5sum pass back multiple arguments or something? I assume cat doesn’t work and that’s why you used awk, but just wondering as to the specifics.

  2. russt Avatar
    russt

    As I understand it, $() simply executes an expression, which is especially useful when assigning the result of the command to a variable.

    ${} can do some variable expansion. There’s some really cool stuff you can do with it, but I haven’t experimented with it much yet. Mostly, I used it for including a variable within a string.

  3. russt Avatar
    russt

    Also, since I forgot to answer the other question you asked, I used awk because md5sum includes the name of the file (or in the case of stdin, has a hyphen at the end). So I used awk to just get the first part of the result.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.