Why?

Recently I had to make a script which runs once per week and gathers images of an admin interface (login required) and sends it to its manager by email for quick overview. This is great because it saves the manager’s time as he doesn’t have to login to the website and click through all of the graphs and wait for them to load, instead he gets a weekly (YMMV) summary. The site was hosted on Webfaction, which, as it turns out, is a cool web host (haven’t come across it before) because it allows you to compile source code and run custom binaries.

How?

The task of automatically logging in the admin dashboard and taking screenshots  proved to be an excellent job for CasperJS, an open source navigation scripting & testing utility written in Javascript for the PhantomJS WebKit headless browser.

Installation

In order to run CasperJS scripts, you need to have NodeJS installed. Installing NodeJS on Webfaction is as straightforward as it can get, as it’s done via their control panel in the Applications tab, as shown on the illustration below:

It’s useful to put NodeJS executable in your path because then you can run node from any folder. You can do this by going to your NodeJS application’s bin directory and execute the following command:

export PATH=$PATH:$PWD/bin/

Installing CasperJS is easy via npm (Node package manager):

npm install -g casperjs

However,  in order for CasperJS to work, you need to install PhantomJS and you can do it by cloning it from its GitHub repository and running the build.sh script:

mkdir -p $HOME/src

cd $HOME/src

git clone git://github.com/ariya/phantomjs.git && cd phantomjs

git checkout 1.5

./build.sh

cp $HOME/src/phantomjs/bin/phantomjs $HOME/bin

To test if PhantomJS has been successfully installed you can run:

phantomjs --version

To have the binaries in my path every time I log in I added this alias to my .bashrc:

alias nodeset="export PATH=$PATH:/home/webapp/node/bin:/home/webapp/bin"

and my .bash_profile looks like this:

# User specific environment and startup programs
PATH=$PATH:$HOME/bin
export PATH
nodeset

CasperJS script for automatically saving images

var casper = require("casper").create({
	verbose: false,
	logLevel: 'debug'
});

var url = "http://your-app.com/", 
	saveDir = "/home/app/savedImages";

casper.start(url, function() {
	this.fill('form#fm_login', {
        'username':    'username',
        'password':    'password',
    });

    this.echo("I filled the form");
});

casper.then(function(){
	this.click('#btn_login');
	this.echo("I'm, logging in");
});

//first graph fetch
casper.thenOpen("http://your-app.com/graph_1", function(){
    this.wait(3000, function() {
        this.echo("I've waited for 3 seconds for the graphs to load.");

        casper.then(function(){
            var fileName = saveDir + '/graph_1.png';
            this.viewport(1920, 1080);
            this.captureSelector(fileName, '#page_content');
            this.echo('first snapshot taken');
        });
    }); 
});

casper.run(function() {
    this.echo('done').exit();
});

First I initialize casper and set few variables (url, saveDir – note that it has to be full path). Then I start casper by visiting the passed in url and fill the login form by using caspers fill() function. After that I click on the login button and woila – I’m logged in. After that I open up the link with the graph and wait 3 seconds for the graph to load, and then I save the selected area by using the captureSelector() function and a CSS selector of the area I want to save.

NodeJS script for automatically sending email with attachment via Gmail SMTP

This node script uses nodemailer which can be installed using npm:

node install -g nodemailer

This script uses Gmail SMTP (if you want to test it make sure to enter correct Gmail account credentials), but nodemailer offers many more options for sending the email. The simple email sending script is here:

var nodemailer = require("nodemailer");
var rootFolder = "/home/app/savedImages/";
// create reusable transport method (opens pool of SMTP connections)
var smtpTransport = nodemailer.createTransport("SMTP",{
    service: "Gmail",
    auth: {
        user: "[email protected]",
        pass: "your.gmail.pass"
    }
});

// setup e-mail data with unicode symbols
var mailOptions = {
    from: "Automatic Graph Sender <[email protected]>", // sender address
    to: "[email protected]", // list of receivers
    subject: "Automatic Graph Sender", // Subject line
    text: "Hello, I'm Roboty, an automatic report sending robot. You will find your files in attachment. I wish you a great day! Bye, Roboty", // plaintext body
    html: "Hello,<br/><br/>I'm Roboty, an automatic report sending robot. You will find your files in attachment.<br/><br/>I wish you a great day!<br/><br/>Bye,<br/>Roboty",
    attachments: [
        { filePath: rootFolder + "graph_1.png"},
        { filePath: rootFolder + "graph_2.png"},
        //add as many as you wish
    ]
}

// send mail with defined transport object
smtpTransport.sendMail(mailOptions, function(error, response){
    if(error){
        console.log(error);
    }else{
        console.log("Message sent: " + response.message);
    }

    // if you don't want to use this transport object anymore, uncomment following line
    smtpTransport.close(); // shut down the connection pool, no more messages
});

Cron jobs

The following script will run every day at exactly 19:50h. I added the PHANTOMJS_EXECUTABLE needed by CasperJS executable, then I wrote the path to CasperJS and the script that we want to execute and finally I redirected all the output to the temporary file for later inspection if needed.

50 19 * * * PHANTOMJS_EXECUTABLE=/home/app/bin/phantomjs /home/app/node/bin/casperjs /home/app/casper/saveImages.js > /tmp/outputCheck.txt

Mailing script runs 5 minutes after the image gathering script and also outputs everything to the temporary file used in some other success status checking script.

55 19 * * * /home/emtool/webapps/node/bin/node /home/emtool/my_folder/casper/mail.node.js > /tmp/emailOut.txt

Hope this proved useful to someone, and hit me with comments or questions you may have.

CodeProject