Why

  • Faster delivery of software
  • Set us on the path of Continuos Delivery
  • Immediate discovery of regression
  • And why javascript – Facilitate front-end team to actively contribute to tests alongside feature development

What

  • Build an automation framework which will simulate user interactions, and run these tests consistently on real and headless browsers alike
  • Set up integration with CI server(Jenkins)
  • Import test results into our Test Management Platform(QMetry)

How

The front-end team uses Mocha/Chai/Karma for writing unit tests. The QA team uses Selenium for automation. When we picked these frameworks and tools, we evaluated them thoroughly for our needs. We also wanted to leverage our existing frameworks and tools as much as we could so that there would be less of a learning curve. Fortunately for us, we found Selenium bindings in Javascript. Actually there are quite a few, but the most prominent of them are webdriver.io and Selenium’s webdriverJs.

We chose Selenium’s webdriverJs primarily for the following reasons:

  • It is the official implementation by the Selenium team who have written bindings in various other languages
  • The patterns of writing a test is very similar to a test written in java world with which our QA team was familiar
  • Its use of promises to prevent callback hell

For more detailed explanation with examples, please refer here.

Our next piece of the puzzle was to figure out if we could use PhantomJs (headless browser) with webdriverJs. We needed this so that we could run the tests on our CI server where we may not necessarily have a real browser. We did have some initial challenges to run webdriverJs in combination with PhantomJs without using Selenium’s Remote Control Server, but looking at the source code (in Javascript) helped us debug and get this to work. The challenges could also be attributed to the lack of complete understanding of Selenium’s automation world.

The last piece of the puzzle was integration with the CI server. With PhantomJs already in place, all we needed to figure out was the reporting format of the tests that our CI server (Jenkins) could understand. One of the reasons we had picked Mocha is for its extensive reporting capabilities. Xunit was the obvious choice because of Jenkins support for it.

And there it is – our automation framework – all of it in Javascript stack.

Testing Stack for Web

In the past couple months, we have successfully automated our E2E tests that provide coverage for regression of our web platform, and use it on a daily basis. Now that we have an established framework and gained immense experience writing tests, we are one step closer to Continuous Delivery. Integration with our Test Management Platform is in the works and we will post our findings soon.

reads

https://code.google.com/p/selenium/wiki/WebDriverJs
http://visionmedia.github.io/mocha/
http://chaijs.com/
http://chaijs.com/plugins/chai-webdriver
http://phantomjs.org/page-automation.html
https://speakerdeck.com/ariya/phantomjs-for-web-page-automation
http://casperjs.org/
http://engineering.wingify.com/posts/e2e-testing-with-webdriverjs-jasmine/
http://xolv.io/blog/2013/04/end-to-end-testing-for-web-apps-meteor
http://code.tutsplus.com/tutorials/headless-functional-testing-with-selenium-and-phantomjs–net-30545

why

  • Our application contains several modules that uniquely addresses a key aspect of wealth management. eg: Investment checkup, Net worth, etc. The application brings together various such aspects to present a comprehensive one-stop wealth management tool to the user.
  • How-ever each user’s needs are unique and so are the motivations to use our application. So we wanted to build landing pages that ties a user’s need to a module thus providing more context and motivation to use the module and eventually our entire offerings.
  • We went native for our mobile presence to provide great user experience and that has served us well. How-ever they are few modules of our application that are self contained with simpler ui design and fewer interactions with rest of the application. We wanted to embed such modules in our native app and determine if we could still deliver a great seamless user experience.

what

  • Build “stand-alone” versions of our web modules.
  • Publish the module with a unique url that will be used to embed it in other apps.

how

We use Requirejs as our module loader and its optimization tool to package a module and its dependencies into single units for production use. The module definition would define all its dependencies. During development phase, all the dependencies are loaded as required. How-ever in production mode, all the dependencies are packaged into a single file along with the module code to minimize our http calls and optimize load times. For eg:

A simple module definition would be as follows:

define([
 'jquery'
 , 'underscore'
 ], function($, _){

 var foo = {};
 // module code
 return foo;
});

And we reference the above module as follows:

<script data-main="scripts/foo" src="scripts/require.js"></script>

But most real-world application will have several modules and would share considerable number of framework libraries (jquery, underscore, backbone and etc) and utility libraries. And our application is no exception. So we defined a common module that would require all the common libs and in turn require this as part of our main module. The sub modules would still continue to list of all its required dependencies, but as part of the build process, we exclude them for optimization and to prevent from any shared lib being re-initialized/reset.

When the application starts, it loads the main module first and there after required modules as user navigates the application. Following is a gist of our build file is defined:

({
 modules: [
 { 
 name: 'app_main'
 }
 , {
 name: 'foo'
 , exclude:[
 'jquery'
 , 'underscore'
 , 'backbone'
 , 'services'
 , 'hbs'
 ]
 }
 ]
})

app_main will be loaded at the start of the application and if a user navigated to a foo module, only then the application will load foo. This worked great for us in keeping our main module lean as we kept adding more features (modules) until the requirement of reusing some of the modules in other apps/clients (mobile).

Fortunately for us, this did not involve lot of work and more importantly did not result in any redundant code. All we had to do was to define another main module that would include our common module and foo module. That is it. We had to refactor our common module a little bit to include only the core libs. Other shared libs were loaded as part of the main module (app_main). This way we did not load any libs that were not needed for foo module.

({
 modules: [
 { 
 name: 'app_main'
 }
 , {
 name: 'foo'
 , exclude:[
 'jquery'
 , 'underscore'
 , 'backbone'
 , 'services'
 , 'hbs'
 ]
 }
 , {
 name: 'app_foo'
 }
 ]
})
//app_main.
require(['common'], function (common) {
 require(['main']);
});

//app_foo
require(['common'], function (common) {
 require(['foo']);
});

This was just not fortune. From the very start of this app development, we not only aimed for modular code, but also to have an ability to extend any module to work by itself. Our entire MV* stack was architected that way and we are glad the returns are truly amazing.

We have a SPA that is comprised of several modules and at any time, any module can be extended to its own SPA app with minimal effort and no codes changes to the modules themselves. Thanks to the amazing requirejs optimization tool and a little foresight.

More details on how we use requirejs and its optimization tools are presented here.

reads

http://addyosmani.github.com/backbone-fundamentals

http://requirejs.org/docs/optimization.html

http://requirejs.org/docs/optimization.html#wholemultipage

http://requirejs.org/