Mobile Webkit CSS Sizing Bug: Hardware Acceleration Edition

It’s a well-known “secret” that you can force webkit engines into hardware accelerated rendering by applying what should be non-effectual[link] css3[link] transforms[link]. The reason you care is though HTML5 mobile apps are great, they still require effort to get that native feeling for simple tasks like scrolling[link to momentum scrolling post] and animations. As a result, if you google something like “momentum scrolling html5″ you’ll see posts that give you snippets like this:

*:not(html) {
     -webkit-transform: translateZ(0);
}

and variations thereof.

The main issue with css like this is that it can quickly crash your mobile app with an out of memory error by asking webkit to hardware accelerate every DOM element in the page, which it apparently wasn’t built to do. The other issue is that it’s just not obvious what else is affected when you switch on hardware acceleration all the way down the render tree.

Here’s a simple example where adding a transform modifies how webkit sizes an element:

What You Expected

Now With Extra Whitespace

And here’s the HTML/CSS to make it happen (you can see the same behavior in Chrome Mac 21.0.1180.89):

<!doctype html>
<html>

    <head>
        <style type="text/css">
            body {
                margin: 0;
            }
            #app {
                width: 100%;
                height: 100%;
                margin-left: -20px;
                background: #323232;
                -webkit-transform: translateZ(0);
            }
            .title-bar {
                position: fixed;
                top: 0;
                left: 0;
                right: 0;
                height: 48px;
                text-align: center;
                background: #323232;
                color: #FFF;
            }
            .title-bar h2 {
                margin: 8px 0;
                font-size: 26px;
            }
        </style>
    </head>

    <body>
        <div id="app">
            <div class="title-bar">
                 <h2>App Title</h2>

            </div>
        </div>
    </body>

</html>

There are three ways to fix the issue:

  1. remove “margin-left: -20px” from #app
  2. remove “width: 100%” from #app
  3. remove “-webkit-transform: translateZ(0)” from #app

Regardless of how you think this should render, the fact is that adding the webkit transform does change the sizing of the #app element in an unexpected (at least to me) way. Though easily diagnosed and fixed, it means I’ll have to spend more time scrutinizing the rendering layout leaving less time for building the actual functionality of an app, for now.

What? margin-left: -20px?

So how did I happen across this randomness? I was playing around with using Bootstrap and the built-in responsive design for mobile site development and it’s grid system uses negative margins as part of its column alignment. Hence, this post.

JS Hacks: Dead Simple Javascript Variable Change Watchers

Writing boilerplate to change variables, re-calculate aggregate values and update views sucks and is error-prone.

De-coupleing the management of your data with whatever other parts of your application that affects makes your application more flexible, more testable, and actually easier to understand because there’s less munging of concerns within a single function.

Most of the newer Javascript frameworks have some sort of pattern for variable binding, in Backbone it’s model change events [http://backbonejs.org/#Model] , in Ember it’s controller bindings [http://emberjs.com/documentation/#toc_setting-up-bindings] and Ember.Observable [http://emberjs.com/api/classes/Ember.Observable.html], in Angular [http://angularjs.org/] it’s done declaratively but in the HTML directly and the same is true with KnockoutJS [http://knockoutjs.com/].

However, if you feel like some of these solutions are overly complex, don’t dig the get()/set() syntax or are interested in building a purpose-driven micro-framework you might be interested to know that Javascript actually supports transparent getter/setter functions just like ruby, java, actionscript and others. Here is a simple trio of functions that exposes this to you in a cross-browser manner and even notifies you of modifications done on an array (IE9+):

function watch(target, prop, handler) {
    if (target.__lookupGetter__(prop) != null) {
        return true;
    }
    var oldval = target[prop],
        newval = oldval,
        self = this,
        getter = function () {
            return newval;
        },
        setter = function (val) {
            if (Object.prototype.toString.call(val) === '[object Array]') {
                val = _extendArray(val, handler, self);
            }
            oldval = newval;
            newval = val;
            handler.call(target, prop, oldval, val);
        };
    if (delete target[prop]) { // can't watch constants
        if (Object.defineProperty) { // ECMAScript 5
            Object.defineProperty(target, prop, {
                get: getter,
                set: setter,
                enumerable: false,
                configurable: true
            });
        } else if (Object.prototype.__defineGetter__ && Object.prototype.__defineSetter__) { // legacy
            Object.prototype.__defineGetter__.call(target, prop, getter);
            Object.prototype.__defineSetter__.call(target, prop, setter);
        }
    }
    return this;
};

function unwatch(target, prop) {
    var val = target[prop];
    delete target[prop]; // remove accessors
    target[prop] = val;
    return this;
};

// Allows operations performed on an array instance to trigger bindings
function _extendArray(arr, callback, framework) {
    if (arr.__wasExtended === true) return;

    function generateOverloadedFunction(target, methodName, self) {
        return function () {
            var oldValue = Array.prototype.concat.apply(target);
            var newValue = Array.prototype[methodName].apply(target, arguments);
            target.updated(oldValue, motive);
            return newValue;
        };
    }
    arr.updated = function (oldValue, self) {
        callback.call(this, 'items', oldValue, this, motive);
    };
    arr.concat = generateOverloadedFunction(arr, 'concat', motive);
    arr.join = generateOverloadedFunction(arr, 'join', motive);
    arr.pop = generateOverloadedFunction(arr, 'pop', motive);
    arr.push = generateOverloadedFunction(arr, 'push', motive);
    arr.reverse = generateOverloadedFunction(arr, 'reverse', motive);
    arr.shift = generateOverloadedFunction(arr, 'shift', motive);
    arr.slice = generateOverloadedFunction(arr, 'slice', motive);
    arr.sort = generateOverloadedFunction(arr, 'sort', motive);
    arr.splice = generateOverloadedFunction(arr, 'splice', motive);
    arr.unshift = generateOverloadedFunction(arr, 'unshift', motive);
    arr.__wasExtended = true;

    return arr;
}

Now you can register a handler that will be called every time any part of your code updates a bound variable. For example:

var data = {
     quantity: 0
     , products:  []
}
, watcher = function(propertyName, oldValue, newValue){ … update some other pieces of the application … };

watch(data, 'quantity', watcher);
watch(data, 'products', watcher);

If you’re stepping through your javascript console as you add these you can see that after each call to watch, the properties on data are redefined as getter/setters. So now, you can do things like:

data.quantity = 7;
data.products.push('beer')

and notice that your watcher function is called automatically with the new property values. Hooray!

Now your watcher function can manage propagating the changes to the rest of your application instead of you manually doing that in every place you might update your variables. If we take this one step further, we can get a [sweet way to de-couple concerns even further by providing multiple watch functions per variable - different blog post].

This code is adapted from @eligrey’s original gist which itself is originally modeled after Gecko’s built in watch() functionality.

JS Hacks: Accessing Variables in a Hidden Scope

Modular application structure is great for loosely coupling code to make it more reusable and more maintainable, always has been, always will be.

However, encapsulation has typically flown in the face of cross-cutting concerns like analytics and logging, which for any production web app is absolutely essential. There are a few JavaScript AOP type libraries out there (based on a quick Google search) but in JavaScript we don’t have a class definition lookup like in ActionScript or the forceful reflection APIs that allow us to bust open private scopes like in Ruby.

The idea here is we’d like to be able to track events that happen in our application, along with the associated detailed information, while littering our application code with metrics crap as little as possible. For example, I’d much rather see this:

Analytics.track('signUpComplete');

than:

Analytics.track('signUpComplete', { username: username, browser: 'Chrome 11', donkeys: isItADonkey() });

the point being that I don’t want my application code to care about what the metrics code needs. One solution is to create and pass an accessor function that captures the scope you want to pull additional information from and pass that accessor function to the external module that needs the additional access.

In your application code:

var accessor = function(name){ try{ return eval(name); }catch(e){ /* log error */ return null; } };
Analytics.track('signUpComplete', accessor);

and in your other module’s code:

var eventsSpec = {
     'signUpComplete': {
          username: 'path.to.username'
          , browser: 'path.to.browser'
          , donkeys: 'isItADonkey()'
     }
};

var track = function(eventName, accessor){
    var capturedProperties = {};
    for(var key in eventsSpec[eventName]){
        capturedProperties[key] = accessor(eventsSpec[eventName][key]);
    }
    // fire off call to google analytics or whatever
};

While this means that your metrics code is tightly coupled to your application code and the way it is structured there isn’t really any way around that, the data lives where it lives and you need to get it. On the other hand, your application code is not tightly coupled to your metrics code, which is the goal, sweet!

Copy/paste the lines below into your browser’s console to see a full featured example of accessing properties, property chains and functions in the various closures:

var globalScopeObject = {
     prop: 'dot accessor in outer block scope'
};

var globalFunction = function() {
     return 'global function call';
};

var wrapped = function(){
     var outerBlockScope = 'this is outer block scope';
     var outerBlockScopeObject = {
          prop: 'dot accessor in outer block scope'
     };

     var outerBlockFunction = function() {
          return 'outer block function call';
     };

     return function(name){
          var innerBlockScope = 'this is inner block scope';
          var innerBlockScopeObject = {
               prop: 'dot accessor in inner block scope'
          };
          var innerBlockFunction = function() {
               return 'inner block function call';
          };

          try{
               return eval(name);
          }catch(e){
               if(typeof console != 'undefined'){
                    console.log(e);
               }
               return null;
          };
     };
};

var test = wrapped();

console.log(test('globalScope'));
console.log(test('outerBlockScope'));
console.log(test('innerBlockScope'));
console.log(test('globalScopeObject.prop'));
console.log(test('outerBlockScopeObject.prop'));
console.log(test('innerBlockScopeObject.prop'));
console.log(test('globalFunction()'));
console.log(test('outerBlockFunction()'));
console.log(test('innerBlockFunction()'));

Try out it and let me know!

The Evolution of a Startup: Web Technologies

This a detailed post describing Personal Capital’s transition from Flex to Backbone.js+RequireJS including the reasoning behind it and the experience of doing it.

By now, it’s clear to anyone who’s actively involved in the web space, including consumers, who’s come out on top in the Flash vs Rest of the World showdown. So, as Adobe transitions into marketing some HTML animation apps after ditching Flex and abandoning Flash Player for mobile browsers, it might be interesting to take a look at why on earth our web product is currently Flex based, and why it won’t be much longer.

Use it or lose it – your market share that is

Between 2009 and 2012 marked at least two dramatic shifts in popular technology we deal with day-to-day, which is not all that unusual for technology in general, but in terms of front-end web development it’s been fast and huge. These two big shifts I’m talking about are the decline of Internet Explorer on the one hand and Flash on the other.

Ironically, the same folks that are relegating both technologies are also now currently trading punches: Apple and Google.

Apple got real busy selling iOS devices and banning Flash from them while telling the world, every day, that they were doing it. Apple was also hard at work gaining an extra 10% PC market share with OS X which doesn’t run Internet Explorer at all. In terms of consumer perception, I’d say it was all Apple. But really, in terms of enabling this new generation of technology, Google’s Chrome and V8 teams deserve 99% of the credit.

For your consideration please:

That’s a pretty phenomenal eating of Microsoft’s lunch as Chrome goes from 0 to 34% and IE goes from 69 to 33% (and thankfully 17% of that to IE9). That leaves a big fat 2% for Firefox and Safari to play with. So there’s probably a million reasons why that played out the way it did, but in my mind, the single biggest factor that drove this current round of web innovation is the javascript performance war that Google’s V8 engine started.

Back in September of 2010, when we first started working on the Personal Capital web product you see today, the shiny new IE8 clocked in at 4989.7ms running the SunSpider javascript performance benchmark. This morning, Chrome on my Macbook Air clocked in at a respectable 243ms. The slope of the line on that graph is steeper than the financial cliff we stumbled off of in 2008. It’s the reason we can have smooth animations in javascript and data-tables that sort themselves in milliseconds. Chrome’s speed and fidelity is the reason we’re even talking about this at all, not to mention all the work the Chrome team has put in along with others to get HTML5 and CSS3 out the door. That’s huge. As Javascript developers in 2012 we’ve got great performance and a blossoming feature set.

In 2010 we had neither; which is the void that Adobe Flash Player filled. Although I can’t find updated numbers for Flash Player penetration today (surprise), the 2010 Flash Player 10.1 had 87.4% (more than IE) of the US browser market covered with a powerful VM that could execute and render at high fidelity at the same time. That March, Adobe also released Flex 4 which is a great framework with a lot of thought put into separation of concerns and mature patterns like state management, complex data collections and intelligent view (re)rendering. The icing, though, were the 4th gen micro-frameworks that sat along side it. Frameworks like Swiz and Robotlegs made AOP and IoC first class citizens in the Flash/Flex world and eliminated hundreds of lines of boilerplate and decoupled code in the process.

That’s why our web product is written in Flex. It allowed us to create a highly interactive browser-based product quickly, at high quality, to our design specifications and deliver it to almost everyone while building it on a solid software architecture. Until Adobe ditched it.

You know, one thing that Flex never really had going for it was a great open source developer community, and I’m not quite sure why that is. We had plenty of good examples from python to ruby to even the jQuery plugin ecosystem. So when it didn’t make sense financially for Adobe to continue to drive Flex, there wasn’t enough of a community there to catch its fall and there wasn’t enough of one there to prevent it in the first place.

So, in 2012, Flex has been dumped unceremoniously into the Apache Foundation’s lap and is being valiantly lipo-sucked by the die-hards in the Spoon Project. Flash Player has been relegated to a video streaming and gaming platform who’s currently trying to integrate OpenGL. Internet Explorer, for its part, might make a comeback as IE10 purports to finally be on par with modern browsers both in terms of performance and a standardized feature set but I doubt Google or Mozilla is interested in sitting around and letting that happen.

The takeaway here is simple and it’s something every developer knows in their heart: you’re never done until you have no more customers. Microsoft and Adobe thought they were done and they both used their dominance to excuse themselves from continually delighting theirs so Google and Apple came in and did it for them. And they did a damn good job.

But enough of the history lesson! Let’s dive into what we’re actually doing now with Javascript and HTML5 and where we’d like to go in the future.

Our Javascript App Design Refresh

We started converting parts of our web application from Flex to Javascript around September of 2011, first as a POC, then for production but just on a simple jQuery enhanced prototype based architecture. At the same time, we were doing the research necessary in order to completely rebuild our entire development stack from scratch. Not only were we looking to maintain use of architectural patterns like dependency injection, event mediation and data binding, but we also needed to figure out how we were going to test, package and ship our software.

Here’s the current setup:

Application Framework

Backbone.js provides just enough framework to be useful but not so much that we can’t cultivate our architecture in the way we see fit. We take advantage of the router, propagated model/collection updates and simplified view setup. We’ve extended collections to do smart merges from our API backend, implemented client-side data caching for data injection (think dependency injection but with data) and have a set of rules to determine how we break down designs into individual view instances. We compared all the frameworks currently listed on TodoMVC and were impressed with how the platform is evolving, but it came down to Ember and Backbone at the time. We like the Backbone community and the learning curve although we’re aware we’ll eventually experience some growing pains. We’re closely following projects like Aura, Marionette, Thorax, and Chaplin. A more detailed write-up of our comparison here warrants another post and we are currently also experimenting with AngularJS for some more advanced UI components.

Architectural Framework

RequireJS is not just a script loader, it’s really dependency injection and gives you nodejs-ish module encapsulation in the browser. Dependency injection is important for efficiency without singletons or long-winded model locator syntax, for isolating units under test and promotes greater code reuse because of the loose coupling. The RequireJS project has the best momentum out of the alternatives we evaluated in our opinion and their shim support iced the cake. Read more about it here.

View Templating & Layout

Handlebars is a clear favorite here because even though we started out using Twitter’s Hogan implementation of Mustache we found the extra bit of flexibility worthwhile to people experienced in developing complicated UIs. Handlebars’ registerHelper mechanism is also a much cleaner way to promote code reuse of view transforms than the function mechanism in Mustache proper. And given the fact that we aren’t going with a framework which enforces a view language like AngularJS or Knockout there really isn’t anything better for dead-simple view definition.

Visual Design & Styling

Compass (and SASS) was chosen because of our desire to empower our design team more in the actual implementation of products; our goal being to provide the HTML structure and JS behavior and let the more opinionated people perfect the pixels. Given the broad library of mixins that create cross-browser CSS and the fact that SASS doesn’t have any significant disadvantages compared to LESS or Stylus, Compass was a natural choice for us. Currently, developers use Scout for compilation but this will eventually be integrated into our build process.

Unit Testing

SinonJS with Chai extensions enables simple and powerful behavior expression. If you’re used to BDD testing in Ruby the patterns here will be familiar. It’s a library that takes full advantage of the language it is written as well as the experience of other communities in evolving test frameworks. You get to write easy to read unit tests for almost anything without much pain. Seriously, if you haven’t been writing unit tests up until now, go check it out.

Build Process

Hudson is our default continuous integration server because we were already using it. While there are other choices out there, it’s got a great community of plugins and it’s quite impressive how well it’s held up building, testing and deploying everything from ActiveX plugins, WAR files and Flex projects to our Javascript applications. We use the RequireJS optimizer to concatenate and uglify our applications prior to release.

Graphing

Raphaël could use some documentation love but that doesn’t take away from its excellence as a cross-browser vector graphics library. All the HTML graphs you see in our site are custom Raphaël plugins that are really close to the heart of our value proposition. We want to give you the best insight into your finances possible and oftentimes that manifests itself as a highly interactive, highly custom visualizations that we hope bring you happiness and prosperity. We also seriously evaluated Highcharts and the graphing extension to Raphaël but ultimately our passion for numbers (and a great product team that can manipulate them) drove us to pick something productive yet 100% customizable. [TODO: link to Justin’s Raphael post here]

A Quick Note About Debugging

The visual debugger in Flash Builder was the single biggest selling point for Flash Builder, but it required a debug version of Flash Player to be installed and Flash Builder (Eclipse) to be running. Yikes.

Now I want to take a moment to wholeheartedly thank the Firebug team for pioneering the in-browser debugging console back in 2006 and doing it right. These days the Chrome Developer Tools and Safari Web Inspector are top notch and even IE9+ has some semblance of a useful developer console which allows breakpointing, runtime manipulation and analysis AND is deployed on customer machines all without having to boot up a separate program. We’re even able to enable rendering hints in webkit browsers. This is one area in which I feel Javascript development is clearly superior to any other technology I’ve worked with.

Functional Testing

Unfortunately, we haven’t nailed this down yet and both our QA team and developers are still doing evaluations. If you have suggestions we’d love to get some in the comments below.

In Summary

You can see that our philosophical approach to all these areas is “just enough is enough” without having to buy in to an opinionated, vendor-locked or overly complex piece of technology. We’re staying low-level so that we can build out a coherent platform that suits our priorities and not spend a lot of time fighting frameworks. And there’s actually a very good reason for that.

Enough with the boilerplate!

So, you get it, we’re excited about what we can do and how we can do it in HTML and javascript. But here’s some cold hard truth: we lost important architectural patterns when making the move from Flex 4 and Swiz to Backbone. Namely:

  1. Data-binding
  2. An intelligent view component lifecycle
  3. Declarative views
  4. Event mediation

To get these back in you need to write a lot of boilerplate or use a more abstracted library on top of Backbone (which is fine). However, a couple of the newer frameworks attempt to implement items 1-3. Ember is actually designed around them and it feels like AngularJS and Knockout are in some part inspired by these features of Flex, but they are all pretty awkward in their own way. While Ember 1.0.PRE explicitly supports a declarative view hierarchy with {{outlets}} and the Objective-C style first responder system is nice, writing your own bindable EmberHandlebars helpers needs to be streamlined as does the syntax for declaring view actions. The overly complex interplay between the binding mechanisms and the models, collections and controllers make them time consuming to debug when they don’t play nice. The router is actually nice to write for though the heavy reliance on string literals for instance lookups and even for property sets/gets (which is an unnecessary recurring theme in JS frameworks) makes me sad as it’s more brittle. Angular, in its own way, is a lovely piece of engineering but wiring that much functionality into the DOM is bit adventurous, opaque and the project just wasn’t far enough long when we first started re-writing our apps. Knockout is in the same family.

We handle event mediation simply through an application event bus that enables modules like metrics and logging to peek at what’s happening in the running application without their code being strewn everywhere. It also allows modules to request the triggering of an action without needing to care about who actually performs it.

Regardless, we’ve been constantly impressed by all the thought and sweat that’s gone into each of the technologies we’ve talked about and as part of our maturation as a team and a company, we’d like to start to contributing back to this wonderful community. We’re currently pushing to get some of this foundational work on our roadmap. In the meantime, I guess we’ll keep writing our own boilerplate.

But wait! There’s more!

Projects We’re Super Excited About

  1. Meteor
  2. Derby
  3. AngularJS

If you’re excited about any real-time or other technologies that help create a fantastic user experience please let us know in the comments below, we like great technology and excitement.

Tech Shoutouts

Finally, we’d like to take some space to say thank you a few of the myriad other open-source projects and utilities that have allowed us to move as quickly as we have and helped us fall in love with the community. In case you haven’t heard of one of these, check them out, they might help you too. In no particular order: livestamp.js, lodash, slimScroll, momentjs, require-handlebars-plugin, express & jade, detectmobilebrowser, and of course, the jquery core team for doing such a good job pushing the edge for so long.