Last week, Personal Capital released its Android app to the Google Play store. You can read more about it here. Developing quality apps that perform well requires more than just writing code. The Android SDK comes with a number of excellent tools for debugging and testing apps. While simple to use, they can expose some of the harder-to-find bugs and reveal opportunities for improving your code. Here are some highlights of the things we get lots of mileage from:

Eclipse

While not specifically an Android tool, Eclipse is a valuable one nonetheless. Android apps can be run with the standard Java debugger just like any Java project, and Eclipse’s debugger is no slouch with conditional breakpoints, exception breakpoints, drop to frame, and other neat features. Most of the SDK tools integrate into Eclipse as well, so all of your tools are in one place.

Logcat

Logs are especially useful when trying to reproduce a bug, both to give you a starting point for your investigation and to confirm that you’ve accurately reproduced an issue. Even when you’re not looking for anything specific there, watching the device log can be surprisingly instructive.

Reading the Logcat can be done with the Android Device Monitor tool in the Android SDK. You can start it from a command line by running <android_sdk_path>/tools/monitor. Alternatively, you can open the Eclipse perspective called DDMS to access the same features.

Select any connected device or running emulator from the Devices tab, then switch over to the Logcat tab to see the logs. There will probably be lots of messages flying by, so put part of your application’s package name in the filter bar to increase your signal-to-noise ratio. You can also filter out messages below a certain priority level by selecting from the dropdown menu. The save button in the upper right of the pane will export selected lines to a file.

Logcat with filtered messages

Logcat with filtered messages

Suppose you get a crash while running your app and you aren’t connected to a computer, or you don’t have Device Monitor running. Is there any way to retrieve those logs? Indeed there is: in your <android_sdk_path>/platform-tools/ directory is a tool called ADB. Connect your device and run the following command to dump all the log messages to a file:

$ adb logcat -d > log.txt

Of course those log messages won’t last forever, so grab them before they get pushed out of the log buffer.

Screen Capture

Yo dawg, we heard you like screenshots…

Yo dawg, we heard you like screenshots…

In case your device doesn’t have screenshot capabilities, or to spare yourself having to transfer screenshots form your device to your computer, Android Device Monitor has a screen capture utility built in.

In Device Monitor (or DDMS in Eclipse), select a device or emulator from the Devices tab and click the icon near the top. A window opens up showing a preview of the device screen, which you can save as a PNG. The preview does not update when you interact with the device; you have to click the Refresh button to reload the preview.

Hierarchy Viewer

Speaking of UI, occasionally you get some layout params wrong and a View (or, you know, half the View tree) mysteriously vanishes into nothingness. The last time this happened to me, it was in a part of the view hierarchy I certainly didn’t expect, and I never would have found it without this tool.

Hierarchy Viewer is included in Device Monitor, or you can open it as a perspective in Eclipse. In the Windows tab, select the window corresponding to the application you are debugging (the active window will be bolded). Once the View tree loads, clicking an individual View will show you a preview of its appearance on screen. Where that View is positioned on screen can be seen in the Layout View tab, and you can inspect various attributes in the View Properties tab.

A View tree in Hierarchy Viewer

If you’ve seen our app, can you guess which screen this is?

If a View that should be on screen is not visible and you do find it in the View tree, check the layout and measurement properties and see if its width or height is inadvertently being set to zero. This can happen for a number of reasons, some potentially obscure, like using the wrong anchor in a RelativeLayout or not setting the alignWithParentIfMissing attribute.

TraceView

TraceView is a profiling tool to help you determine areas of code that are performance bottlenecks. It can also alert you to operations on the UI thread that should be moved elsewhere. Due to technical limitations and to the latency it adds, method tracing is not something you would leave running for the duration of testing. Instead, you should start tracing just before performing some action of interest, and then stop tracing just after.

The easiest way to use TraceView is through Android Device Monitor (or DDMS in Eclipse). While your app is running, select the process in the Devices tab. Click the  icon at the top of the pane to start tracing, interact with the app for a bit, and click the icon again to stop tracing. TraceView will open the trace data for you to inspect.

TraceView at a glance

TraceView at a glance

The top pane will show a timeline of thread activity, while the bottom shows profile information for all method calls that took place. In the timeline, you can zoom in and out with your mouse scroll wheel, or click and drag a horizontal region to zoom in. Pay particular attention to the Main (or UI) thread; any unusually long gaps in the timeline could indicate something making your app unresponsive.

The bottom pane shows all the method calls and reports data regarding execution time (real time or percent of total), both exclusive and inclusive. Exclusive time is only the time the method was executing instructions; inclusive time also counts calls to other methods. You can drill down through method calls and sort them by any column. Sorting by exclusive real time, for instance, could help find particular methods that are taking the most time to process.

Drilling down in method profile

Drilling down in method profile

Wrap Up

Sometimes the simplest thing—screenshots, exported logs, etc.—vastly improves our ability to turn around a bug posted to our issue tracker. Other times, you need comprehensive measuring tools to pinpoint a critical flaw. Whatever platform you are building for, tools like those packaged in the Android SDK are invaluable. Even for non-developers, these tools can be leveraged to help understand the behavior of the app and provide better feedback for developers.

If you are already using these tools in your development process, great! Consider introducing all of your engineers to them as well. Empower others to explore, get their hands dirty, and perhaps engage in the feedback part of your development loop; they’ll probably appreciate the knowledge you’ve shared. And don’t count out your client base either—believe it or not, even some of our end users send us logcat output on occasion.

 

We have a simple engineering philosophy: our user experience has to be the absolute best, and our calculations 100% accurate, while everything else just needs to be “good enough” for “now.” This is why we can release high quality products early and fast. Behind the scenes, however, we are continuously refactoring our code base as “good enough” changes over time.

Our November release went live with a new HTML5 dashboard and account detail screens, as well as many interaction, performance, and refactoring upgrades for the web application as a whole. Here is how we did it:

Distributing Static Content We now serve all web application assets packaged and minified from Amazon’s CloudFront. This helps us serve our static content (js/css/media) to end users with low latency and high data transfer speeds. It also helps us separate the concerns of static content (front end) and server webpages (back end), which reference the static content. This gives us the ability to change static content at any time without having to re-deploy our entire application. Faster iterations. Better products. You can read more about it here.

Responsive Design One of the biggest variables in web interface engineering is client screen.  Our new HTML5 dashboard uses CSS media queries to offer a responsive design, rendering a single column for common resolutions, and two columns for wide resolutions. This wider resolution provides an efficient use of space and access to interface elements without the need to scroll. This gets us one step closer to providing that “financial picture at a glance” goal.

Financial Visualizations These are the interface embellishments that bring our web application to life.  We chose Raphael.js, a vector drawing API which makes programming SVG with javascript a pleasure.  We took this customized approach, rather than an out-of-the-box charting solution, because at Personal Capital, the defaults just aren’t good enough. We work very hard to design the most effective charts we can, and even harder to make sure that when we release them they live up to customer expectations. New visualizations featuring custom algorithms can be seen on the new dashboard and account management screens.

Datagrids This is the bread to our visualization butter. In one month, we switched over to a custom HTML5 implementation (from our previous Flex-based implementation) improving not just performance, but the set of features as well. Searching, sorting, and editing are all upgraded and we added a couple new features like multi-row editing and tag sorting. Angular.js is the technology that allowed us to achieve these gains in such a short period of time. Angular makes re-rendering based on live updates a non-issue, as the markup itself clearly and concisely documents what it will express as the data changes.

Polymorphic Backbone Views: Backbone framework has proven to be both scalable and fast. On the scalability side, the ability to extend and polymorph Backbone’s views helped tremendously in simplifying the code for Account Details. On the performance side, we were able to greatly reduce the number of unnecessary HTML redraws by writing custom functions that would update specific DOM elements.

That’s it for the new web app updates this month. Stay tuned for the next round of web app improvements in January!

why

  • Distribute content to end users with low latency and faster response times.
  • Ability to deploy new static content without having to re-deploy the entire application.

what

  • Identify a Content Distribution Network (CDN) solution that meets our needs
  • Setup origin server(s) to store the content that will be distributed via CDN.
  • Setup a distribution to register the origin server(s).
  • Reference all static content in the application with the distribution’s url.
  • Version static content to bust the cache both in the browser and CDN.

how

Before we get into implementation details, what do we mean by “static content”?  Here, we define static content as all the web resources that are not specific to a particular user’s session.  In our case, this includes css files, javascript files and all media files.

Identify CDN solution

Since most of our infrastructure is hosted in Amazon Web Services, the AWS CloudFront CDN was a logical default choice to try.  Like most of the AWS application-level services, CloudFront tries to hit a reasonable balance of the 80/20 rule:  it is low-cost, easy to operate, and covers most basic functionality that you want in a application service, but does not offer much in the way of high-end differentiating features.

Setup origin server(s) to store the content that will be distributed via CDN.

An origin server in CloudFront can be either a web server, or, as in our case, an AWS S3 bucket. We have setup an origin for each of our environments (dev, qa, prod). We then have a Hudson job (build) for each of our environments. The Hudson job checks out the corresponding branch from our git repo for the static content; processes it as mentioned in this post; updates a version.xml file with the Hudson build number; zips the file; and copies it to a S3 bucket. This bucket, however, is not the CloudFront origin. It is like a docking area to store all zipped build files from our various Hudson jobs. We have another Hudson job (deploy) that copies a given zipped build file to an origin bucket. More on that in a minute.

We have a different Hudson job for each environment because the static content could be processed differently based on an environment. For example: in our dev environment, we do not combine or minify our js files. In our qa environment, we combine but do not minify our js files. In prod, we combine and minify our js files.

Back to the Hudson deploy job mentioned above. This job takes two input params: the name of the zipped build file to be deployed and the environment to be deployed. It simply unzips the build file into a temporary directory and uses s3sync to upload the content to the appropriate S3 origin bucket for the given environment.  And from there, it is available for distribution via CloudFront.

In addition, in our dev environments, we use a Continuous Integration (CI) process, where our Hudson jobs send SNS messages to our the web server when a build is available.  The web servers pulls the static content build from the S3 staging bucket and then serves it directly from Apache.  This allows for a more targeted integration test of the static content without bringing the CDN mechanisms into the mix.

Setup a CloudFront distribution to register the origin server(s).

We have a CloudFront distribution for each of our origins.  A CloudFront distribution associates a URL with one (or more) origin servers.  Creating the distribution is easy via the AWS Console.  In our distribution, we force HTTPS access – since our containing pages are served via https, we want to ensure the embedded static content is as well, to avoid browser security warnings.

CloudFront does allow you to associate a DNS CNAME with your distribution, so that you can associate your own subdomain (like static.personalcapital.com) with your CloudFront distribution.  This is more user-friendly than the default generated CloudFront domain names, which are like d1q4amq3lgzrzf.cloudfront.net. However, one gotcha is that CloudFront does not allow you to upload your own SSL certificate.   So, you have to choose between either having your own subdomain or having https – you can’t have both (some other CDN services do allow you to load your own SSL certs).   In our case, we chose https, and use the default cloudfront.net URL.

Reference all static content in the application with the distribution’s URL.

We have an environment config property that stores the corresponding CloudFront distribution’s URL. This property is available for all server-side webpages where it is referenced as follows:

 <link rel="stylesheet" type="text/css" href="<%=staticUrl%>/static/styles/css/main.css">

We then needed to make sure that all our static content references its resources via relative URLs to keep them independent of the distribution URL. For example, an image reference in main.css would be as follows:

 background: url('/static/img/dashboard/zeroState.png')

which would resolve to the root URL of main.css which is the distribution URL. I would like to know if there is a better way to solve this.

All our javascript uses relative paths anyway because of “requirejs” so we did not have to make any changes there.

All other references to static resources were on the server side where they had the config property to correctly reference the static resource.

Version static content to bust the cache both in the browser and CDN.

All our static content have aggressive cache headers. Once a static resource is fetched by the browser from the server, all future requests to that resource will be fetched from browser’s cache. This is great but when a newer version of this resource is available in server, the browser won’t no know about it, until its cache entry for that resource expires.

To prevent this, we use a common technique called URL fingerprinting wherein we add a unique fingerprint to the filename of that resource and change all its references in the webpages (JSPs) with the new filename. The browser, as it renders the updated webpage, will now request the resource from the server since it treats it as an entirely a new resource because of its new name.

The Hudson build job mentioned above processes our static resources, versions them with the build number and also stores the build number in version.xml. The version.xml file is then used by the application to retrieve the version number and pass it onto web pages at run-time. This helps us achieve our second goal that of keeping our static (front-end) development independent from our sever (back-end) development. This is very powerful as it gives us the ability to change our static content any time, have it deployed to production and not worry about updating server webpages with the latest version number. Pretty neat ah !!

Versioning of the resources also helped us out a great deal with our CloudFront distribution. CloudFront distribution behaves very similar to how browser handles resource caching. It does not fetch a newer version of the resource from the origin server unless one invalidates the current resource in the distribution. This has to be done per resource and it has a cost too.  The CloudFront documentation offers more considerations regarding invalidation vs versioning.

There is one other workaround you could use to force CloudFront distribution to fetch the content from its origin server. Set the distribution to consider query strings in the resource urls. And then pass a unique querystring along with the resource url and it will force the distribution to fetch the resource from the origin server.

That is it !!

reads

Personal Capital Engineers have ambitious vision, and as a friend put it, we’re always pregnant with more projects than we have engineers for. Our latest delivery is Personal Capital Android App.

Like our iOS app, the Android app is filled with interactive custom components and visualizations, from the rotating pie chart for cash management to the asset allocation tree map. Our philosophy in both design and implementation is to create custom native components tuned to the each device’s unique attributes to allow for an intuitive design and controlled behavior across varying device resolutions

In our Android app, we use many of the design concepts and lessons learned from our earlier development of our iOS app. For example, we implemented the same unique navigation system in Android that we use in iOS – it employs the concept of live views to speed up wayfinding for the user. We will be talking about some of these “shared clients that don’t share codes” concepts in our meetup as well.

If you haven’t downloaded the app yet, you can do so here: https://play.google.com/store/apps/details?id=com.personalcapital.pcapandroid

Watch this blog for more posts around our experience with Android Development.

Automation support for iOS apps is still a relatively new technology and like all other technologies that are still quite not mature it has a lot of challenges. Below I would like to discuss one of such challenges, identifying and accessing individual custom cells inside a UITableView

Let’s say we create a new phone book application and let’s say that the 3 pieces of information that this application stores are: the last and the first name of the contact along with the type of contact that this person is (family, friend, work). The contact type would be displayed in a non-text form (an icon for instance). Let’s also assume that there would be a screen that lists all the contacts that we have in our phone book and that is the screen that we are going to test.

As of right now every table cell is identified via all the text field values augmented inside of them. For example, a cell for John Smith can be accessed via the following statement:

target.frontMostApp().mainWindow().tableViews()["Empty list"].cells()["John Smith"];

The problem with this identification scheme is that potentially it lacks uniqueness. if we have 2 John Smiths we would not be able to distinguish between the two cells. Moreover, what if one John Smith is classified as family while the other one is a friend, we cannot query how these contacts are classified since these values are non textual

This is when accessibilityLabel property comes to our rescue. In our example, we want to encode 2 non-textual values in the accessibilityLabel. By using the following formula to generate accessibilityLabel for {record unique id},{contact type description} not only do we generate unique identifiers for the table cells but we can now query the contact type of the corresponding record for the cell. In our example the two accessibilityLabel values for the 2 John Smiths (assuming their unique record ids are 1 and 2 respectively) could be accessed via the following statements:

   target.frontMostApp().mainWindow().tableViews()["Empty list"].cells()["1,family"]; 
   target.frontMostApp().mainWindow().tableViews()["Empty list"].cells()["2,friend"];

The native code for creation of cells would look something like this

-(UITableViewCell *) tableView:(UITableView *) tableView cellForRowAtIndexPath:(NSindexPath *) indexPath  
{ 
   UITableViewCell * res = … 
   Contact * c = … 

   //initialize an instance of UITableViewCell 
   //get an instance of Contact that corresponds to the indexPath 
   NSMutableString * accessibilityLabel = [[NSMutableString alloc] init]; 
   [accessibilityLabel appendFormat:@"%d", c.contactId]; 
   if(c.contactType == ContactTypeFriend) 
   { 
      [accessibilityLabel appendString:@",friend"]; 
      res.imageView.image  = [UIImage imageNamed:@"friendsIcon"]; 
   } 
   else if(c.contactType == ContactTypeFamily) 
   { 
     [accessibilityLabel appendString:@",family"]; 
     res.imageView.image  = [UIImage imageNamed:@"familyIcon"]; 
   } 
   else 
   { 
     [accessibilityLabel appendString:@",work"]; 
     res.imageView.image  = [UIImage imageNamed:@"workIcon"]; 
   } 
   res.acessibilityLabel  = accessibilityLabel; 
   [accessibilityLabel release]; 

   return res; 
}

If we want to access a cell for a contact with a known record id but unknown status we would need to write some additional JavaScript code

function findCellByRecordId(recordId)
{
   var cells = target.frontMostApp().mainWindow().tableViews()["Empty list"].cells();
   for(var i=0; i &lt; cells.length; i++) 
   { 
      var cell = cells[i]; 

      //parse the name and check the first part 
      var cellNameParts = cell.name().split(","); 
      if(cellNameParts[0] == ("" + recordId)) 
      { 
         return cell; 
      } 
   } 

   //at this point we did not find anything 
   return null; 
}

Now we have means to locate cells based on contact unique record id, we would want to have an ability to see query what kind of contact type this cell shows

function getRecordTypeFromCell(cell)
{
   return cell.name().parts[1];
}

Now that you can access cells based on the unique record id of the contact that these cells represent as well as the ability to query the status of the contact, you can start writing automated tests for this screen. Happy unit testing!

Note: download sample Phone Book project