Thoughts about Proxying the HTML5 canvas

This post is a blackboard for my thoughts around how to write tests that verify what's drawn on an HTML5 canvas. I'm still relatively new to Javascript, DOM and Canvas, so if there's an established solution to mocking canvas operations, let me know. My research hasn't yielded anything.

As part of my recent involvement with Dygraphs, I've become interested in automated Javascript testing. I'm currently writing some very basic automated sanity tests for Dygraphs using jstestdriver, and those tests should be committed to the Dygraphs repository fairly soon.

Many of the tests I want to write have to do with the graph's internal state, and not its visual aspects, but at some point I will need to test what's drawn. Because Dygraphs has an HTML5 canvas, I have limited options:

1. Visual inspection. To some degree this is what we already do. Dygraphs has over 80 pages that represent visual tests. Most tests require a quick glance, but some require playing with values on the page to see how the rendering changes. (This last problem is something I've contributed to.) And finally we just can't look at every single test with every release. Besides, we're looking to create automated tests, so let's move on to the other choices.
2. Perform pixel-perfect tests against golden images. This works well when setting up tests, but the lack of a consistent canvas implementation across all browsers means that I should expect to have to create a golden image per test per browser version. This type of test can also be a problem when making small changes (such as minor tweaks in color and so on) require generating new golden images across the board, which comes with its own problems.
3. Perform fuzzy-match tests against golden images. This addresses many issues, but I still need to generate golden images. Not ideal.
4. Pixel count. This isn't my idea; someone else suggested it. This could theoretically work, but it seems complicated. "Assert that the canvas has 50 blue pixels" reads less like an effect and more like a side-effect.
5. Implement a fake canvas of some kind. This is the type of solution I want to implement. Replace the canvas with some object so I can verify that a line is drawn from a to b by writing something like assertSegment(point(0, 0), point(50, 50));.

What I really want is a proxy that not only receives calls from a client and logs them for post-operation validation, but also actually draws on a real canvas. That way in the event of a regression I can view the backing canvas, which will serve as a clue to what changed. It also helps me understand just what kind of test is being written in the first place.

Today I set about to prove out the idea of a canvas proxy, and the truth is, it's just not going to happen the way I want. Here are the two showstopping problems I see so far:

1. You can't fake properties. (You really can't do that with Java and EasyMock either, but in Java you don't often see statements like canvas.style.top = top;.
2. The proxy has to be a DOM element. Otherwise calls like document.appendChild(canvas) will fail. HTML canvas elements, in the end, are just DOM elements. The Proxy is not. I could start by creating a DOM element, and add methods to it (such as getContext()) but while my experience with faking the DOM is limited, my intuition suggests that's a dead end.

The DOM element problem can be mitigated by faking not the canvas, but the rendering context that comes from calls to getContext('2d'). I always expected to need to fake out the context. This might work, but we will still have trouble with properties.

context.strokeStyle = "#000000";
context.fillStyle = "#FFFF00";
context.beginPath();
context.arc(100,100,50,0,Math.PI*2,true);

It may be possible to mitigate the properties problem with a hack that keeps properties in sync between the proxy and proxied object. For example:

ProxyContext = function(proxied) {
  this.__proxied = proxied;
  this.__copyIn();
}

ProxyContext.prototype.__copyIn = function() {
  this.strokeStyle = proxied.strokeStyle;
  this.fillStyle = proxied.fillStyle;
  ...
}

ProxyContext.prototype.__copyOut = function() {
  proxied.strokeStyle = this.strokeStyle;
  proxied.fillStyle = this.fillStyle;
  ...
}

ProxyContext.prototype.beginPath = function() {
  this.__copyOut();
  proxied.beginPath();
  this.logCall("beginPath");
  this.__copyIn();
}

This could work. I'm pretty sure it could work. Hey Internet, why won't this work?

I fear that a decent solution requires an abstraction layer that turns all mutations on the context into functions (e.g. context.strokeStyle becomes context.setStrokeStyle().) Is that good Javascript practice, or is that just not the Javascript way?