Testing JohoDB with Jasmine

I’ve never really engaged in much unit testing with Javascript, as a solo developer I pretty much have to spend a lot of time optimising for years of my life implementing a given application. As Javascript was typically easy to observe when used only for front end effects, so I saved my testing rigour for the data layer of my web apps, where all the insidious stuff can happen. Not ideal but it’s the best I can do given current resources.

Testing the JohoDB application using Jasmine

Of course, when writing libraries it’s a whole different game. With JohoDB you’re going to be deploying to a moving specification, with new technology and a wide variety of browsers. There’s no option but to test every feature if I’m serious about going live with it. I pretty quickly went with Jasmine, and have been able to customise it’s output very easily to get what I wanted out of my test page.

Here’s some features that come out of the box with Jasmine:

• Create your test suites in a hierarchical structure to make it easier to follow.
• You can click on individual test suites to filter it down, speeding up testing when you’re  working on a particular feature. In my case I customised it a little so some ‘setup suites’ would always run.
• Each error come with messages and a traceback to help you get started on the bug hunt.
• Asynchronous functions can be given a time limit, again with a message so you can figure out where the chain broke.
• Everything is run through functions, so if  you need to run a test suite multiple times with different input you can do so easily. In my case I have a function with tests, and then it gets passed the IDB database, and then the SQL Database. That way I know both are being tested with exactly the same API calls.

Jasmine is something you should seriously look into if you’re writing non-trivial applications in Javascript. It has very quickly enabled me to clean up some portions of my code in JohoDB.

To be specific I’ve now got the database encapsulated properly so I can run the two side by side, no dramas, and delete / rebuild the database for every single test suite. It really is a stress test for how the promise system is holding up to keep the API manageable, and it’s going great so far.

There’s still more work to do, but once the existing library is well tested, It’ll be time to build up features so I can reach a version to test in an actual application. I can’t wait.

Why I Decided to Invent the Wheel – Browser Databases

Offline storage is a moving target in web application development. IndexedDB is becoming commonly accepted, however WebSQL is still all you’ve got when it comes to running a database on the client. When I wrote the Elephant Prototype I built it entirely with IndexedDB because I wanted to learn how to work with it on a large scale application, and seeing as how the standard was gravitating towards it I may as well get in front of the game.

I achieved my goals but being my first time building a non-trivial Javascript application rather than just a jQuery AJAX et al implementation, I made a few decisions that weren’t long term maintainable. The major ones being the API I wrote was entirely IndexedDB specific, and the fact that it was entirely done using callbacks, which became remarkably messy when it came to more complex queries for which the standard IndexedDB API is insufficient.

With that, I knew that I was going to have to make a serious improvement on my approach to offline storage if I was serious about this whole offline web application thing. Local and Session Storage is adequate for simpler, easily JSON serialised data, but for a real application with genuine relationships and all that good stuff, it’s just not the business.

Investigating My Options

In getting started with my recent project, Pomodoro Go!, I immediately wanted to tackle this problem. I tried to use JayData which claimed full support for IndexedDB and WebSQL, however I came up against two solid roadblocks very quickly:

• JayData does not handle relationships by itself, so it’s value was already diminishing to me.
• JayData was already having issues for me in some fairly trivial examples involving IndexedDB, mainly with race conditions causing things to just not work. 

After trying that and getting disillusioned, I had a look at other solutions around and they were either not supporting IndexedDB yet, or trying to reinvent storage with their own layer of stuff. All I wanted was a unified API that could use WebSQL or IndexedDB as it was designed, without the headaches, but I couldn’t seem to find it.

Enter JohoDB

With that I gave up on some saviour library that would make my life easier, and decided to write my own library now called JohoDB that would handle either WebSQL or IndexedDB using an API similar to Django’s Object Relational Mapper. It would address the following requirements:

• Same output and input regardless of database.
• Decoupled development of the main API and the database API, so I could respond to new database technologies in the future without a major rewrite.
• Decoupled usage from the application specific code, to allow for a high level of flexibility. Your models don’t have to have a .save() method like in Django, rather you send your object to the API for it to get saved.
• Promise driven rather than callback driven, to make the asynchronous nature of Javascript storage much less of a headache, as well as allowing the developer a much greater level of control.
• Leverage existing technology without reinventing the wheel. The standards are standards for a reason, and JohoDB at the base level should always use those standards as they were intended rather than trying to create it’s own storage layer.

With that in mind, I set to work and now have something to show for it. While not at all production ready I’m quite confident in meeting all the above challenges with this: JohoDB. The website linked demonstrates the API from top to bottom, I’ve implemented Foreign Key and Many To Many Relationships, and it works in both WebSQL and IndexedDB. It’s been open sourced so feel free to check out the source code.

I’m really excited about this right now, currently I’m working to build up a good suite of tests for it and just build inertia to something that is production ready, but it’s certainly already looking bright from my perspective.