Offline storage is a moving target in web application development. IndexedDB is becoming commonly accepted, however WebSQL is still all you’ve got when it comes to running a database on the client. When I wrote the Elephant Prototype I built it entirely with IndexedDB because I wanted to learn how to work with it on a large scale application, and seeing as how the standard was gravitating towards it I may as well get in front of the game.
I achieved my goals but being my first time building a non-trivial Javascript application rather than just a jQuery AJAX et al implementation, I made a few decisions that weren’t long term maintainable. The major ones being the API I wrote was entirely IndexedDB specific, and the fact that it was entirely done using callbacks, which became remarkably messy when it came to more complex queries for which the standard IndexedDB API is insufficient.
With that, I knew that I was going to have to make a serious improvement on my approach to offline storage if I was serious about this whole offline web application thing. Local and Session Storage is adequate for simpler, easily JSON serialised data, but for a real application with genuine relationships and all that good stuff, it’s just not the business.
Investigating My Options
In getting started with my recent project, Pomodoro Go!, I immediately wanted to tackle this problem. I tried to use JayData which claimed full support for IndexedDB and WebSQL, however I came up against two solid roadblocks very quickly:
- JayData does not handle relationships by itself, so it’s value was already diminishing to me.
- JayData was already having issues for me in some fairly trivial examples involving IndexedDB, mainly with race conditions causing things to just not work.
After trying that and getting disillusioned, I had a look at other solutions around and they were either not supporting IndexedDB yet, or trying to reinvent storage with their own layer of stuff. All I wanted was a unified API that could use WebSQL or IndexedDB as it was designed, without the headaches, but I couldn’t seem to find it.
Enter JohoDB
With that I gave up on some saviour library that would make my life easier, and decided to write my own library now called JohoDB that would handle either WebSQL or IndexedDB using an API similar to Django’s Object Relational Mapper. It would address the following requirements:
- Same output and input regardless of database.
- Decoupled development of the main API and the database API, so I could respond to new database technologies in the future without a major rewrite.
- Decoupled usage from the application specific code, to allow for a high level of flexibility. Your models don’t have to have a .save() method like in Django, rather you send your object to the API for it to get saved.
- Promise driven rather than callback driven, to make the asynchronous nature of Javascript storage much less of a headache, as well as allowing the developer a much greater level of control.
- Leverage existing technology without reinventing the wheel. The standards are standards for a reason, and JohoDB at the base level should always use those standards as they were intended rather than trying to create it’s own storage layer.
With that in mind, I set to work and now have something to show for it. While not at all production ready I’m quite confident in meeting all the above challenges with this: JohoDB. The website linked demonstrates the API from top to bottom, I’ve implemented Foreign Key and Many To Many Relationships, and it works in both WebSQL and IndexedDB. It’s been open sourced so feel free to check out the source code.
I’m really excited about this right now, currently I’m working to build up a good suite of tests for it and just build inertia to something that is production ready, but it’s certainly already looking bright from my perspective.
Unemployed Code 
The database API is elegant and lovely.
Regarding foreign key implementation, how key constraint are enforced? Do you check consistency in creating and deleting of the record?
Cheers for the comment, appreciated.
Right now there’s no real enforcement in Foreign Keys at a database level, IndexedDB doesn’t have features for it and the WebSQL implementation doesn’t yet leverage it. That said, on deletion of an object that is the subject of a ForeignKey Relationship, it will automatically update the related objects.
In this model you can see the ‘house’ field:
db.addSchema("Person", {
id: {type: "string", primaryKey: true, autoHash: true},
firstName: {type: "string", required: true},
lastName: {type: "string", required: true},
age: {type: "int"},
house: {type: "fk", relation: "House",
relatedName: "residents", onDelete: "setNull"}
});
See that one of the options is onDelete: “setNull”, at the moment this can be either ‘setNull’ or ‘cascade’, which is analogous to what you’d see in SQL ON DELETE.
It would be possible to validate/enforce foreign keys on saving, basically I’d just need to put in a validation promise that checks ahead of time. I’d have to do this regardless of what SQL can provide, again because of IndexedDB. Not implemented yet but certainly makes sense to do.
Thank you.
In your example, `House` belongs to `Person`. So how House primary key is available when creating a person. A little example is appreciated.
Arr, i see. House persist first and then Person.
That’s it, in my current implementation if the house was created with an array of ‘occupants’, that array, even if it didn’t have primary keys already, would automatically get set up with the foreign key of the house.
The reverse is not currently true, if you created a Person with a foreign key relationship to a house, that house is expected to already exist. Anyway, part of the road to v1.0.0 is having good documentation so the way JohoDB handles all this stuff is easily understood.
Awesome! Thanks for putting in the hard work to create this 😀
Also noticed you’ve got some typos on the project website: your filtering examples use “{suburb=”Hobart”}” instead of “{suburb: “Hobart”}” 🙂
Haha, cheers for that. Seems the problem with writing an API similar Django’s is I’m mentally going to slip into that syntax if I’m not careful. 😀