Databases. The creatures that often live somewhere in the cloud and hoard all of the data for your application. Although they can be troublesome to work with, they allow you to turn your isolated application into one that can share and communicate with the world.
There are many different options available, and different options are going to make sense for different people and different scenarios, but I’ve spent a long time looking for the one that “felt right” – the one that would serve well as a good default when building web-based mobile applications. This is how I felt about Ionic for building the front end of mobile applications, and it’s how I feel about CouchDB for the back end of mobile applications.
CouchDB is a document based NoSQL database that can easily be synced to other CouchDB databases. If you are unfamiliar with what NoSQL is all about, and specifically if you are not familiar with document based NoSQL databases, I would recommend reading An Introduction to NoSQL for HTML5 Mobile Developers first.
There’s many advantages to using CouchDB including the ease of which it can be scaled, and the speed of read and write operations, but the killer feature when it comes to mobile applications (and the reason I think it makes a great default) is its ability to synchronise between multiple databases. Synchronising two or more databases can be astoundingly difficult, and in order to provide offline functionality in an application, we need to provide local data to the user that can be read and modified, and then later synced back to the main database or databases when a network connection becomes available.
When using CouchDB, it is trivially easy to do this. A local database can be synced to a remote database with PouchDB (which is compatible with any CouchDB database, and which we will often be using) with a single line of code:
PouchDB.sync('mydb', 'http://url/to/remote/database');(you can also sync databases using CouchDB’s HTTP API directly)
All we need to do is supply the URL for the remote database. I’ve yet to come across another backend solution that so greatly simplifies such a complex task.
CouchDB even has an integrated web server with an HTTP API so you can interact with the database directly through its URL if you don’t want to include some kind of server middleware for your application (like Node or PHP).
CouchDB is simple and elegant, but it’s also strange and different. Initially, it’s going to be difficult to figure out how to build the things you are used to building (especially if you are used to relational databases like MySQL). Part of CouchDB’s philosophy is that it will make it hard for you to do things that won’t scale well.
I intend to create a series of tutorials on building Ionic 2 applications with CouchDB as the backend (I have already created tutorials on CouchDB in the past, but I’d like to do it more thoroughly), but first I want to cover a little theory. In this article, I will be introducing the general concept of CouchDB, how the data is structured, and how it is able to so effectively scale and provide effortless synchronisation features.
What’s PouchDB?
If you’re using CouchDB and you’re building mobile applications, then you will most definitely come across PouchDB at some point. PouchDB is a critical ally to CouchDB in the mobile world and will allow you to run a local instance of a CouchDB style database on the device that can automatically sync to a remote CouchDB database. This allows for effortless offline data synchronisation and we will most likely always use PouchDB locally when building an application with a CouchDB backend.
I will mostly just be talking about CouchDB in this article since I am just attempting to explain the basic concepts, but keep in mind that these two technologies will go hand in hand.
Data Structure
Data in CouchDB, like in many (but not all) NoSQL databases, is stored as documents and there is no pre-defined schema. Unlike a relational database where we need to describe what the structure of our data is before we insert it (a schema), and can only insert data that matches that schema, with CouchDB you can store whatever kind of data you want, whenever you want. This doesn’t mean that there is no structure to a CouchDB database, it is critically important that you add data in a way that makes sense and will perform well for your application. This creates a lot of flexibility for data storage, but it also has its downsides and it can be quite hard to learn because there is no “100% correct” way to store the data, and in many cases, it’s better to do things you will probably have learned to avoid like duplicating data.
A document in CouchDB is essentially just a JSON string that looks something like this:
{
"_id": "00a271787f89c0ef2e10e88a0c0001f4",
"_rev": "1-2628a75ac8c3abfffc8f6e30c9949fd6",
"item": "apple",
"prices": {
"Fresh Mart": 1.59,
"Price Max": 5.99,
"Apples Express": 0.79
}
}This is essentially how all document based databases work, it’s just a collection of these JSON strings. They are called documents because that is an analogy for what they are supposed to be – a document like an invoice, an email, or a receipt that contains all of the relevant data in one place. We can immediately see in the document above the various prices for apples at different stores, but in a relational database these would likely be split up into different tables and joined using a foreign key when necessary.
A document, like an email, can also have “attachments”. An attachment is essentially just another document that is related to document it is attached to (a song might be attached to the information for that song).
The main difference in a CouchDB database, when compared to other document based databases, is that it also stores a “revision” or _rev field for each document, but we will discuss why that is relevant in the next section.
Consistency, Revisions, and Offline Syncing
Lack of consistency is the reason for most of the advantages CouchDB provides – easy scalability, fast reading and writing, and synchronisation.
Consistency refers to the ability to keep a set of data in sync, meaning that wherever or whenever you access the data it is going to be a true reflection of what that data should be, and it might seem a little concerning on the surface that CouchDB does not treat consistency as a priority but instead takes an “eventual consistency” approach – the data will be consistent eventually, but you can continue to use the database whilst it is not consistent.
In reality, this is a tradeoff. An RDMS like MySQL prioritises Consistency and Availability over Partition Tolerance, whereas CouchDB prioritises Partition Tolerance and Availability over Consistency.
So, what does this lack of consistency actually mean? To give an example, let’s imagine a scenario that we will likely encounter later. Let’s say that we have a todo list application where two users are both trying to edit the same todo. We’ve set up offline syncing so we have three databases in play here:
- User One’s local CouchDB database
- User Two’s local CouchDB database
- A remote CouchDB database
Thanks to CouchDB’s synchronisation features, all three of these databases are in sync and contain the same data. Let’s also assume that both users are currently offline so their changes won’t be replicated immediately to the main database. If both users attempt to edit the same todo, what happens?
CouchDB isn’t worried so much about consistency, so it will let both users update their local database with the changes (without checking what the state of the data is in the remote database). As of right now, the two users have different sets of data, so the data is in an inconsistent state. When the users eventually come back online their changes will be synced with the main database, but clearly, only one of the users updates can be used to update the todo.
To deal with this situation, CouchDB assigns documents with a “revision” number, which is stored in the _rev field that we saw in the document example earlier in the example. In order to update a document in CouchDB you must first retrieve the entire document, modify it in any way you wish, and then send the entire document back. When sending the document back it is important that you also send the _rev field, and that the _rev on your document matches the _rev on that same document in the remote database. If the _rev fields match, CouchDB will process the update and increment the _rev number. If the _rev fields do not match, the update will be rejected.
What this means for our situation with two simultaneous users, is that whoever syncs their update to the remote database first will “win” and have their update accepted, because their _rev will match the _rev stored in the remote database. However, when the second person attempts to sync their data, their _rev field will no longer match because it will have been incremented because of the “winners” update.
Even though the update will be rejected, it won’t just be completely lost. CouchDB will keep that document and mark it with a special property "_conflicts":true, so you would be able to decide how to handle this scenario in your own application (i.e. do you just want to just ignore this change, or would you prefer to notify the user and let them decide?).
The benefit of this approach is that we can do really quick reads and writes in the local database (we don’t have to worry about going through the network to access the database), and we can easily add multiple instances of our database without having to worry about how we are going to handle communication between them.
Scaling
When it comes to scaling any kind of application, things can get a little complicated. If you are using a MySQL database for example, you can’t just fire up another server and copy your MySQL database over to it, send half of your traffic to one and half to the other. It’s a complicated problem with no easy solution.
However, CouchDB is designed to scale easily. I am not going to get into scaling strategies in this tutorial, but I wanted to highlight that you can, more or less, just fire up some more CouchDB instances to help scale your application.
Summary
The immediately noticeable benefit to CouchDB is that you can easily create mobile applications that work offline and sync when online, and the added bonus down the track is that your application is going to be a lot easier to scale, and potentially a lot faster.
Over the following weeks, I will release a series of tutorials that will walk through the basic concepts (and perhaps some more advanced concepts as well) of building backends for Ionic applications with CouchDB.