An Introduction to NoSQL for HTML5 Mobile Developers

If you’ve been hanging around in web land then there’s a good chance you’ve at least heard of NoSQL. It’s a broad term that can mean a lot of different things, but at its core it refers to a database that isn’t a typical relational style database (like MySQL, MSSQL, PostgreSQL and so on).

For reasons I will discuss later in this post, I think an understanding of NoSQL is a great asset for HTML5 mobile developers, so we are going to spend some time learning the concepts.

Over the coming weeks I will be creating tutorials on how to make use of different NoSQL databases in Ionic 2 applications, including:

MongoDB
Redis
CouchDB
and more…

but before we dive in the deep end I figured we should cover the basic concepts, and broadly cover what “NoSQL” is. I’m approaching this as someone with quite a bit of experience with relational databases (which I’ve been using for around 6 years), but who is reasonably new to NoSQL. I’ve been deep diving into NoSQL lately but am by no means an authority on the subject. This positions me well to cover the basics as the conceptual struggles are still fresh in my mind, but we won’t be covering anything too advanced (not yet, at least).

The Difference Between Relational Databases and NoSQL Databases

A relational style database stores data using a table structure, and has a predefined “schema”. A schema defines the structure of the tables, and the type of data that will be stored. In MySQL for example you might define a “schema” for a table like this:

CREATE TABLE Cars (
	id INT(6) AUTO_INCREMENT PRIMARY KEY,
	make VARCHAR(30),
	model VARCHAR(30),
	year INT(6),
	purchased DATETIME
)

We’re not storing any data here, we’re telling MySQL how we want to store our data. So when inserting a some data into this database (a row in the table) we must supply the make and model as a string, the year as an integer, and the date the car was purchased in the DATETIME format. After adding some data to the Cars table it might look like this:

If we then also wanted to store some additional information, like the engine type of the car, we would first need to modify our schema to include that. Not many relational databases have just a single table though, the reason we refer to them as “relational” databases is because data in one table can be “related” to data in another table.

We might also have a Customers table with information about the customers, and those customers may own one or more of the cars. This structure might look something like this:

We can tie information together using the id in another table, by doing this we are able to perform queries that join data from different tables together to get the results we want. Given the example above, we could easily join the Cars and Customers table to show that Josh has purchased 2 cars, and Dave has purchased 1. This is one of the great strengths of traditional relational databases, in that they are designed specifically to allow for complex queries like “find all customers in Australia who have purchased a Toyota”.

NoSQL on the other hand has no predefined schema and does not store data using related tables. NoSQL is not one specific thing, but in general a NoSQL database is not relational, and does not have a structured query language (although even this is not strictly true for all NoSQL databases). That’s not to say you can’t run queries against NoSQL databases, you can, just not in the way you would be used to if you come from a SQL background.

Since NoSQL databases do not have a schema there’s not really any work involved in “setting up the database”, you simply just add your data. This makes them flexbile, in that if you want to make a change to the way you store data down the track it is very easy to do.

This does not mean that a NoSQL is just a bucket you can dump your data into with no thought given to its structure. How you store your data will depend on the type of NoSQL database you are using, but a well thought out structure can greatly improve performance and the ease of which you can retrieve data and perform queries against your NoSQL database.

Types of NoSQL Databases

There are many different types of NoSQL databases that we can use, including:

Key-Value (redis, Amazon DynamoDB, memcached)
Columnar / BigTable (HBase, Cassandra, Amazon SimpleDB)
Document (CouchDB, MongoDB, Riak)
Graph (Neo4J, AllegroGraph, InfoGrid)

We will primarily be looking at two types of NoSQL databases: Key-Value and Document. There are both very similar in nature, but there are differences.

A Key-Value database is a simple way to store values indexed by a key. You would already be familiar with this concept if you have used the browsers local storage or Ionic’s own SqlStorage. To use the browsers local storage as an example, you would set a value on a key like this:

localStorage.setItem('name', 'Josh');

and then you could later retrieve the value by using the key like this:

localStorage.getItem('name');

A document database is similar but stores “documents” instead. I think this terminology is quite confusing because as a beginner when you think of a “document database” you would probably think of a database that stores its data using files. This isn’t the case though, a document in this context is simply a JSON object like this:

{
    _id: 1,
    name: 'Josh',
    country: 'Australia',
    interests: ['Ionic', 'Writing', 'Travelling']
}

You could also store a similar JSON object as a value in a Key-value database, and then retrieve that entire JSON object later, so what’s the difference between the two approaches? The difference is that with a document database the database knows about the structure of that JSON object, and provides a means to query against it, rather than just directly accessing a specific key value. If the above was just a chunk of data we needed to retrieve we could easily store it in a Key-value database, but if we wanted to run a query to return objects where the country was ‘Australia’, then it would be better to use a document database.

If you need a way to save and retrieve simple data, then a Key-value store might serve you well. But if you want to store more complex data that you are going to query against (like the data we used before in the Cars example), then a document database may be better suited.

Benefits of NoSQL Databases

I think NoSQL databases, document databases especially, fit nicely into the HTML5 mobile world since they often rely on JSON and Javascript syntax, but there are more concrete performance considerations as well.

The main benefit that NoSQL databases have over traditional relational databases in terms of performance is that they “scale out” instead of “scale up”. In general what this means is that as your NoSQL database grows you can add additional servers that contain partitions of your databases (even up to 1000’s). Whereas with relational databases, often more power is added to a single server. NoSQL databases are much easier and cheaper to scale in this manner.

On the contrary, this same thing is also one of the major downsides of NoSQL – it sacrifices consistency and stability for performance. NoSQL databases are able to achieve this performance increase by not adhering strictly to the ACID (Atomicity, Consistency, Isolation, Durability) principles that relational databases do. Relational databases are guaranteed to execute all of their reads and writes with no interference, if some data is available somewhere it will be available everywhere.

Instead, NoSQL databases usually take the BASE (Basically Available, Soft-state, Eventual Consistency) approach, which basically means that the database will be accurate eventually (as data is consolidated among potentially thousands of different nodes), but potentially inaccurate data could be read before that point in time.

So the take away here is that NoSQL could be good for things like blogs and comments, but perhaps not for things like financial transactions.

Creating a NoSQL Database with MongoDB

If NoSQL is new to you, your eyes have probably glazed over by now. I’ve thrown a ton of theory at you with no real fun or practical examlpes. So we’re going to change that by getting our hands dirty with MongoDB – a document style NoSQL database. We’re not going to be doing anything fancy, just something super quick to get you started.

The following install steps are specifically for Mac, and we will be using brew. If you don’t have brew installed already just follow the instructions on this page. If you are using a Windows computer, then you can follow the instructions here to install MongoDB.

You can install MongoDB with the following command:

brew install mongodb

and then start it with the following command:

brew services start mongodb

Now you can create a new MongoDB database and connect to it with the following command:

mongo test

You will now be able to interact with the test database through the terminal. To come back to this same database later you can just run mongo test again.

As I mentioned we’re not going to be doing much right now (we will go through a more complete example with Ionic 2 later), but lets at least insert some data. We’ll use the same JSON object from the example before.

Run the following command:

db.people.insert({
    name: 'Josh',
    country: 'Australia',
    interests: ['Ionic', 'Writing', 'Travelling']
})

Now we can retrieve the data we just inserted with the following command:

db.people.find();

This will return the data we just entered, but you might also notice that it includes an _id that is automatically generated for us. We might use that id to find this record specifically later:

db.people.find({
    "_id": ObjectId("5769193b6ac0052be4d4bed0")
})

We could also specify that we want to return that record, but only the name and country fields:

db.people.find(
    {"_id": ObjectId("5769193b6ac0052be4d4bed0")},
    {name: true, country: true}
)

or perhaps a more efficient way to run this query would be to return all fields except interests:

db.people.find(
    {"_id": ObjectId("5769193b6ac0052be4d4bed0")},
    {interests: false}
)

If we only wanted to return people who live in Australia we could run the following command:

db.people.find(
    {country: "Australia"}
)

Of course there is only one object in the database right now so this isn’t very interesting, but you get the idea. We’ll go through more complex examples soon but I think this example highlights one of the key differences between NoSQL databases and traditional relational databases. We didn’t have to do any work to create the database (define a scehma), we were able to just add some data right away and start interacting with it.

Summary

As you have probably been able to figure out from this post, NoSQL is a huge topic and a total paradigm shift from relational style databases that many people would have started with. If you’re completely new to databases then in a way that’s going to work in your favour since you won’t have any “bad habits” from having a SQL way of thinking ingrained in your mind.

I hope this article has given you a decent background of the NoSQL landscape, and in future posts we will start doing some more practical things.