Last week I wrote an article that gave a broad overview of the course software I built for the Elite Ionic course – a Progressive Web Application built with Ionic.
This mostly detailed my approach to building it and any interesting issues I faced throughout the entire process. One point I did touch on was the tech stack I used, but I didn’t go into much detail. To recap, this is the tech the course uses:
- Node/Express server hosted on Heroku
- Superlogin for authentication and registration
- SendOwl for sales and license key generation
- CouchDB (Cloudant) as the remote database
- PouchDB for storing data locally
- SendGrid for sending emails
I describe how everything fits together in the previous article, but the basic idea is that when the user purchases the course they are able to register an account in the application, and then the course content (all of the modules and associated lessons) will be synced to their device.
In this article, I am going to dive a little deeper into how the CouchDB/PouchDB integration is able to achieve the core goals of the application, which are:
- Provide only authorised users access to the content
- Be able to easily update the content
- Make the data available offline
I’ll be discussing the specific data structures I used for the database, and processes I used to retrieve that data, but I won’t be spending much time discussing the specifics of CouchDB/PouchDB. If you are interested in learning more about how CouchDB/PouchDB can be integrated into an Ionic application, there is an entire module dedicated to it in Elite Ionic, and I also have a lot of free tutorials about CouchDB and PouchDB on my blog.
The General Structure
The application has two remote CouchDB/Cloudant databases – one for storing all of the course content, and one for storing the user accounts for the application.
When the user logs in to the application, a one-way sync is triggered to replicate all of the data in the Cloudant database that contains the content for the course to the local PouchDB database. If the data has already been replicated previously, then it will just sync any changes to the local database.
The code required to trigger this replication is as follows:
this.db.replicate.from(this.remote, options).on('complete', (info) => {
this.syncSubject.next(true);
});I’ve set up a syncSubject observable that triggers once the replication has completed so that I know it is safe to start querying the database. This is just a uni-directional once-off replication, but it is also quite easy to have the replication occur both ways (i.e. changes to the local database are reflected in the remote database, and vice versa) and it can also continuously sync live as soon as any changes occur. For this application, that would be overkill – the user never makes local changes to the data so a two-way replication is not required, and the course content is updated reasonably infrequently (so a simple refresh of the browser to pull in the latest version isn’t unreasonable).
Restricting Access to the Database
It’s easy enough to set up a CouchDB/Cloudant database and have it sync to a local PouchDB database, but by default, it will allow anybody who knows the URL of the database to access that data. In order to restrict access to the database, you need to add user accounts to it. Once an account is added, you can then access the database by supplying the appropriate credentials:
https://username:password@mydatabase.cloudant.com/dbnameThis is where SuperLogin comes in. Once you set up SuperLogin on a Node/Express server, it will create a bunch of routes that can be interacted with (for registering, logging in, logging out etc.). It also integrates directly with the remote CouchDB/Cloudant database, so will handle creating the necessary user accounts for you in the user’s database.
Each user will be defined by their own document in this database which, among other information, will contain information about their “session”. When using Cloudant with SuperLogin, it will automatically generate an API Key in the ‘Permissions’ tab of your database, which will allow the user to access the database with whatever permissions you configure through SuperLogin (in my case, I grant _reader and _replicator permissions).
When the user creates their account or logs in, SuperLogin will provide the details of the generated API Key, and then those details are used in the URL when replicating the database. This allows me to ensure that only people who have created an account in the application are able to access the course content.
Document Structure
The overall document structure of the database is quite simple – primarily there are two types of documents:
- Modules
- Lessons
The document for a module looks like this:
{
"_id": "hpa",
"type": "module",
"order": 1,
"title": "High Performance Applications in Ionic",
"icon": "hpa.png",
"summary": "Pay attention to the little details that make a big difference"
}The entire purpose of the module document is pretty much to just declare that the module exists, the only information it holds is its title, the name of the file to use as the icon, and a brief summary. A module only becomes interesting when we start associating lesson documents with it, which look like this:
{
"_id": "hpa-large-lists",
"type": "lesson",
"video": false,
"moduleId": "hpa",
"order": 13,
"title": "Dealing with Large Lists",
"summary": "How to handle performance issues with large lists",
"keywords": "",
"_attachments": {
"13-large-lists.md": {
"content_type": "text/markdown",
"revpos": 2,
"digest": "md5-Mrr7JvSoZhw1Xu8bqgI1AQ==",
"length": 11131,
"stub": true
}
}
}A lesson document is associated with a particular module by supplying the modules _id in the moduleId field. There is, of course, some information in here like the title and summary of the lesson, but the important part is the _attachments field. In CouchDB, we can upload attachments to documents – an attachment is basically just another document – which might be things like images or audio files. In this case, I am uploading markdown files that contain the lesson content. The application is then able to use that attachment to render out the lesson content.
That’s pretty much the entire structure of the database – module documents that have associated lesson documents.
Design Documents & Views
With the strategy I have outlined above, I am able to effectively store all of the content I need for the course, and make sure that only authorised users have access to it. However, I also need a way to effectively retrieve that data for the application.
The application displays a list of modules to the user, and upon choosing a module it will display a list of all of the available lessons in that module. In order to be able to retrieve and display the correct information, I need to create some “views” in the database. A view is a way to generate a specific set of data in the database that you are interested in.
There are two “design” documents in the content database for Elite Ionic. A design document is a special kind of document that provides a way to perform some kind of application logic in a CouchDB database, and one thing we can do with a design document is to create “views”.
There is a design document for the “modules” that looks like this:
{
"_id": "_design/modules",
"language": "javascript",
"validate_doc_update": "function(newDoc, oldDoc, userCtx){ if(userCtx.roles[0] !== '_admin'){throw({forbidden: 'operation forbidden'})} }",
"views": {
"by_module": {
"map": "function(doc){ if(doc.type == 'module'){emit(doc.order);} }"
}
}
}This design document defines a validate_doc_update function to determine if a write to the database should be allowed to be committed or not, and a by_module view.
The validate_doc_update function looks like this:
function(newDoc, oldDoc, userCtx){
if(userCtx.roles[0] !== '_admin'){
throw({forbidden: 'operation forbidden'})
}
}It checks the userCtx parameter to see if the user making a write request to the database has the _admin role.
When user accounts are created they are only given _reader and _replicated permissions on the database – since they do not have _writer permissions they will not be able to update documents in the database anyway. Nonetheless, it doesn’t hurt to have this validate_doc_update function that restricts updates specifically to a user with the _admin role.
The by_module view is defined by a map function that looks like this:
function(doc){
if(doc.type == 'module'){
emit(doc.order);
}
}This is a pretty straightforward view: if the document has a type of module then it is added to the view. I’m using the order as the key, which allows me to sort the modules in the order I wish by changing this order value.
The other design document is for “lessons” and it looks like this:
{
"_id": "_design/lessons",
"language": "javascript",
"views": {
"for_tdd": {
"map": "function(doc){ if(doc.moduleId == 'tdd'){emit(doc.order);} }"
},
"for_nosql": {
"map": "function(doc){ if(doc.moduleId == 'nosql'){emit(doc.order);} }"
},
"for_hpa": {
"map": "function(doc){ if(doc.moduleId == 'hpa'){emit(doc.order);} }"
},
"for_comp": {
"map": "function(doc){ if(doc.moduleId == 'comp'){emit(doc.order);} }"
},
"for_ux": {
"map": "function(doc){ if(doc.moduleId == 'ux'){emit(doc.order);} }"
}
}
}This defines 5 separate views – one for each module. The purpose of these views is to emit all the lessons for a particular module, which I determine by its moduleId property (which will be set to the _id of a specific module). An order value is again used to allow the lessons to be sorted into the order that I wish.
So, if I were to access the for_hpa view, it would only contain the lesson documents that are associated with the High Performance Applications in Ionic module.
This design document does not contain a validate_doc_update function because the function defined in the previous design document applies to all documents in the database – not just the module documents.
Querying CouchDB Views in PouchDB
With everything set up on the backend, I just need a way to retrieve the relevant data from the local PouchDB database. If I want to return all of the available modules, I can do so by performing the following query through PouchDB:
let options = {
include_docs: true,
};
this.db.query('modules/by_module', options);This will return the data from the by_module view defined in the modules design document, and it will also return the associated documents for each row in that view.
If I want to get the lesson documents for a particular module, I can do so with the following query:
let options = {
include_docs: true,
};
this.db.query('lessons/for_' + id, options);where id is the id of whatever module I am interested in. If I want to get the document for a particular lesson I can do so by supplying that lessons _id:
let options = {
attachments: true,
};
this.db.get(id, options);This time I specify the attachments option because I don’t just want the details of the lesson, I want the actual markdown file that is attached to the document that contains the lesson content.
Summary
I’ve created plenty of tutorials on PouchDB and CouchDB in the past, and whilst this doesn’t dive particularly deep into how these two technologies work, hopefully, it is useful to see the general structure of a “real life” application that has been built with PouchDB and CouchDB.