Building an offline-first app with React and CouchDBMaintaining data integrity regardless of connectivity

chevron_leftAll Articles
Guillaume St-PierreSoftware Developer @ Manifold

I doubt I’ll make the news by saying that mobile devices and web applications have taken the world by storm. Nowadays, most applications need a constant connection to the internet. Data loss or slow networks have become a problem most developers need to work around. When it comes to allowing users to go offline without losing their data, CouchDB offers some amazing features that are worth a look.

What is CouchDB?

CouchDB is a NoSQL database built to sync. In short, the CouchDB engine can support multiple replicas of the same databases that live all over the world, and can sync them in real time with a process not dissimilar to git. These replicas are also not limited to database servers. CouchDB compatible databases like PouchDB allow you to have synced databases on the browser or on mobile devices.

When you create a document in Couch, revision specific fields which allow for easy merges with its copies are created. When the databases sync, CouchDB compares the revisions and change history, tries to merge the documents, and triggers merge conflicts if it can’t.

{  
   "_id":"SpaghettiWithMeatballs",
   "_rev":"1–917fa2381192822767f010b95b45325b",
   "_revisions":{  
      "ids":[  
         "917fa2381192822767f010b95b45325b"
      ],
      "start":1
   },
   "description":"An Italian-American delicious dish",
   "ingredients":[  
      "spaghetti",
      "tomato sauce",
      "meatballs"
   ],
   "name":"Spaghetti with meatballs"
}

All this is handled through a built-in REST API and a web interface. The web interface can be used to manage all your databases and their documents, as well as user accounts, authentication, and even document attachments. If a merge conflict occurs when a database syncs, this interface gives you the ability to handle those merge conflicts manually. Finally, it has a JavaScript based programming interface for its views using the map-reduce pattern.

Notice a trend here? CouchDB comes with a lot of built-in tools and really, really wants to be your main backend. Whether or not you decide to use it as your main backend is up to you, but the amount of tools CouchDB provides makes it a great option for a simple plug-in backend.

Capture d’écran 2018-11-29 à 11.43.15

REST API?

CouchDB is accessed through a REST API. Anything you want to do should be done through that API. It handles authentication for you during those REST calls which prevents unauthorized access.

The API is very straightforward. If your database lives under /couchdb, then it would be accessed through /couchdb/database1. A document on that database would be accessed through /couchdb/database1/document_id.

Similar to how Couch adds some data on your documents, it also adds specific databases. For example, the _users database stores the database’s user accounts while the _security database controls the access for the users. There is also a special _design database that contains design documents written in JavaScript for indexes, document validations, format queries, and filtering.

JavaScript?

In order to create a special view or validate document updates, you’ll have to write some JavaScript. Views are a great example of how JavaScript is used in CouchDB. When querying data, you should rarely query the documents themselves. Rather, you should query a view that will be executed using the map-reduce pattern.

Simply told, the code you write will be executed on all the documents to transform the data. Then, the resulting array will be reduced to remove everything that returned empty. For example:

function(doc) {
  if(doc.date && doc.title) {
    emit(doc.date, doc.title);
  }
}

Capture d’écran 2018-11-29 à 11.50.42

When executed, this function will run on each document, emitting some data if the date and title fields exist. emit is a function to send data back from the view which takes two arguments: a key and a value. Here, the data will be returned as an array of objects where the title is mapped under the date.

Result:

[
    "2009/01/30 18:04:11": "Biking",
    "2009/02/17 21:13:39": "Bought a Cat",
    "2009/01/15 15:52:20": "Hello World",
    ...
]

This is but a very simple example taken from couch’s own guides. If you’d like to read more, these guides are a great place to start.

Amazing, I’m switching everything to CouchDB right now!

Hold on there. I'm happy my clickbait title worked, but first you need to consider if CouchDB is for you.

You should never choose a database/language/framework/insert-anything-else-here simply because it looks cool. Software must be chosen with your use cases and preferences in mind, and CouchDB is not necessarily the best tool for everything.

pie

CouchDB works wonders for common applications where it’s not an issue if the data may not be the same depending on which CouchDB instance you ask. It’s also great as a secondary database for non-sensitive user data or for edit-heavy applications like Google Docs. It’s also used by huge companies like IBM, NPM, the BBC, and the LHC scientists at CERN (Yes, that CERN), which give it some big support.

CouchDB can also work against you in many other cases. It does not care about making sure the data is consistent between instances outside of syncing, so different users may see different data. It is also a NoSQL database, with all the pluses and minuses that comes with it. On top of that, third-party hosting is somewhat inconsistent; you have Cloudant and Couchbase, but outside of those, you are on your own.

I may sound like the opposite of a sales representative, but it’s always a great idea to consider your use cases when choosing a database. When building an app, the drawback of CouchDB may make it a detriment to your application rather than a great tool. The cost of hosting might not be worth it, CouchDB can be very expensive compared to a Mongo or even a PostgreSQL cloud database.

Don’t just listen to some guy writing a blog post. If you feel like CouchDB is perfect for you, then it’s time to fasten your seat belt because you’re in for an awesome ride.

CouchDB for offline apps

Now that we have a good idea of what CouchDB is, let’s talk code. For the purposes of this post, I built a very simple React app using PouchDB and CouchDB. While it is very simplistic, it shows how we can sync the two databases and add some nice features for the user. Check out the code here. Let’s walk through it together.

The app we’ll build is a to-read list where a user can add books they want to read, and track the ones they’ve already read or delete them. For those who are asking: no, this is not a to-do list. Definitely not.

First, we need to install CouchDB. The simplest way to do this is to use Docker. I personally prefer to write docker-compose files over running Docker commands, so we’ll do just that, using the base CouchDB docker image with very basic configurations.

version: '2.3'
services:
    couchdb:
        container_name: couchdb
        hostname: couchdb.local
        image: 'apache/couchdb:${DOCKER_TAG:-latest}'
        environment:
            COUCHDB_ADMIN_USER: admin
            COUCHDB_ADMIN_PASS: secret
        volumes:
            - '~/data:/opt/couchdb/data'
        ports:
            - '5984:5984'

With that in hand, run docker-compose up -d and you will have a working CouchDB instance available at http://localhost:5984. Navigate there and you’ll see a greeting message from Couch.

Capture d’écran 2018-11-29 à 12.00.32

Even better, navigate to http://localhost:5984/_utils/ to access the CouchDB dashboard. At that point, enable CORS so that the local React app you’ll be building can access Couch. Go to http://localhost:5984/_utils/#_config and click on the CORS tab. CORS should be enabled and the origins domain should be “All domains (*)”.

We also need to create a database for our data. Go back to http://localhost:5984/_utils/#/_all_dbs, click “Create Database” and make up a name. I chose “not_a_todo_list”.

Next, we need to start building the React app. I like using parcel when building simple apps like these, so run npm install parcel-bundler react react-dom pouchdb to get everything we need to get started.

We need to set up some basic files to get started, as is usual with React, so create a src folder and two files in it: index.js and index.html.

Embedded content: https://gist.github.com/Minivera/1635c91a50a22cfe3a51924477fc2e77.js

Embedded content: https://gist.github.com/Minivera/6a6e6abfc9e136830afcc9dba7841959.js

In the src folder, create a Components subfolder. In there, we will create a second subfolder called App and an index.js file in Components/App. The Components/App/index.js is the base application component; that’s where we’ll be creating our client PouchDB database. First, import pouch and create a database.

Embedded content: https://gist.github.com/Minivera/a7d694f49e608ede5c1abf43287dec2f.js

This code creates two PouchDB instances. One is saved inside the browser, mobile device, or any other client, and the second is a link to our backend CouchDB instance running on Docker. We then sync the two to ensure that changes on one database are replicated on the other, if possible.

With that in hand, create an App component in this file and give whatever component it renders the PouchDB instance.

Embedded content: https://gist.github.com/Minivera/6ddd0c1a9f5c607eb0050b5568ee119a.js

Run parcel ./src/index.html to see the app in action.

Fetching and mutating data

Now that we have a database connection that syncs, we can start fetching data!. To do so, create a new component in a new subfolder in Components called ToReadList.

Embedded content: https://gist.github.com/Minivera/25de6374826c198447eb545b37688fe3.js

So what does this huge chunk of code do? Let’s dissect the code a little.

First, we set the initial state of our component in the constructor. We want to show our users that the data is loading until the data has been returned, so set the state to:

Embedded content: https://gist.github.com/Minivera/c26c4886acce6593d9b254b237fb95d1.js

Next, we add a componentDidMount hook to fetch the data. This hook starts a function called fetchData which resets the state, fetches all of the documents from the database and waits on that fetch. When the promise resolves, set the state with the data received or show an error in the console if an error occurred.

Embedded content: https://gist.github.com/Minivera/8d271d81c547c17bcddbef0f00eade8a.js

As you can see, the rows are not the documents themselves, but rather a bunch of data containing the doc, so make sure to map those to only get the data you want.

But what if we want to reload the list if the data changes subscription style? That is very easy to do, simply add this code to componentDidMount.

Embedded content: https://gist.github.com/Minivera/f99bf0c5d12a2d150b43137435513301.js

Now listening to that change is easy. fetchData automatically reruns whenever anything new happens to our database.

It’s important to keep the result of db.changes as it could potentially keep sending signals even if the component is unmounted. To prevent that, when the component unmounts, cancel the listener.

Embedded content: https://gist.github.com/Minivera/41a586c8816d78593a300662eb904fb4.js

Adding, changing, or deleting data is also very simple.

Adding a new “to-read” to the list can be done using the put method on the database.

Embedded content: https://gist.github.com/Minivera/a86b7fdc0bc9c6542c387c5ae6d8bde3.js

Why is the _id a JSON representation of the current date? When CouchDB fetches data, it orders them by _id by default. This ensures that the to-reads are always sorted by _id without having to do anything.

We use the same call for updates, but we give it the _id of an element that already exists. CouchDB knows to update that element rather than create a new one. For example:

Embedded content: https://gist.github.com/Minivera/45abcf853f148f2e2f3c6568d79c7926.js

To delete, use the remove method on the database with either an _id or the full document. I prefer the latter as it is more convenient than extracting the id.

Embedded content: https://gist.github.com/Minivera/f8823bfb20323f8b8da38af68e31d00c.js

If using the _id, you also need to specify which revision you want to delete. PouchDB also allows you to delete using put and setting _deleted to true.

Embedded content: https://gist.github.com/Minivera/8b0d3b257e05208d4c21dc3ebfeab166.js

Add some components to display the elements that live in the state, run parcel ./src/index.html and you should see your data showing up and changing even if you change the data from CouchDB’s UI.

We now have a working app with a synced and offline capable database!

Notify the user when offline

For the finishing touches, a big feature I'd want out of an online app would be to show what happens when the app goes offline. First, a notification should be sent out to the user about the app being offline and that their work will only be saved once the database is available again. Let’s go back to our App component and make this happen by adding new state for our database inside the App component.

Embedded content: https://gist.github.com/Minivera/d5d780cd616cdb3b214a6971da50dee1.js

So what is this all doing? CouchDB offers us a very simple way to know if it is alive or not through the _up endpoint. So all we have to do is query this endpoint at a regular interval and change the state depending on the status. This is the meat of the offline monitor:

Embedded content: https://gist.github.com/Minivera/03964f614cf88d2eabc316d5308e0320.js

We can now create a request object for the _up endpoint and start the fetch. When it resolves, only if no error occurred, we make sure it tells us that it is healthy and set the state accordingly. On an error, we set online to false. We run this code every 2 seconds as a health monitor, and stop it if the component unmounts.

This could be better implemented, but as a quick solution, it works extremely well and shows the potential of a React app with PouchDB.

The easiest way to test this is to disable CORS from the CouchDB dashboard. The offline state will change to true until you enable CORS again.

Check out the code at https://github.com/manifoldco/definitely-not-a-todo-list.

StratusUpdate

Keep your head in the cloud with Stratus Update

With our new weekly newsletter we’ll keep you up to date with a curated selection of the the latest cloud services, projects and best practices.
Click here to read the latest issue.