Dropbox Datastores and Drop-Ins

Recently, Dropbox announced its new Datastore API and Drop-ins, a couple of great new features aimed to leverage the power of accessing files and (now with Datastores) other general information from any device and keep that data synced across all platforms, painlessly.

Datastores

Today, the Datastore API only supports single-user use-cases, but multi-user scenarios are in future plans for Dropbox.

Let's begin by discussing what datastores are. You can think of them as a small database to keep key/values pairs of information. Now, you may say that your application could use a web service with a database and your data will be the same across all devices, and while this is true, by using the Datastore API, we are taking away the overhead of handling a back-end service.

With this in mind, applications that don't need to store a large amount of user data and don't require heavy processing, can delegate the database management to Dropbox and forget about handling it manually. Take for instance a multi-platform game. You would want to allow the user to play it on their iPad in the morning, head to work and while in the traffic, continue playing on their iPhone. In this scenario you'd normally need that user to log into the system, play, and then save their progress.

Now with the Datastore API you can forget about the whole login process and the overhead of handling the progress data, you just use the provided SDK and store the information you want to store, later that day when your user is opening your application from another Dropbox connected device, you can easily retrieve their information. In this case, Dropbox handles the storage, security, and privacy of the information.

Although, as of right now the Datastore API only supports single-user use-cases. According to Dropboxer Steve M., multi-user scenarios are in future plans for Dropbox.

Persistent TODOs App

If you have ever used a JavaScript framework/library and came across example applications, chances are that in one of those apps there was a TODO app of some kind. So, in the spirit of keeping things consistent and to demonstrate some of the most common features, let's build a TODO app using the Dropbox Datastore API.

Since this is a tutorial on the Dropbox functionalities offered to developers, I'm not going to be explaining the HTML nor the CSS in the application, you can find those in the files accompanying this tutorial.

Step 1 - Setting Up the Application

First of all, like with most public APIs, we need to create a new application within the system. To do this, log into your Dropbox account and head to the App Console. Click on "Create app", select "Dropbox API app" and "Datastores only" and finally give your app a name.

You may be tempted to select "Files and datastores", however if your application is not actually using this permission, when you request production status, it will be denied, adhere to the Dropbox policies for every application you create.

Now you have a new application in Dropbox and you can start making use of the Datastore API (and other resources). In order to do this, you'd need your App Key. If you use the JavaScript SDK, as we will in this tutorial, you don't need your App Secret (keep this string secret).

Step 2 - Adding the SDK

We're going to be using the JavaScript SDK provided by Dropbox to interact with the Datastore API. To install it, simply add the following script declaration to your HTML document above the script for your application.

<script src="https://www.dropbox.com/static/api/1/dropbox-datastores-0.1.0-b2.js"></script>

Step 3 - Creating Our Application

Our application will be a single JavaScript file under the "js" folder. Create a new "app.js" file with the following code:

var client      = new Dropbox.Client({ key: YOUR_DROPBOX_APP_KEY }),
    TodosApp    = TodosApp || {

        todosList: null,

        init: function() {},

        checkClient: function() {},

        createTodo: function( e ) {},

        updateTodos: function() {}
    };
$( 'document' ).ready( TodosApp.init );

What this does is create a new Dropbox Client object using the App key obtained from the app console. It then defines our application object and when everything is ready, we call the init method.

Checking the User's Session

The first thing our application should do is check if we have an access token for the user of our application. Add the following code to the init method:

client.authenticate({
        interactive: false
}, function( error, response ) {
    if ( error ) {
        console.log( 'OAuth error: ' + error );
    }
});

TodosApp.checkClient();

Here we are trying to authenticate the app's user to the Dropbox API server. By setting the interactive option to false, we are asking the method to not redirect the user to the Dropbox site for authentication, this way our application can continue its normal flow. We are going to manually send the user there later on.

Now we need to check if the user is authenticated and if so, proceed to load in their data. Add the following code to your checkClient method:

checkClient: function() {
    if ( client.isAuthenticated() ) {
        $( '#link-button' ).fadeOut();
        $( '#main' ).fadeIn();
    } else {
        $( '#main' ).fadeOut();
    }
}

Here we are updating our interface accordingly, based on whether the user has been authenticated or not.

Authenticating the User

So far we have our application interface being updated accordingly, if the user is authenticated or not. Now we are going to handle the process of authenticating the user in the system. Add the following code to the else statement of the checkClient method:

$( '#link-button' ).click( function() {
    client.authenticate();
});

This is merely a callback which is called when the user clicks the "Connect Dropbox" button in the interface. Note that we are not setting the interactive option this time, thus allowing the redirection. If the user is not logged into Dropbox, a login form will be shown and the system will ask the user to allow the application.

Retrieving User Data

Once the user has been granted access to the application, it will redirect back to us. In this case, the call to the isAuthenticated method will return true, at this point we need to retrieve the user's Dropbox stored data. Add the following code to the if statement of the checkClient method:

client.getDatastoreManager().openDefaultDatastore( function( error, Datastore ) {
    if ( error ) {
        console.log( 'Datastore error: ' + error );
    }
    todosList   = Datastore.getTable( 'todos' );

    TodosApp.updateTodos();
    Datastore.recordsChanged.addListener( TodosApp.updateTodos );
});

This method retrieves the authenticated user's Datastore and accesses the todos table. Contrary to an SQL table, the table structure doesn't have to be defined prior to usage, as a matter of fact, the table doesn't even exist until we add data to it.

What this also means is that the table can contain any data, and one record doesn't have to have the same data as the others. However, it is a good practice to preserve a similar, if not equal structure, amongst records.

Rendering Records

At this point we have the user's todos information, however it is not displayed to the user. In order to do this, just add the following code to the updateTodos method:

var list    = $( '#todos' ),
    records = todosList.query();

list.empty();

for ( var i = 0; i &lt; records.length; i++ ) {
    var record  = records[i],
        item    = list.append(
            $( '&lt;li&gt;' ).attr( 'data-record-id', record.getId() ).append(
                $( '&lt;button&gt;' ).html( '&amp;times;' )
            ).append(
                $( '&lt;input type="checkbox" name="completed" class="task_completed"&gt;' )
            ).append(
                $( '&lt;span&gt;' ).html( record.get( 'todo' ) )
            ).addClass( record.get( 'completed' ) ? 'completed' : '' )
        )

        if ( record.get( 'completed' ) ) {
            $( 'input', item ).attr( 'checked', 'checked' );
        }
}

This method simply sets a container element for the HTML tag that will contain our list of todos, then we retrieve the records in our todos table by calling the query method from the Datastore API. Next, we clear the list of items and finally we render every record to the screen.

Deleting a Record

Now that we have the ability to retrieve the user's stored TODOs on application startup, let's work on deleting those records. In our render method, we'll create an X button. Add the following code to the bottom of the updateTodos method:

$( 'li button' ).click( function( e ) {
    e.preventDefault();

    var id  = $( this ).parents( 'li' ).attr( 'data-record-id' );
    todosList.get( id ).deleteRecord();
});

In this code we just get the id of the record to delete, retrieve the actual record by calling the get method, and delete it by calling deleteRecord on the obtained object. Since we previously set the recordsChanged callback, our interface will update like magic.

Updating a Record

So far so good, we can retrieve a list of tasks from the user's Datastore and we can delete a record from it. Now how about updating a record? For this new feature we are going to add in the ability to mark a record as completed or not. Add the following code to the bottom of the updateTodos method:

$( 'li input' ).click( function( e ) {
    var el  = $( e.target ),
        id  = el.parents( 'li' ).attr( 'data-record-id' );

    todosList.get( id ).set( 'completed', el.is( ':checked' ) );
});

Like with the delete method, we retrieve the id of the object to update, retrieve the record object itself, and set its completed property according to its current state.

Creating a Record

Finally, we need to be able to create new records in the user's Datastore. In order to do this, we need to react to the form submission event that the add-todo form will trigger. Add the following code to the bottom of the if statement in our checkClient method:

$( '#add-todo' ).submit( TodosApp.createTodo );

This is simply a listener for the submit event on the add-todo form. Now for the actual record creation. Add the following code to the createTodo method:

e.preventDefault();

todosList.insert({
    todo: $( '#todo' ).val(),
    created: new Date(),
    completed: false
});
$( '#todo' ).val( '' );

With this, we have completed our example application. As you can see, the CRUD operations for our data have become really simple and we can access it across multiple devices. While using this service, we don't have the need to create a full back-end service to store the user's information.

Datastore Extras

As an extra service to developers, Dropbox let's you explore the data inside your Datastores. To check this, go to the App console and select Browse datasores from the submenu, select the application you want to check the Datastores for and you'll be presented with a list of the existing tables and each record in the table.

Space Limits

When creating Datastores, you have to take into account the amount of information you plan on storing. Every application has up to five MBs per user, to use across all datastores. As long as your data doesn't hit this limit, the Datastore won't contribute to the user's Dropbox quota. Keep in mind that any data over this limit will count towards the user's Dropbox storage quota, and writing operations may be limited.

Field Types

Datastore records can be seen as JSON objects, however there are certain constraints about the data that a field can contain, for instance, even though you can see a record as a JSON document, you can't have embedded documents. The types of data you can store are as follows:

String
Boolean
Integer - 64 bits signed
Floating Point
Date - POSIX-like timestamp
Bytes - Arbitrary binary data up to 100 KBs
List

A list is a special kind of value that can contain an ordered list of other values, but not lists themselves.

Drop-Ins

Another great feature added to Dropbox for developers are Drop-ins. There are two types of Drop-ins, the Chooser and the Saver. With these new features you can add support to your application to either select (for sharing or some other purpose) files directly from Dropbox with the Chooser and to directly store files to Dropbox with the Saver.

So continuing with our example, lets add Drop-ins to our TODOs application.

Step 1 - Setup

As with the Datastore API, we need to create an application for Dropins, head to the App console, select Create new, choose Drop-in app and give it a name.

Now we have a new application, contrary to the applications for other Dropbox APIs, this one doesn't need production access, so once you're ready, you can offer it to your users with no hassle.

Now the only thing we need to do to add Drop-ins support to our application is add the SDK, add the following code to the scripts declarations in the HTML file, above the script for your application:

<script src="https://www.dropbox.com/static/api/1/dropins.js" id="dropboxjs" data-app-key="YOUR_DROPBOX_APP_KEY"></script>

Note the id with a value of dropboxjs, if you remove or change this, Dropbox won't be able to get your application key, hence breaking the Drop-in functionality.

Step 2 - Chooser

OK, so now we have the Drop-ins API in place, let's begin with the Chooser Drop-in. To add the Choose from dropbox button, use the following code.

<input type="dropbox-chooser" id="dp-chooser" name="selected-file" style="visibility: hidden;" data-link-type="direct" />

This will generate the button for you and when you click it, it'll open a window allowing you to select files from the user's Dropbox. To style this element, use the class dropbox_choose. In my case, I'll simply center it on screen. The data-link-type attribute specifies if the obtained link will be a direct link to the file (useful for download, or display) or preview, in which case going to the link will take you to the Dropbox interface.

Both links have disadvantages, for instance a direct link will expire within four hours of its creation, and a preview link may stop working if the user owning the file removes or changes it. The preview link type is the default used by the chooser.

Working With the Result

Adding the code above will generate a "Choose from Dropbox" button, which when clicked will show us a window to select the desired file. To retrieve the selected file(s), yes it supports multiple selection, your application needs to respond to the DbxChooseSuccess event in the input element. Add the following method to your JavaScript application:

listenChooser: function() {
    document.getElementById( 'dp-chooser' ).addEventListener( 'DbxChooserSuccess', function( e ) {
        $( '#selected-image' ).attr( 'src', e.files[0].link ).fadeIn();
    }, false );
}

After creating the event listener, you need to add it to the application, at the bottom of your init method, add the following line of code:

TodosApp.listenChooser();

This event will give us a payload containing, among other things, an array of files selected by the user. In this case, we are selecting a single file and appending its link property to an image tag already in the DOM. Each file in the array contains some other fields, like the file size, its name, etc. For a full list of the properties in each file go to the Chooser Drop-in documentation.

Step 3 - Saver

Last but not least, we have the Saver Drop-in. This Drop-in let's you save files directly to the user's Dropbox folder. Just like with the Chooser, you need a Drop-in application to use the API. Fortunately, we already created one and working with this Drop-in is even easier than everything else so far, simply create a link as follows:

<a href="http://.../dropbox-nettuts.zip" class="dropbox-saver"></a>

The href and class parameters are required for the Drop-in to work. The href is the file that will be saved to the user's Dropbox folder, and the class tells the application that it is a Saver Drop-in. Additionally, you may add a third parameter: data-filename which will be used as a user-friendly name for the file to save. If you don't specify one, the name will be taken from the href parameter.

As with the Chooser, there are a couple of more advanced things you can do with the Saver Drop-in, but for a reference on those check the official documentation.

Conclusion

As you can see with these new features in the Dropbox Developer Services, we can add powerful data storage capabilities to our web and mobile applications easily. This frees us from the overhead of creating a back-end service when little data processing is needed.

Hopefully by now you feel comfortable adding CRUD support to your application using the Datastore API and adding the capabilities to read and write from your user's Dropbox accounts, using the Drop-ins API. Please note that both APIs are really new, the Datastore API is still in Beta version, however you can see the potential they represent. Be sure to check the official Dropbox documentation for further information on these and other great services the platform has to offer.

HIGHLIGHTS OF THE DAY