Building Calliope – Sproutcore data sources (Part II)

Game (aka Local) data source

In the local scenario the user has logged in, has selected one of its open games (which list if handled by the global data source) and is taken to the board in order to be able to play the game. In this scenario, the application must load the current game state (that is, the position of all the letters in the game), present it to the user and update it continuously, replicating the actions of the other players.

Notice that this task is more complicated, and cannot be solved using the “load-all-retrieve-by-id” approach used previously: the data involved is potentially quite big (imagine a game with a hundred turns, for example) and could take more than a while to load; actions are pumped in continuously, and as such cannot be loaded once and for all, but must be refreshed constantly.

In order to reduce bandwidth usage and loading times, here is what the application has to load in detail:

  1. The last turn of the selected game, along with its letters’ configuration (service */turns/{number});
  2. The list of pivotal actions belonging to the game and to its last turn (service */actions?pivotOnly&turn={turnNumber});
  3. Periodically, the list of new actions to perform (service */actions?afterAction={actionId}).

where * stands for /me/games/{id}.

Notice that in doing this, the application loads from the server only relevant data, not the whole dataset. In order to handle this scenario effectively, in a modular way – and given that each and every service call above includes a game id – I decided to write separate data source, that I called (surprisingly) GameDataSource.


An instance of GameDataSource is created as soon as a game id is available, that is, when the user has selected the game to play. The new data source is then cascaded to the global one and assigned to the data store. This way, every request issed to the data store is forwarded to the game data source first, and to the global data source as a second try.

A schema of the cascade data store

In order to build such a structure, the following code is used:{
    gameId: selectedGameId


With reference to the three above points let’s have a look at what requests are issued to the data store and how they’re handled by the GameDataSource.

(1) The last turn of the selected game, along with its letters’ configuration

The application already has the number of the last turn of the selected game: it is found in the turnCount field of the game model itself. So, loading in a specified turn is only a matter of issuing a direct find by id:

store.find(Calliope.Turn, currentGame.get("turnCount"));

This call is similar to the ones used for the global portion of data, but since in this case we don’t preload records through a query, the data source cannot contain the requested record. All this translates to a call to the retrieveRecord method in the data source. The retrieveRecord method is called every time the data source must load a record by id (which can happen by a direct call as above, or by the automatic following of a relationship) and it doesn’t have a copy of that record already available. Here is the implementation of retrieveRecord:

  retrieveRecord: function(store, storeKey, id) {
    // retrieve the id from the storeKey
    if (SC.none(id)) {
      id = SC.Store.idFor(storeKey);
    // retrieve the record type
    var recordType = store.recordTypeFor(storeKey);
    if (Calliope.Turn === recordType) {
        address: this.getUrl("/turns/%@".fmt(id))
      .notify(this, this._retrieveRecordDidComplete, recordType, store, storeKey)
      return YES;
    return sc_super();

The implementation is pretty straightforward: if the find request is related to a turn (Calliope.Turn record type), a GET is sent to the URL /turns/{id}, where {id} is the parameter passed in the call to find, that is: the turn number. The method getUrl is a simple utility method that prepends /me/games/{gameId} to the passed string; in the example above, the full URL becomes: /me/games/{gameId}/turns/{id}.
The service call above returns the turn information along with its letters (details here), in the following form:

  "time":"2011-03-11 21:56:44",
  "letters": [
    { "id":10369, "lid":1, "l":"A", "c":"s" },
    { "id":10370, "lid":2, "l":"A", "c":"s" },

This data is passed to the notify method _retrieveRecordDidComplete, which is in charge of decoding it and loading it into the store as one Calliope.Turn entity and a number of related Calliope.TurnLetters. Here is the code:

  _retrieveRecordDidComplete: function(response, recordType, store, storeKey) {
    var body = this._getResponseBody(response, storeKey);
    if (!SC.none(body)) {
      if (recordType === Calliope.Turn) {
        // load turn letters
        var letters = body.letters;
        // replace body letters with just the ids
        body.letters = {
        // fill letters with the turn reference
        for (var i = 0; i < letters.length; i++) {
          letters[i].turn = body.turnNumber;
        // load letters into the store
        store.loadRecords(Calliope.TurnLetter, letters);
      // load record into the store
      store.loadRecord(recordType, body, body.turnNumber);

Sproutcore expects related records to be expressed as a list of ids; the letters field of the data structure above is not in the expected format, because it contains the whole objects instead of just the ids. What the _retrieveRecordDidComplete essentially does is extracting the list of letters from the original data structure, pre-loading them into the store with a call to loadRecords and then replacing them into the original structure with the list of the ids only. After doing that, the data structure is then in the format that Sproutcore expects, and can be loaded directly into a record of type Calliope.Turn. And this is everything that must be done in order to load the turn.

Notice that the call to the server takes place asynchronously, so the turn record cannot be returned immediately after the call to the initial find. What is instead returned is an empty SC.Record in state BUSY_LOADING, that you can observe in order to know when the data is ready to be used, like this:

  var turn = this.get("store").find(Calliope.Turn, currentGame.get("turnCount"));
  // add a status observer to the record
  turn.addObserver('status', this, function() {
    if (turn.get('status') === SC.Record.READY_CLEAN) {
      // remove the status observer
      turn.removeObserver('status', this, arguments.callee);
      // DO STUFF

End of Part II

If you read this far please take the time to leave a comment to let me know what you liked, what you didn’t, and what you would do differently. Thank you.

« Go back to the index


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s