Step Five (Building a SPA): REST, MVC and JSON

Having a self-contained application is a good start, but in a real SPA application it will need to manipulate data.  In order for a SPA to get data it needs to communicate to the server and receive a session token that will provide the authentication to further data calls.  This means that the SPA is an empty shell of views and controllers.  What we need to do now is make requests to the server to get data and bind it to the variables within the views.  In part six (6), we will look at the security aspects of SPA data calls, but for now we are focused on the calls themselves.

To get data into our AngularJS SPA, controllers are going to create separate calls to the server to get the data.  These calls are not going to be direct calls to the database, but instead interact the the data’s model.  The model sends the data in a JavaScript Object Notation (JSON).  The SPA’s controllers will take this information and update the view on the page.  This process from control, to model and to view is known as MVC.  A proper SPA makes calls to the server:

  • these calls are centered around the function (need) of the call, and does not perform an SQL query directly
  • the data has a defined schema

In order to have a clear path of development, we will focus on the three main steps for us to get data into the application. These are:

  • Route a call from AngularJS to a server-side JavaScript function
  • From the function, call a database and return the results via JSON to the SPA
  • Bind the data from the call into the AngularJS variable

The code in this example can be found on github.  Like the previous skeleton, this is very simple.  Over the next couple of blogs, I will add security to the calls and clean up the hierarchy in the view structure.  These changes, as the blog continues, are normal to Angular apps as they get more complicated.

REST

Data calls are done via a technique called “representational state transfer” or REST.  A simple way to look at REST is to look at its database equivalent: Create, Read, Update, and Delete (CRUD).

REST uses the HTTP method in order to say what action a database call is attempting to make.  In an HTTP method, most people know that there is a GET request and a POST request.  The former is used to get a page while the later is used to transmit data (or post) from a form. To support REST, two other methods from HTTP 1.1 are used: put (for update) and delete.

So let’s say we had a database of users (:humanName), where for each user identifier (:systemName) we have a profile (:email, :lastLogin) and a password (:password).  We then have a CRUD where:

  • Create: POST /user/  where the header contains a post of :email, :humanName, :systemName and :password  When inserted, we received a 200 OK 
  • Read: GET /grade/:userid  When replied to, we get a 200 OK with the JSON record containing :email, :humanName, :systemName and :lastLogin
  • Update PUT /grade/:systemName where the header contained a post of :email, :humanName, :lastLogin, and :password  When updated, we received a 200 OK
  • Delete DELETE /grade/:systemName  When replied to, we get a 200 OK telling us it was deleted.

REST vs Model

Stopping at the idea that REST mimics an aspect of CRUD is where many web application developers get it wrong.  The idea behind REST might be to mimic the database action, but the idea behind MVC is to encapsulate the action as a model.   What does that mean? It means that we look at why someone asks for data and make those calls consistent.  It tends to change the database call away from being SQL-like to one that is functionally driven.  Later, it will also allow us to ask who has the privilege to make such a request and also to aid in deriving data on the server-side.

Let’s look at user management to give us an idea of how functional calls to a model look.  In a basic application there is a user who needs to be associated with data, we will need functions to handle the user.  We are skipping the session validation for now:

  • add a new user
  • update profile
  • suspend user
  • unlock user
  • get profile
  • login
  • logout

These functions are going to bound to our routes.  Unlike our client-side routes, these routes are going to be placed in the server-side NodeJS application.  This means that there will be a function bound to a route, like so:


app.post('/user/login/', userProfile.login);

In this example, the login request is sending credentials. We want to hide them from the URL by putting them into the header. This means we will use a POST method with system name and password in the header. The route is called by the client JavaScript and is going to pass the authentication request to a function called ‘login’ that is in the userProfile module. It will return the session token, as a result of the call. This token will later provide access control over the allowed functions for the user. Normally, read requests are done through the GET method.

Without knowing how we are going to do the calls, we can create the abstracted routes in our main application file.

var userProfile = require(‘./lib/user.js');

app.post(‘/user/add/', userProfile.add);
app.update(‘/user/view’, userProfile.update);
app.get(‘/user/view’, userProfile.view);

app.post(‘/user/login’, userProfile.login);
app.get(‘/user/logout’, userProfile.logout);

app.get(‘/admin/list’, userProfile.list);
app.post(‘/admin/suspend/:systemName’, userProfile.suspend);
app.post(‘/admin/unlock/:systemName’, userProfile.unlock);

If you were paying attention to the routes you would notice that there are two main routes that later we are going to add access control to user and admin. You might also notice that only the admin has a systemName variable to the route. This is because the systemName for the user is associated with their session token that would have been past in the post header when initially authenticating.  This prevents client from sending its own permission.

Now that we have a sever-side route to function binding, we need to think about what these functions actual do and how they communicate with the database.

No SQL does not mean No Structure

There is a tendency to believe that a No SQL database is a form of freedom from structure. SQL databases are like strong type-checking programming languages. And that is a good analogy. That when we loose the structure of tables and joins, we become responsible for the structure ourselves. That allows us to be very flexible, but also puts the ownership of validation and correctness on us.

We need structure as code gets more complex, and we need to get more efficient. After all, if there is no structure then there is nothing for us to test and validate against. In No Sql that structure is placed into a schema. So when dealing with No Sql databases, we want to have an idea of the schema for the document. It is common to use JSON input/output to allow us to articulate that structure.

Let’s use JSON to describe a JSON schema that helps us enforce the model. In our user example we might want fields like:

  • humanName
  • systemName
  • email
  • password
  • role
  • lastLogin
  • active
  • joinDate

This could create a schema, like so:

human = { humanName: String,
systemName: String,
email: String,
password: String,
role: { type: String, default: “user” },
lastLogin: { type: Date, default: Date.now },
active: { type: Boolean, default: true},
joinDate: { type: Date, default: Date.now } }

In fact, there are validation schema routines for Mongo db, and doing it this way is constant with Mongoose.

An important note about the password field. Storing a password is always a bad idea.  We will get around this in the next part by using a one way hash with salt. 

Why is there an “active” element?  In databases, there is a delete command.  However, normally this is something we only do when cleaning up a database.  Instead, we determine if the record is relevant, and we add that check into the process. A good example of checking to see if a record is active is a user account.  We do not want to loose the database information on a user account, so we suspend a user account, not delete it.  This is a further reason to avoid mapping database functions to REST calls.  Normally, we do not want a user to be able to delete data.

Lastly, note that there is no index to the record.  Though mongodb will generate one, this is a bad idea in exposing it to the user. Indexes are used by the database for speed and integrity.  They are not an element of functionality, so they show not show up in a model.  Also, exposing the index means that you are exposing the database.  Something that is to be avoided in an MVC implementation.

Adding a User

There has been a significant amount of words and little action to this point.  Let’s get to at least populating the data in the database with fake users.  I am going to skip the details of mongo db installation here as there are enough sites about that, and go to setting up the NodeJS relationship with Mongo db and start adding some users.

After installing mongo db you should be able to:

  • Call ‘mongo’ at the command prompt and get the mongo shell
  • Call ‘node’ at the command prompt and get the node shell

Now we are going to:

  • install mongoose
  • create a module to encapsulate the database
  • write two basic functions to use in our application for now

Now, in order to call mongo db, we will need to install a driver.  This is done our normal package management (npm install mongodb).  But we want to do this the right way, so we are going to implement a driver with a schema instead, this one being mongoose.

npm install -save mongoose

Mongoose will be used by NodeJS, and therefore we do not use the global (-g) option when we install it.  We do want it to be inside of package for NodeJS, so we will call it with the save (-save) option.

Node Modules

We are going to use a JavaScript module.  This is the first time we are going to be writing our own module.  In a module, we want to encapsulate all but what is actually called.  Those parts that we want to be exposed we need to explicitly define.

There are three parts to creating a module.  First we will create a blank anonymous function and assign it to a variable.  This creates the function so its base is assigned to the variable, not the current scope.

userProfile = function () {};

Then we assign our variable and functions to the base function.  This base object will act like a hash, and allow us to attach anything we want.

userProfile.helloWorld = function {
console.log(“Hello World.”);
};

Then we export the module.

module.exports = userProfile;

This allows to call this module from another routine, and be able to access the functions from the scope that they were defined in.

module.exports = userProfile;
app.get('/admin/user', userProfile.helloWorld);

Mongoose Module

Let’s create a module (lib/user.js) that handles the user elements from the mongo database via mongoose. We need to first connect to the database. This connection is local to the routine’s scope. We do not need to expose them to the calling module.

var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/test');

var db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error:'));

Now we are going to use that schema we created before and create a Human Class for the module to use.

humanJsonSchema = { humanName: String,
  systemName: String,
  email: String,
  password: String,
  role: { type: String, default: 'user' },
  lastLogin: { type: Date, default: Date.now },
  active: { type: Boolean, default: true},
  joinDate: { type: Date, default: Date.now }};

var humanSchema = mongoose.Schema(humanJsonSchema);
var Human = mongoose.model('Human', humanSchema);

Next, we are going to write the parts of the code that the main node application will see and leverage.

var userProfile = function() {};

We will start with saving a user. If you are not use to Express or callbacks, this can get very tricky. The callback and Express functions are topics that really needs to be addressed in its own blog.  For this blog, I am going to be high level and terse. First, the arguments of the function appear in the order: request, result and error. The arguments and order can be found in the API (http://expressjs.com/api.html). If we call a function at the end of parameters, we can use it internally for a callback (we will define this in a bit). Second, express places header information from the request in the first argument, which most people will label ‘req’ to be consistent with the API.

A passed variable can be referenced by knowing where it comes from. The two most common elements of the request are:

  • body (req.body). Information is commonly placed here in a post.
  • body(req.param). This information comes from the URI, and is placed there during gets.

Nowadays every likes to chain there javascript. We could have done this in three steps.

  1. get values from the req(uest). Error check and assign them to manageable local variables
  2. create a new local variable from an instance of the Class (in this case new Human).
  3. Then save the instance into the database.  In the save argument, place a callback that takes the results and sends them to the client

Here a used chaining, and only used a local variable when performing an operation on the passed variable. Then I created the instance and immediately saved it.

userProfile.saveUser = function(req, res) {

var role = req.body.role || "user";
var humanName = req.body.firstName +" "+req.body.lastName;

var insertResult = new Human({
  humanName: humanName
  , systemName: req.body.systemName
  , email: req.body.email
  , role : role
  , password: req.body.password })
.save(function(err, result) {
  if (err) {
    res.send(500, { error: "database error:" + err });
    return;
  }
  res.send(200, result);
  })
};

Let’s get back to that callback concept. Note that there is no ‘return’ in the function, nor do we store the results before trying to format or manipulate the results. For sequential programmers, this is the most difficult part of javascript. Javascript does not wait around. As soon as the call to the database is completed, javascript will not wait for the results. By attaching (chaining) a function, it acts like a command line pipe and will wait for the response so that the callback function can perform further instructions.

The “Callback” is when the programmer passes a function that performs the next operation in the arguments of an asynchronous call. We pass a callback function in order for it to occur on the results of a function. In this example, when the save occurs the callback is executed on the results. If there is an error, we place the error into the response, and do likewise with a success.

Mongoose is doing a lot of work here. We are really using very little of its capability.

So, what about getting a list, our other function. This time there are no parameters, we are just going to return the list. We call the class directly, and retrieve all the elements. These are placed in the res (response).

userProfile.list = function(reg, res) {

Human.find(function(err, result) {
  if (err) {
    res.send(500, { error: "database error:" + err });
  return;
  }
    res.send(200, result);
  })
}

Now that we have our two functions, we export them so that they can be called by the main application.

module.exports = userProfile;

Routing to the Module

Routing to the modules now becomes trivial on the server side.

var userProfile = require('./lib/user.js');
app.post('/user/signup', userProfile.saveUser);
app.get('/admin/user', userProfile.list);

We define ‘userProfile’, and this creates the connection and class. We use the functions as the arguments to the route.

There is more that we will do to improve the server side. Mongoose has a high number of features that we have not touched. We are also short on error checking, validation, and recovery. But we have enough here to start with building a client side.

Half way point – Time to update the client

The client side we have a number of structural things to do:

  • create two new forms (user sign up and list users)
  • create their respective controllers
  • update the layouts to point to the new views and add menu items
  • update the routes to route to them
  • update the index to load their controllers

Enter a User

It is easier to develop SPA code when starting from the HTML. It is here that you will define variables and imagine the functionality of the page. In a more structured approach, the user experience (Web UX) is addressed before any coding is done.

Creating the View (profile/index.html)

The outermost part of the view, we make sure that we are naming the controller that this view leverages. This is the controller name that we will put when we create the controller.

profile/index.html

profile setup

Inside this div, we place the formatting and the form. The key elements of the form are:

  • ng-submit : defines the function in the controller that will be activated when the submit button is clicked. It will appear as an attribute to the form.
  • ng-model : defines the AngularJS variable that is associated to the element. It will appear as an attribute to a input element.

With that said, we just need to create the form. In this example, I define the variables in the controller as being an element to the variable user. That is the name I will use in the controller. Doing it this way will make it easier for me to pass variables via an http method.

angular form

Now, at this point if you update the routing, you could see the form in the browser:

htmlForm

Creating the Controller

Like all Angular apps, the controller is going to do all the work. We are going to define the variable and communicate to the server using the controller. First, we need the controller to have the name that the view is looking for. We also want to add the $http and $window scopes to the controller so that we can use http ($http) to communicate and redirect ($window) to another page based on the results.

app.controller(
  "profile.Signup",
  function( $scope, $http, $window, requestContext, _ ) {

On the page, the html refers to a user hash. This is defined in the controller.

$scope.user = {};

When the user interacts with the page, the user variable will be updated. We defined this in the ng-model attribute. So, user.email would be user = {email: “”}. This means that the variables are already in a hash.

In the control code, we just need to attach the http call to the submit. We pass the data, by simply placing the user information into the data argument.

$scope.submit = function() {
  $http({
    url: '/user/signup/',
    method: "POST",
    data: $.param($scope.user),
    headers: {
      'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8'
    }
  }).
  success(function(data) {
    $window.location.href = "/#/listUsers";
  });
};

Also remember that Javascript waits for no man or process. Therefore, we chain the success check to the posting of the data. If successful we send the user on. Here we send the user to the list of users so we can debut whether the data was inserted.

We can use the chrome development tools to see if the parameters are actually being passed:

signupUrl

That is all we need to update this template. We do however want to update the main index.hmtl to make sure this controller is active.

Listing the Data

Now that we think the data is in there, we need to be able to see it. You can log into mongo and do a find. We placed the Human schema into the test database. But mongo can be funny, instead of ‘humans’ it sees ‘man’ and changes it to ‘men’, so its ‘humen’.

mongo commands

You should see a JSON output of the data entered.

Let’s create a page to see this. Again, we go through the process of first creating a view. This means we start the new view (/public/view/admin/index.html) with defining the controller that will interact with it.

anotherdiv

In this view we will create a table. The Angular attributes we need are:

  • ng-repeat : acts like a foreach statement
  • {{ }} : double brackets allow AngularJS variables to be displayed.

This makes the HTML look very natural.

table code

Creating the Controller

We need to add the http and windows variables to the template’s arguments.

app.controller(
  "admin.ListController",
  function( $scope, $http, $window, requestContext, _ ) {

Then we are going to create a function to encapsulate what we do when we load the page. I find that grouping helps readability. After getting a JSON response from the http.get call, we chain two functions based on success or error. We will get into handling error better in another posting.

function onViewLoad($scope, $http, $window) {
  $http.get('/admin/user/').success(function(data) {
  $scope.users = [];

  for(var i=0; i<data.length; i++) {
    $scope.users.push(data[i]);
  }
  }).error(function(err){
    console.log("return err");
    return $window.location.href = '/#/';
  });
};

onViewLoad($scope, $http, $window);

When we are successful, we iterate through the response and push the data into the users variable. This is the same one that we use ng-repeat on in the html. With the controller done, its time for house keeping.  If you had been cleaning your code as you went along, the page should show the users being entered.

userList

House Keeping

We now just want to take a moment and make sure that all the variable interdependencies are addressed:

  • Controller names appear in the views
  • Layouts refer to the new pages and the layout structure is correct
  • The menu refers to the new pages
  • The index page includes the new controllers

Lastly, we need to add the routes to the server side.  This is in the (public/js/manage-routes.js) file:

routing

Conclusion

At this point, you should be able to enter users and see a list.  There is still work to be done to make it a robust and even more to make it secure.  The advantage of SPA written with Mongoose is that the project can be hosted on Heroku.  There are other alternatives to MongoDb, such as ElasticSearch and FireBase.  Overall, there are a range of options for databases to connect to using a SPA.  But, making the interaction to the database a functional one, and not one centered around the database’s operations (CRUD), will provide a cleaner means to change the backend while also providing better security.  Next, I plan on completing the user authentication and access control.

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: