CSC Event 2

We’re very pleased to announce our 2nd CSC Event taking place this time in the evening of July 16th.  It will still be at the fantastic Soho Hotel in London which everyone thought so successful last time but this time we’re trying something a little different.

The event begins at 6pm and will run until 9pm with only two sessions followed by drinks and possibly further drinks afterwards.

Last time our biggest feedback was that it was shame to choose between sessions and that it would be good to see both Technical and Work & Life so we will be running two sessions one after the other.  A Technical Session in the Screening Room and then we’ll all move to the Crimson Bar for a Work & Life session and since we’re in the bar, we’ll finish up with some drinks.

As always we hope for audience participation and we’d ask you to register and invite friends and colleagues – this is far outside the IBM world and our discussions are very broad.

Registration is here

Technical Session:  This will consist of 3 x 10 min presentations followed by an 30 min open floor discussion
“Exciting Technologies”
Where is the next surge in technical innovation coming from and how do we position ourselves for that?  Wearables?  Cloud Storage?  Collaborative Working? Searching and Finding Content?  Every day we discover or hear about new technologies and technology areas that sound promising.  Which ones are exciting and how do you decide between standing back, monitoring, beta testing or positioning yourself for adoption?

Work and Life Session:  This will be an open floor moderated discussion. All voices welcome!
“When People Are The Problem”
People collaborating together can do amazing things, but there is always the potential for personality clashes, misunderstandings and friction, particularly when change and issues of ownership are involved. In this session we will discuss how these issues can arise and what techniques we all use to finesse them or whether, when reality crashes in, collaborative working can work at all.

Oh yes, we'll need a database

I suppose this is the biggie. Since 1996 my database of choice has been the venerable NSF http://en.wikipedia.org/wiki/Notes_Storage_Facility#Database which is a Document Oriented Database. Coincidentally the rest of the world has now caught up and there are some more modern options available to achieve the same things now. The real flavour of the moment is MongoDB http://www.mongodb.org, like the NSF it’s document based and can be replicated between multiple locations. Unlike the NSF it is much more scalable and the old limitations that we were constrained by have largely been mitigated. Of course there is no such thing as a silver bullet, but really there is no comparison between the NSF and modern document oriented databases. You have no idea how sad that makes me, but it is true.

Anyway, back to the real world. Every application needs a database, and my choice in this case is MongoDB. As ever, this is not a tutorial, the MongoDB website provides quite reasonable documentation to get you going: http://docs.mongodb.org/manual/tutorial/install-mongodb-on-os-x/

Now, you could just go with Mongo directly, but in various tutorials I had read about MongooseJS as well. MongoDB is just a database, it doesn’t force you to implement schemas, validation, or really any structure to your data. It may well be the case that you don’t want any of that, in which case go for your life. The syntax for talking to MongoDB is very simple REST calls. But with the addition of MongooseJS we get another layer of abstraction and assistance.

The primary boon for me has been the creation of simple schema’s. Each of the document types in my application has a schema defined; I know what fields will be created, I can create default values for optional fields and I can also create server side validation rules in a nice structured way. It’s also really useful to have a standard way of running code every time a document is saved back to the database.

And at this point we have our first venture into the cloud. It’s very easy to get everything we’re talking about running on your local machine, but what about when you want to show it off to the rest of the world? In my case I’m working with Bruce so for him to test we need an environment. There are hundreds of options available to you, but as with all of these posts I’ll describe what I’m doing.

Heroku Console

First we have the node.js hosting, where we’re using Heroku. It’s nice and simple to create a development instance that runs on a single dyno and then add plugins for things like console management, monitoring and so on. But we also need somewhere to host the database itself. For this we’re currently using MongoLab which again is very easy to set up a development database instance. The key thing about both of these development environments is that they are free. So I can do a Git push from my repo, and add configuration to the application to point to Mongolab for data storage and suddenly my application is running in the cloud.

MongoLab console

This is all worryingly simple isn’t it? 

Jade HTML Template Engine

In the last post I mentioned that when you start using Express, you are nudged towards Jade to create your HTML.

At its simplest, Jade provides a way to make your HTML more terse. To produce HTML like this:

We would only need to enter Jade like this:

We can break up the contents of our pages, so in this example the head contents are defined in a different file and can be shared across multiple pages. The syntax is very simple, indentation is used to nest elements within each other and you can add ID, class and other attributes with simple markup.

But really the power is that you can also do simple scripting inside the Jade file. So you can conditionally load chunks of the page based on data that you pass into the page. And then you can also pass in variables to the Jade template to be used when building the HTML. In the example above, the title is passed into the page as a variable from the route configuration.

Of course you can bind field onto your database using similar techniques. 

The structure of my real world application has a layout Jade file which itself is made up of several Jade files: head, header, footer, and foot. In the head I have all of my CSS and web attributes. The header is the navigation header for the site. The footer is a static footer navigation bar and the foot contains all of the JavaScript files that need to be downloaded to the client. In between header and footer I can insert my content.

The momentum in the web development world is very much towards AngularJS at the moment which doesn't really sit with Jade too well. It is possible to use both but they tread on each others toes rather a lot so it may end up being more trouble than it's worth. But for my purposes Jade and jQuery are working well together for me at the moment.

Express web application framework

Having created a simple application and installed our first NPM package, now things start to get serious. We want to start building out our real application, there will be more than one page, we’ll need client side JavaScript and CSS files. Organising all of this is important, you will have hundreds of files before you know it. This is where Express comes in.

If we install the basic Express npm package then we can get one of the biggest advantages for us, that is routing. I can define URLs and URL patterns that make up my application and then map functionality onto each one as I see fit. As you get more advanced you can start defining middleware steps that test security before allowing the user to get to a page. It makes your application configuration orders of magnitude simpler to look after. From my point of view at the moment, if you wanted to create a node.js web application that didn’t use Express then I’d want a very good reason why.

The basic installation is very simple, simply edit your package.json file and add express as a dependency:

Then, from the terminal, run “npm install”. You’ll see a bunch of new files installed under node_modules and can now start modifying the server.js file:

The key section is the line that starts app.get. This tells the application that it wants to listen for GET requests at the URL “/hello.txt” and what to do when someone requests that page, in this case just return some Hello World text. If the user requests a different URL then they’ll get an error. And of course you can also configure your routes to listen for PUT, POST and DELETE requests. And hopefully at this point you’ll see what’s happening, you now have the simple ability to support CRUD in your application, whether that be from web browsers or creating a REST API.

In the real world you’ll want to move your application logic off to other files before the server.js file becomes too large and unwieldy, so you can equally have lines like this:

In this example I’ve moved the business logic into a file called hello.js that lives in a subfolder from the root of my application: app/controllers/hello.js. You can put all of your hello world application functions into that file to keep things nicely organised.

In fact I actually move the routes themselves off to a separate file, so that the server.js really is just the instantiation of objects rather than any real business logic.

But this is not the extent of what Express can offer you. If you’re starting from scratch you’ll probably want to server up static content like CSS, images, client side JavaScript etc. Express can manage all of that for you as well, but it’s easier to create a new application:

Structure of new demo app

Structure of new demo app

The first line generates a new application with placeholder content including CSS and Jade files and a place to insert your own client side JavaScript. This is more complex than it sounds because, of course, all of the routes to the static files have to be configured. Once the app has been generated you can move to its directory, and then run an npm install to get all of the required modules as defined in the auto generated package.json file. Finally we can run the application with npm start. This kicks off the application listening on port 3000 and we can load it in the browser.

The point of a framework is to make life easier, there is no reason you couldn’t set all of this up manually yourself, but there really is no reason to. The rest of the world seems to use Express as a standard and it does everything you could possibly want, well at least I’ve not come across anything that I couldn’t do yet. 

NPM or Node Module Packages

After getting your development environment set up and your fancy schmancy Hello World application running what next?

Let’s say we want to display a nicely formatted version of the current date and time. The current best of breed for date and time handling in JavaScript is called Moment and we can choose to use it either server side or client side. For our purposes here, we’ll use it server side, which means we need to install it into our application.

First we’ll want to create a file called package.json which acts as the main configuration point for the application. One of the things it stores is which NPM packages we have installed and which version of each package we want to use.

From the terminal we can now install moment: 

npm install —save moment

You’ll find the package.json file has been modified to include a dependency for moment and you’ll also see a new folder in your project called node_modules that contains the source code for the package. Rule of thumb is that you’ll never be editing files in that folder.

What we can now do though is start to use the newly imported package in our own code.

So we’ll add some extra code to the line which returns Hello World:

And when that runs in the browser we see something like:

Hello World
The day will end in 8 hours

As I write this there are nearly 75,000 packages available from NPM, so this is where you have to get into the mindset that if you are writing something new, then you are probably not writing something new. Leverage what has already been put out there.

Of course there are always issues. In your dependencies in package.json you’ll see that moment has been added with a version number, something like “^2.6.0”. You can also set this to be "latest", but this means that whenever you run an “npm install” against your application, that NPM will look to see if a more recent version has been released and upgrade it for you. You need to be very aware of this. I recently ran a general install and found that Express had been upgraded from version 3 to 4. This broke my entire application and I had to roll back in Git to recover. So you have to become highly attuned to the impact of what is being upgraded. Things that are important to you, like Express, you should probably fix the version (by removing the ^ from the dependency), whereas things that are more utility helpers like moment you can probably just change to “latest”.

There is a huge amount more that you can do with NPM and it’s worth reading up on it. It’s probably your biggest friend when getting started with node.