Web-Technology Lecture 9

Express framework

Express Connection to Node.js Express framework is built on top of the Connect framework to make Node.JS web-app development It’s a full-scale webapp development framework similar to Rails for Ruby or Django for Python Arguably, the most popular Node.JS framework de facto standard server framework for Node.js (according to Wikipedia :) Unpinionated, minimalistic Lots of info on http://expressjs.com Active community Current version is 4.x. Version 5.0 is expected soon to bring full support of ES 2017 (Async functions, Promises ..) and more There are other frameworks built on top of Express.JS But we will not go further :)

Flow of the HTTP request through a Node.JS server

Flow of the HTTP request through an Express.js server Adds useful features & abstracts away complexity e.g. downloading a file is 1 line in express and about 40-50 in vanilla Node.js Changes one monolithic request processing function into a set of smaller handlers doing thing at a time They can be standard or even 3d-party functions They still have request and response as parameters

Middleware a middleware component is a function that intercepts HTTP requests and responses executes its logic either ends the response or passes it to the next middleware component has three parameters: req, res, next

Types of middleware Logger Router Serving static files Views renderer Error handler Session Authentication DB access …

Trivia Who is using express.js: More information here: expressjs.com … and here: developer.mozilla.org/ … and here:

Hello world webapp What’s missing? var express = require('express'); var app = express(); app.get('/', function (req, res) { res.send('Hello, World!'); }); app.listen(8081); var express = require('express'); var app = express(); app.get('/', function (req, res) { res.send('Hello World!'); }); What’s missing? $ node hello.js

Setting up basic Express.js app Create a directory to hold your webapp, and make that your working directory Create your webapp package specification (package.json) This starts a customization process - you either provide data, or accepts the defaults  Install Express in the ”myapp” directory --save command adds Express to the dependencies in package.json $ mkdir myapp $ cd myapp $ npm init $ npm install express --save

Setting a webapp with express-generator Install it (it installs express command-line tool) Make sure to run this as an administrator, e.g. Create an app and generate its skeleton: Options –view and –ccs help specify templating engines Install dependencies $ npm install express-generator -g ├── app.js ├── bin │ └── www ├── package.json ├── public │ ├── images │ ├── javascripts │ └── stylesheets │ └── style.css ├── routes │ ├── index.js │ └── users.js └── views ├── error.jade ├── index.jade └── layout.jade $ sudo npm install express-generator -g $ express myapp $ cd myapp $ npm install

Templating engines Allow to write and program HTML markup in an easier way <div> <h1>Ocean's Eleven</h1> <ul> <li>Comedy</li> <li>Thriller</li> </ul> <p>Danny Ocean and his eleven accomplices plan to rob three Las Vegas casinos simultaneously.</p> </div> div h1 Ocean's Eleven ul li Comedy li Thriller p. Danny Ocean and his eleven accomplices plan to rob three Las Vegas casinos simultaneously. - var x = 5; div ul - for (var i=1; i<=x; i++) { li Hello - }

Start your server Start in debug mode: On Windows: On MacOS and Linux: On Windows: Start in execution mode: $ DEBUG=myapp:* npm start > set DEBUG=myapp:* & npm start $ npm start

Logger function logger(req, res, next) { console.log('%s %s', req.method, req.url); next(); } var app = require(‘express') (); app.use(logger); app.listen(8081);

Customized logging Needs to be installed: npm install morgan morgan(format, options) Creates a new morgan logger that can accept format and options parameters format can be a predefined string specifying what to log or a function e.g. morgan('tiny')will log minimal information morgan(‘combined’) gives more details :method :url :status  :res[content-length] -  :response-time ms options is an object that helps to fully customize logging e.g. morgan({stream: accessLogStream}) – redirects logging from process.stdout to another stream var app = require(‘express’)(); var morgan = require('morgan'); app.use(morgan(‘tiny’)) .get('/', (req, res) => { res.send('Hello!') }).listen(8081); GET / 200 - - 1.384 ms GET / 200 - - 0.834 ms GET / 200 - - 0.570 ms

Basic routing Routing determines how an application responds to client requests Each route can have 1 or more handling functions executed with the parameters match app.get('/', function (req, res) { res.send('Hello World!') }) app.post('/', function (req, res) { res.send('Got a POST request') app.put('/user', function (req, res) { res.send('Got a PUT request at /user') app.delete('/user', function (req, res) { res.send('Got a DELETE request at /user')

Routing: Not found After all known routes have been processed, a 404 route can be added to handle “not found” situations: app.use(function(request, response) { response.status(404).send("Page not found!"); });

Routing: with parameters Customizable routing handling parameterized requests to particular routes parameters are encoded as named parts of the URL prodcessed parameters are automatically added to request.params This router will react to a get request to any URL of these kinds: http://localhost:8081/users/3/products/7 http://localhost:8081/users/9/products/2 app.get("/users/:userid/products/:productid", function(req, res) { var userId = parseInt(req.params.userid, 10); var productId = parseInt(req.params.productid, 10); // … }); Problem with this code?

Routing: regular expressions Routes can be specified as RegEx’s E.g. this will enforce that “userid” is an integer: app. get(/^\/users\/(\d+)$/, function(req, res) { var userId = parseInt(req.params[0], 10); res.send('' + userId); }) http://localhost:8081/users/3 http://localhost:8081/users/sergey

Serving static files Static files (images, CSS, JavaScript files, etc.) can be served using the built-in middleware Such middleware is usually “.use”d in the very beginning to make sure that if there is a static file to serve, it will handle it before the routers kick in It will intercept request to static files and will look for them in the /static/ folder If found the files will be returned to the client E.g., if you have form.html in the /static/ folder, a request like this will work: http:// localhost:8081/form.html If you have form.html in the root directory, client will not be able to access it Put index.html into the /static/ folder for a request like this to work: http://localhost:8081/ var express = require("express"); var path = require("path"); var app = express(); var staticPath = path.join(__dirname, "static"); app.use(express.static(staticPath));

Routing: splitting your app As your app grows, you might want to split all the routing options, or at least to separate them from the rest of the app.js // app.js var express = require("express"); var path = require("path"); var apiRouter = require ("./routes/api_router"); var app = express(); var staticPath = path.resolve (__dirname, "static"); app.use(express.static(staticPath)); app.use("/api", apiRouter); app.listen(3000); // api.js var express = require("express"); var api = express.Router(); api.get("/users", function(req, res) { /* ... */ }); api.post("/user", function(req, res) { /* ... */ }); api.get("/messages", function(req, res) { /* ... */ }); api.post("/message", function(req, res) { /* ... */ }); module.exports = api;

Routing: other app.methods app.all('/secret', function (req, res, next) { console.log('Accessing the secret section ...’); next(); // pass control to the next handler }) .all – matches all HTTP methods GET, POST, PUT, DELETE, ….. .route() – creates chainable route handlers for a route path app.route('/book') .get(function (req, res) { res.send('Get a random book') }) .post(function (req, res) { res.send('Add a book') .put(function (req, res) { res.send('Update the book')

Routing: other response.methods Description res.end() End the response process res.json() Send a JSON response res.redirect() Redirect a request res.render() Render a view template res.send() Send a response of various types res.sendFile() Send a file (better use express.static – it is safe and optimized) res.sendStatus() Set the response status code and send its string representation as the response body

Handling errors Error-handling middleware should go last, after other app.use()’s and routing calls Several error-handling middleware functions can be combined to carry out different tasks logging and response generations handle different errors Error handling middleware has 4 parameters instead of usual 3: (err, req, res, next) next(err) should be used to pass control from one error handler to the next If you put next(err) to your last error handler, the control will pass to the express default error handler If you want to throw an error yourself in your other middleware, you can use next(err) as well – the error handler will catch it

Handling errors express = require('express'); var app = express(); app.get("/login", function(req, res) { var userid = req.query.userid; if (!userid){ next(new Error("userid not found")); } else { // do stuff } }); app.use(function (err, req, res, next) { console.error(err.stack); next(err); if (req.xhr) { // ajax call res.status(500).send('AJAX failed!'); } else { } res.status(500).send('Something failed!'); app.listen(8081); Handling errors

store – sets a custom MemoryStore Sessions in Node.JS var express = require('express'); var session = require('express-session'); // extra declarations var app = express(); // initialise and use session middleware app.use(session({secret: 'ssshhhhh'})); // now session is a property of request var sess; // session variable //process GET requests to the root app.get('/',function(req,res){ sess = req.session; //init sess varfiable if(sess.uname) { res.redirect('/shop'); } else { res.render('index.html'); }); // rest of the routers app.listen(8081); There are more Node.JS modules implementing sessions Express-session stores session object on the server and passes session-id to the client as a cookie Some useful express-session methods: genid – generate a new session id name – replace the default name of a session id cookie “connect.sid” (many apps from the same host?) resave – forces saving session to the local memory storage secret – to sign the sessions ID cookie destroy – kills the session reload – reloads session from the store regenerate - regenerates the entire session object store – sets a custom MemoryStore Lennart Herlaar - UU

Server-side databases Benefits of using RDBMSs Ease of design: stable and clear data structures Ease of management: clear user privileges and availability of maintenance utilities Scalability: manage large quantities of data with good performance Multi-user access: built-in locking and transaction mechanisms Robustness: reliability and integrity of data Expanding web application with an RDBMS for storage is natural E.g. storying products and orders in a online shop Lennart Herlaar - UU

Computing models revisited Three-tier client-server model is good fit for Web-based applications Web-browser is a thin client responsible for presentation Web-server serves the application logic Database server manages the data tier Nice decoupling of application logic and data/transaction logic A variety of implementation options Major programming languages have been provided with integration APIs for major DB systems Lennart Herlaar - UU

Databases There are dozens of RDBMSs available: Oracle, MS SQL Server, MySQL, PostgreSQL, SQLite "What is the best RDBMS?" …a Holy War has began What is the best phone platform? OS?.. A better question is: “What is the best RDBMS in my situation?” "Use the tool that does the job best" Requirements, licensing, functionality, stability, simplicity, scalability, performance, expertise, support, cost, … Lennart Herlaar - UU

Databases on the web HTTP is stateless Operates based on page requests Consequences for interacting with a database 1 HTTP request – many queries is OK Usually, queries are relatively simple Queries are generated within a context Many requests – 1 transaction is not advisable A user can stop interacting with your website at any time Sessions are not persistent Trade-off: Speed versus data reliability and integrity? Lennart Herlaar - UU

Popular choices MySQL and SQLite are popular choices Free, fast, relatively easy a decent set of feature ACID-compliant database platforms MySQL is a client / server RDBMS SQLite is a file database No server process "Competitor" writes based file locking! Lennart Herlaar - UU

SQLite features SQLite files are portable SQLite works without users / passwords SQLite is "weakly typed" SQLite is not very serious ... ... Not a server database platform for large sites ... But it does well for practical assignments SQLite has “Zero-install footprint”! Lennart Herlaar - UU

Sets the execution mode to verbose to produce long stack traces Node.JS meets SQLite Installing: SQLite DB can be kept in memory, but we’re interested in persistent DBs, hence we need a DB file Extra stuff to work with our DB file safer Importing SQLite and connecting to the DB file The execution mode of our DB can be put either in serialized or parallelized – defining how transactions will be executed. As we want to create tables and then query the data, we call the DB in a serialized mode npm install sqlite3 Sets the execution mode to verbose to produce long stack traces var fs = require("fs"); var file = "test.db"; var exists = fs.existsSync(file); If a file does not exist, it will be created, otherwise the file will be opened for access var sqlite3 = require("sqlite3").verbose(); var db = new sqlite3.Database(file); db.serialize(function() { //Do stuff... }); Lennart Herlaar - UU

Creating a table We will create a table only if this is the new DB file db.serialize(function() { if(!exists) { db.run("CREATE TABLE Stuff (thing TEXT)"); } }); Lennart Herlaar - UU

Inserting data // Start preparing the statement var stmt = db.prepare("INSERT INTO Stuff VALUES (?)"); //Insert random data var rnd; for (var i = 0; i < 10; i++) { rnd = Math.floor(Math.random() * 10000000); stmt.run("Thing #" + rnd); // running the statement } // Finishing the statement. Not necessary, but improves performance stmt.finalize(); Lennart Herlaar - UU

Querying data db.each("SELECT rowid AS id, thing FROM Stuff", function(err, row) { console.log(row.id + ": " + row.thing); }); Lennart Herlaar - UU

Putting it all together var fs = require("fs"); var file = __dirname + "/" + "test.db"; var exists = fs.existsSync(file); if(!exists) { fs.openSync(file, "w"); } var sqlite3 = require("sqlite3").verbose(); var db = new sqlite3.Database(file); db.serialize(function() { db.run("CREATE TABLE Stuff (thing TEXT)"); var stmt = db.prepare("INSERT INTO Stuff VALUES (?)"); var rnd; for (var i = 0; i < 10; i++) { rnd = Math.floor(Math.random() * 10000000); stmt.run("Thing #" + rnd); stmt.finalize(); db.each("SELECT rowid AS id, thing FROM Stuff", function(err, row) { console.log(row.id + ": " + row.thing); }); db.close(); Putting it all together Never forget to close a DB Lennart Herlaar - UU