Friday, October 6, 2017

What The Internet Never Told You About Building A Login System

The Internet is replete with "how to build a login form" and "which password hashing algorithm to choose" articles but strangely silent on how to build a professional (well, maybe, semi-professional) login system.

Any login system has two sides: the client side and the server side.  The client side is easier so we'll start there.

To login, a user should provide a username or email address and a password.  For your system, well, which should it be: a username or an email address (or maybe a choice)?

Email addresses are more consistent and, chances are, you'll want to verify accounts by requiring a verified email address.  On the other hand, usernames are shorter and have a little more flair though a user is far more likely to forget their username than their email address.

Many clients send the password in the clear to the server.  But, when I researched the issue, I opted for a more sophisticated approach.  It is:


SHA-256 is a rather insecure hashing algorithm.  The point here is not to secure the password so much as invalidate an attacker's database of SHA-256 hashes and force him to generate a new one.  An attacker might already have a huge database of SHA-256 password hashes.  But, by adding the username or email address to the password string and then doing the SHA-256 hash, an attacker will have to have a much, much larger database of SHA-256 username-password hashes.  Adding an arbitrary string, such as the name of your web site, to the username-password will force an attacker to generate a new database of SHA-256 hashes customized for your login.

This hash is sent from the client to the server as the user's "password".  If an attacker steals your database and recovers the client-hashed password, they will be able to login to your site but the password won't work at any other site.  The arbitrary string will ensure that the hashes are different.  The attacker will be forced to recover the plain text password.

Having client-side SHA-256 hashes, even with usernames and arbitrary strings, isn't enough.  You are protecting other sites (a little bit) from being vulnerable to broken into if your password database is stolen.  But you are providing no protection for yourself (or your admin accounts) and SHA-256 is far too weak to be effective all on its own.  Enter server side hashing.

Like I said earlier, the server side of a login system is more complicated than the client side of the login system.  What server side hashing algorithm should you use?

MD5, SHA-256, PBKDF2, bcrypt, scrypt or Argon2?

Well, no.  You shouldn't choose a hashing algorithm at all.

Hashing algorithms and hardware change over time and, since you are building professional system, meant to adapt over the years, your server should support multiple algorithms.  Your login system, at least the server side, has three requirements:
  1. Hashing algorithms must be assignable per user, not one for the entire system
  2. Each user's hashing algorithm has to be changeable into any other algorithm
  3. Adding a new hashing algorithm should only require one function to be changed
No doubt, your database has a "password" column that contains the user's hashed password.  Having a password in a single string is extraordinarily convenient.  Rather than storing hashing algorithms in a separate column, it is best to store the hashing algorithm in the same column as the hashed password, as part of the same string.  Ideally, you'd use a standard password string format.

As far as I'm aware, there is only one such format in wide use: that's the Linux /etc/shadow password format (a.k.a. the shadow password format).  It looks like this:


The algorithm field indicates the algorithm:


The arguments field might not exist or might contain algorithm arguments.  bcrypt requires a work factor.  Argon2 has 3 different arguments: iterations, memory and threads.

The saltthenpasswordhash is the salt with the hashed password for this user.  Other articles (by other people) describe the reasons why each password should have its own salt and not a global salt.  Nowadays, it is common for libraries to automatically generate a salt when creating new hashes so there is less reason for me to go over those arguments here.  Since salts are generated along with the hash, it is easier to do it the right way and harder to do it the wrong way, with a global salt.

Storing all this information in a single string in the "password" column of your database using the standard /etc/shadow format will satisfy the first requirement that hashing algorithms be assignable per user.

For new users, I use a global variable to indicate the hashing algorithm that a new user should use so, when a new user is created, their password hash and their hashing algorithm (from the global variable at the time the account was created) is stored together.

When a user logs in, they use the hashing algorithm in their password string from the "password" column of the database to hash the SHA-256 client-side hash.  Then, the hash from the login attempt is compared to the hash from the database, preferably using the special hash comparison function from the encryption library (to blunt timing attacks).  If the hashes are not equal, the login attempt fails.

If the hashes are equal, the login attempt succeeds but you're not quite done yet.  The user's current hashing algorithm is compared to the hashing algorithm in the global variable.  If they are not the same, the client-side SHA-256 hash (which has already be validated to contain the correct password) is hashed using the global variable's hashing algorithm.  The result, in shadow password format, is stored into the "password" column: the user has been upgraded on-the-fly to the latest algorithm.

The global variable provides the ability to "turn up" and "turn down" security by specifying different algorithms.  When the global variable is changed into a stronger (slower) algorithm, new users and any user logging in will be upgraded to the new algorithm seamlessly.  If you make a mistake and deploy an algorithm that bogs down your server, you can "turn down" security merely by changing that algorithm global variable.  Existing users will undergo a one-time delay when they login using the too-slow algorithm and are downgraded to a faster algorithm.

Designing your login system such that a single function provides hashing new passwords, hashing a password with a given salt (for login checks) and comparing two hashes will allow you to deploy new hashing algorithms over the years by changing one function.

Rather than just sending passwords in the clear and hardcoding MD5 (oh, Yahoo!, will you never learn?!), a login system with client-side hashes and upgradable server-side hashing algorithms will provide you a basic framework for the future.

I don't claim these practices to be the best or even adequate.  I'm not a security researcher and don't claim to be one.  But SHA-256 client side hashes, the shadow password format and easy deployment of new server side hashing algorithms are important issues that are worth debating and considering.  And, hopefully, will give you a place to start the conversation.

Friday, September 22, 2017

AngularJS to React/Redux and back again

I've found AngularJS to have many analogues (corresponding pieces) to React/Redux.  Let me show you.

A Redux state almost exactly corresponds to an AngularJS $scope variable.

In a Redux reducer file, you have an initial state that looks like this:
const initialState = {
  inited: false,
  loading: true,
  user: {
    first: '',
    last: '',
    email: ''
  error: ''

This initial state almost directly corresponds to an AngularJS $scope initial state.  If it doesn't, you can easily move the AngularJS variables that are outside the $scope variable into the $scope variable to match up exactly.

Not every AngularJS controller has an "inited" variable but many do.  In Redux, an "inited" variable is essential to signal that the Redux state has not been populated by data from the backend yet.  An easy way to do this is to check the "inited" prop in the componentDidMount() method in the React component that requires it (or any React component that requires it) and, if it is false, load up the data and set the "inited" value (well, call an action to call a reducer to do it) to true.  Even if you don't have an "inited" variable in your AngularJS $scope variables, it is easy to add one.  The AngularJS controller function might be converted like this:
componentDidMount() {
  if (!this.props.inited) {
    // do this only once
    this.props.model('inited', true)
    // in AngularJS, this is the controller function
Speaking of Redux reducers and so on (and since I brought it up in the code directly above), it is very convenient and useful to make a "model" action along the lines of an AngularJS ng-model.  The "model" action will take a string indicating the value to change, e.g. props.model('user.first', 'John'), and ultimately create something that looks like this:
<input type="text" value={ props.user.first } 
  onChange={e => props.model('user.first',}/>
Often, I've found it easy to copy the HTML from my AngularJS templates (HTML) into the React render() method and fix them up.  There are a three tricks that you have to remember:
  1. Do a global search and replace on "class=" to replace with "className=".  Before you make this change, your React component may render but will not apply the CSS classes.
  2. Search for each "style=" (e.g. style="background: blue") and convert each instance (e.g. style={ { background: 'blue' } }).  (Don't forget to make it a JavaScript object by having the double curly braces.) . Really, if your AngularJS still contains "style" attributes, it's just better to convert them into CSS classes and "class" attributes rather than go through the hassle of converting them to React.  Before you make this change, your React component will often render a blank screen; this is a big hint that you have unconverted "style" attributes.
  3. Change each AngularJS template to have a single root element, not multiple root elements.  Multiple root elements are allowed in AngularJS but there is no reason to use them and it is easy enough to just wrap it all in a div tag so it is easy to convert to React.  React components (specifically, JSX and the render() method) require a single root element.
react-router-redux is a pretty convenient analogue to AngularJS router modules (either the built-in or ui-router).

Factories and services can be stripped of their AngularJS specifics and placed in your React application's modules folder.  Usually, I place them in a subfolder of modules.  Often, factories and services use the AngularJS $http service; you can convert these calls to use fetch (from the isomorphic-fetch npm module) or Axios (or superagent or request or whatever).

AngularJS filters are essentially just functions; I just strip out the code and drop them into the modules folder itself.  A call to a filter is easy to convert.  In AngularJS, a filter looks like this:
{{ 'mytitle' | translate }}
In React, it becomes:
{ translate('mytitle') }
In AngularJS, we have the "ng-if" and "ng-repeat" attributes which are used a lot in almost every AngularJS application to do control flow.

The AngularJS "ng-if" attribute is used like this:
<div ng-if="showing">
  My title
In React, it becomes a conditional, like this:
{ props.showing? (
  My title
) : '' }
An AngularJS "ng-repeat" attribute is a little more complicated.  From this:
<th ng-repeat="column in columns">
  {{ column }}
To this in React:
{ props.columns.length? props.columns
  .map((column, index) => (
<th key={ index }>
  { column }
  ).reduce((p, c) => [p, '', c]) : ''}
Admittedly, this post is a little slapdash and not really enough to convert an entire AngularJS to React/Redux. But I think that you will agree that there are a lot of concepts that easily port from AngularJS to Redux with a little tweaks, here and there.

Wednesday, September 20, 2017

What's wrong with Ember.js?

Once you have React and Angular in your toolkit, should you concern yourself with Ember.js?

Executive summary: No.

React comes from Facebook; Angular comes from Google.  If you live in the Silicon Valley like I do, Facebook and Google cast a long shadow.  Many companies, large and small, "cargo cult" Facebook and Google; that is, they adopt technology blindly based on what Facebook and Google do, hoping that that imitating success will yield real success.  Many startups adopt "hot" technologies to appeal to venture capitalists ("hey, we're just like Google!") and to garner attention from developers and the press.

Right now (9/20/2017), React is in the cat bird seat.  React developers command the extravagant salaries and most of the pursuit by recruiters.  In contrast, AngularJS (1.x) is a workhorse: valued by many companies but also some willing to leave positions open rather than paying a premium and others paying a limited premium in developer salaries.  Angular (2/4) is rarely seen: companies who adopted Angular (2/4) seem oddly out of touch, like they didn't get the memo, and seem to have little interest in paying salary premiums.

Where is Ember.js?

Ember.js is at LinkedIn, a respectable name to be sure, but hardly the influencer that Google and Facebook are.  Ember.js is also seen at the occasional AI or hardware startup.  No salary premium for Ember.js developers: those companies think that being an Ember.js developer shows that you are a free-thinking hippie that cares more about concepts than money.

Ember.js gets an honorable mention by those job postings that read, "must know at least one of React, AngularJS, Ember.js or other JavaScript framework".  Hey, that's something: Vue.js doesn't get this kind of respect.  But we all know that, when a job posting reads like that, it really means, "Hell, we're don't know jack about any of these JavaScript frameworks so, if you've done any of it, it's all the same to us."

Recently, I did a multi-day investigation and technical review on Ember.js and, while Ember.js seems technically competitive with React and Angular in some respects, there was no indication that they were ever going to emerge from their rather distant third place, open source, hippie-dippie "alternative" framework status.  Having recently reviewed Backbone.js, I see a lot of intellectual inheritance from Backbone.js in Ember.js and, believe me, that's not a compliment.  That smell of Backbone.js in Ember.js makes it feel ... old.

I have to give the Ember.js team respect for hanging it there all this time.  But, sorry, I will not be staking my future on Ember.js, either now or anytime soon.  Nor will I be considering any Ember.js positions.  And I'd advise others not to, either.

But, hey, Ember.js, if, someday, you break through, give me a call.  I'm in the book.

Tuesday, September 19, 2017

What was Backbone.js and why did it die?

As AngularJS (Angular 1.x), Angular (Angular 4) and React rose, Backbone.js declined. What was Backbone.js? Why did it decline?

Let's just dive in and look at some Backbone.js design and features.

Backbone.View.extend(), Backbone.Model.extend(), Backbone.Router.extend(), etc.

Backbone.js is based on a Java-style class structure with classes, objects and inheritance. It makes a conscious choice to embrace this approach, instead of JavaScript prototypical inheritance. The extend() method creates a subclass of the "class" which it is invoked upon. While other frameworks adopt this approach as well (usually by upgrading to TypeScript or ES6) for their view classes, Backbone.js is rather dogmatic about it and, as we will soon see, it requires a lot more classes and a lot more objects whereas other frameworks were more limited and less, well, so darned adamant about creating all those things.

_.bindAll() call in a Backbone.View constructor

When a new Backbone.View class is created, its constructor often contains multiple calls to _.bind() or a single call to _.bindAll(). I have written about the function of _.bind() before so I won't repeat that here. OK, OK, I'll repeat it a little. _.bind() converts a method into a standalone function call. In Backbone.js, this is often required because event handlers only work with functions, not with methods. Backbone.js uses events extensively so the _.bind() calls are necessary. Calls to _.bind() and _.bindAll() confuse many people and cause odd errors when not used so Backbone.js got a reputation as being confusing.

React requires a bind() call in some but not all circumstances but it is not designed around events in the same way that Backbone.js is. The use of _.bind() and _.bindAll() isn't a deal killer but certainly doesn't help Backbone.js' reputation.

render() method of a Backbone.View

Every Backbone.View class has a render() method, similar to React. The render() method in Backbone.js is usually a string concatenator, something like an old-fashioned version of React, before JSX was widely adopted. In Backbone.js, it is additionally encouraged to call the render() methods of child Backbone.View objects, take the returned strings and insert them in the correct position in the parent Backbone.View string. All this string concatenation feels very awkward and old-fashioned.

events property of a Backbone.View

The events property looks something like this:

events: {
 'click button#add': 'addItem'

It is a map with magic strings that makes nobody happy.  The “click” refers to the onclick event; the “on” prefix is dropped.  "button#add” is a jQuery selector.  “addItem” is the method to invoke on the current object.

Backbone.js is the one framework that separated event handlers from the HTML. This approach provides no benefit to the developer and is somewhat inconvenient and buggy.


Backbone.js separates views and data; Backbone.Model was the base class for data. Instead of working with plain old JSON objects like many other frameworks, Backbone.js expects JSON objects (from the server) to be deserialized into full blown Backbone.Model objects.

Backbone.js data objects support events and event handlers (as do view objects). The idea seems to be that, once JSON data is converted into full-blown Backbone.js objects, the view objects and data objects are stitched together with the right events.

The Backbone.js data objects is an alternative to AngularJS' $scope variable or React's props variable (argument). Backbone.js event mechanisms require the developer to orchestrate how data changes propagate to the view. In contrast, AngularJS' digest cycle and React's virtual DOM and diffing algorithms take care of this orchestration for the developer.


Backbone.Collection is the Backbone.Model version of an array. Instead of using ordinary JavaScript arrays, Backbone.js requires the developer to deserialize arrays into collections so that events and event handlers can be used to monitor data changes.


Backbone.sync is an Backbone.js specific implementation of something like GraphQL but is incompatible with GraphQL. Its API is reminiscent of server side persistence libraries like Hibernate with methods like fetch() and save(). Backbone.sync uses HTTP operations like GET, POST, PUT and DELETE to perform server CRUD operations using pre-determined URLs. A server needs to confirm to Backbone.sync's conventions in order to work with Backbone.sync.

GraphQL has attained some level of popularity due to its support from Facebook; Backbone.sync never has.


Backbone.Router supports partial views. It seems OK.

Underscore.js templates

Backbone.js supports HTML templates via Underscore.js.

An Underscore.js HTML template looks like:

<h1><%= title %></h1>
<% if (title === 'first') { %>
This is the first title <% } %>

Backbone.js also supports other libraries like Mustache but these must be downloaded and integrated separately. This presents a problem: you cannot re-use templates that were written in one library with a different library so templates were mostly not reusable.

Backbone.js does not provide any support for dynamic data in the template. In AngularJS, Angular or React, if the data changes, the template (or JSX) is automatically re-rendered whereas, with Backbone.js templates, the developer must detect the change, invoke the template directly and re-render the updated template in the right place.

Also, Backbone.js does not provide any support for (reusable) components. In AngularJS, directives allow the developer to insert a component with HTML and behavior. Similarly, JSX in React allows a developer to insert a component with HTML and behavior.

Backbone's weakness in templates and lack of component functionality is one of its biggest weakness and an important reason why it is not popular.


Backbone.js is a very awkward framework, requiring much of the developer trying to use it.

It is also a very ineffective framework, lacking popular features while providing other features that were not that useful.

Backbone.js is very imperative, manually connecting everything with events and event listeners, instead of declarative, changing JSX (in React) or changing HTML templates (in AngularJS) to invoke the event handler.

Saturday, September 12, 2015

The Eh (Canadian) programming language

I know a guy who is Canadian and, just for fun, I wrote a programming language for him.

The language is the Eh programming language.   The syntax is similar to JavaScript, except that instead of a line ending with a semicolon, it ends with a comma, a space, the word "eh" and a question mark (?).  For example:
var src = 'script', eh?
var s = '', eh?
s += 'Hello', eh?
s += ' ', eh?
s += 'World', eh?
s += ' from ' + src, eh?
console.log(s), eh?
This will print the following to the JavaScript console:
Hello World from script
An Eh script can be included inline using the "text/eh" type:

Or loaded from an external script file:

Here's the source code for the Eh compiler (in JavaScript) named eh.js:
// eh compiler
window.onload = function() {
  // execute eh code
  function executeEh(src) {
    eval(src.replace(/,\seh\?/g, ';'));
  // load eh scripts
  var scripts = document.getElementsByTagName('script');
  for (var s=0; s < scripts.length; s++) {
    if (scripts[s].type === 'text/eh') {
      if (scripts[s].src) {
        // execute external eh scripts
        var extScript = new XMLHttpRequest();"GET", scripts[s].src, true);
        extScript.onreadystatechange = function() {
          if ((extScript.readyState== 4)
              && (extScript.status == 200)) {
      } else {
        // execute inline eh scripts
Eh supports all major JavaScript libraries and APIs.

Here's a Plunker to demonstrate:

If you make an Eh script, comment below and let me know!

Tuesday, September 1, 2015

Put your Git branch in your Mac or Linux prompt

There is a lot of examples out there but here's a basic way to show your Git branch in your profile.

First, edit your .bash_profile.
$ vi .bash_profile
Press "i" to put vi in "insert mode".  Then, add the following to your .bash_profile:
ps1_git_branch() {
git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/ (\1)/'
export PS1="\w\$(ps1_git_branch)\$ "
After making this change and starting a new Terminal, the prompt will show the following in non-Git folders:
~/Documents $
In Git folders, the prompt will show the following (if you are on the "master" branch):
~/Documents/git/code (master)$
Very convenient.

Friday, August 14, 2015

JavaScript private data using createPrivateThis()

In JavaScript, all object properties are public.

But, coming from Java and a long line of languages before it, I've seen how private properties can enforce better designs.  Is there a convenient way to enforce private properties in JavaScript?

Up to now, there were four proposed solutions:
  1. Imaginary privacy: Put an underscore (_) in front of all private properties to indicate that they are private.
  2. Everything in the constructor: Use closure and attach all your functions to the this object; don't use Object.prototype.func = anymore.
  3. Weakmaps:
  4. Everything public: Why care?
I am now proposing a fifth solution: createPrivateThis().

My objectives were simple:
  1. Provide Java-style public and private data in JavaScript.
  2. Prevent read access to the private data from outside the object.
  3. Allow functions to be created on the prototype object as normal.
  4. Allow functions on the prototype object to access public and private data using the this pointer.
Here's the code that I came up with:
 * Wrap the prototype functions and put the wrappers on the
 * object itself.  A wrapper merges the object and private
 * properties before the call, makes the call then unmerges
 * the object and private properties.
 * Normally, you create your private data in the constructor
 * function and give distinct names to public and private
 * data.  Then, during function calls, you use the this
 * pointer for BOTH public and private data.  In ambiguous
 * situations, the private data is operated on with the
 * exception that adding/deleting data defaults to the
 * (public) object.
 * It is possible to create private functions and variables
 * with the same name using the transitory getPrivateThis()
 * method but this is NOT recommended and is an advanced use.
 * @param {Object} obj The object that needs private data.
 * @return {Object} The private data pointer (this_p).
function createPrivateThis(obj) {
  // use Underscore.js or our own mini version
  var __ = (typeof _ === 'undefined')? {
    isFunction: function(f) {
        return f 
           & {} == '[object Function]';
    has: function(o, k) {
      return o.hasOwnProperty(k);
    keys: function(o) {
      return Object.keys(o);
    clone: function(o) {
      var k, o2 = {};
      for (k in o) {
        if (o.hasOwnProperty(k)) {
          o2[k] = o[k];
      return o2;
    each: function(oa, f) {
      var i, k;
      if (oa.constructor === Array) {
        for (i=0; i < oa.length; ++i) {
      } else {
        for (k in oa) {
          if (oa.hasOwnProperty(k)) {
            f(oa[k], k);
    extend: function() {
      var a, arg, d = arguments[0];
      for (a=1; a < arguments.length; ++a) {
        arg = arguments[a];
        for (var k in arg) {
          if (arg.hasOwnProperty(k)) {
            d[k] = arg[k];
      return d;
    union: function() {
      var a = Array.prototype.concat.apply({}, arguments)
      var b = [], a1, i;
      while (a.length > 0) {
        a1 = a.pop();
        for (i=0; (i < b.length) && (a1 !== b[i]); ++i) ;
        if (i === b.length) {
      return b;
  }: _;
  var priv = {};
  var getPriv = function() {
    return priv;
  // iterate over each function in the object prototype
  __.each(obj.constructor.prototype, function(value, key) {
    if (__.isFunction(value) && !__.has(obj, key)) {
      // intercept the function on the object itself
      obj[key] = function() {
        var pre = __.clone(priv);
        // combine the object and its private properties
        var full = __.extend({}, obj, priv);
        // add the getPrivateThis() access function
        full.getPrivateThis = getPriv;
        // invoke original function with combined object
        var r = value.apply(full, arguments);
        // remove the getPrivateThis() access function
        delete full.getPrivateThis;
        var objKeysToUpdate = {}, objKeysToDelete = {};
        var privKeysToUpdate = {}, privKeysToDelete = {};
        var allKeys = __.union(__.keys(pre), __.keys(priv),
          __.keys(obj), __.keys(full));
        // decide what to do with every key that we ever saw
        __.each(allKeys, function(key) {
          if (__.has(pre, key) !== __.has(priv, key)) {
            // private property was added or deleted
            // using getPrivateThis()
            if (__.has(full, key)) {
              // add/modify object property with the same name
              objKeysToUpdate[key] = full[key];
            } else if (!__.has(full, key) && __.has(obj, key)) {
              // delete object property with the same name
              objKeysToDelete[key] = 'delete';
          } else if (__.has(full, key) && __.has(priv, key)) {
            // modify private property
            privKeysToUpdate[key] = full[key];
          } else if (!__.has(full, key) && __.has(priv, key)) {
            // delete private property
            privKeysToDelete[key] = 'delete';
          } else if (__.has(full, key) && !__.has(priv, key)) {
            // add/modify object property
            objKeysToUpdate[key] = full[key];
          } else if (!__.has(full, key) && __.has(obj, key)) {
            // delete object property
            objKeysToDelete[key] = 'delete';
        // do what we decided to do with the keys
        __.each(objKeysToUpdate, function(value, key) {
          obj[key] = value;
        __.each(objKeysToDelete, function(value, key) {
          delete obj[key];
        __.each(privKeysToUpdate, function(value, key) {
          priv[key] = value;
        __.each(privKeysToDelete, function(value, key) {
          delete priv[key];
        return r;
  return priv;

The core idea is to create wrapper functions on the this object which automatically intercept the calls to the functions defined on the prototype object.  The automatically generated interception functions combine the this and the private data, make a call to the prototype function, then uncombine the data back into the this and the private data.

In practice, createPrivateThis() is used like this:
function Animal() {
  var this_p = createPrivateThis(this);
  this_p.nickname = "NoName";
  this.type = 'animal';

Animal.prototype.getPrivateNickname = function() {
  // this.nickname is private but we can access it inside here
  return this.nickname;

Animal.prototype.setPrivateNickname = function(nick) {
  // this.nickname is private but we can access it inside here
  this.nickname = nick;

Animal.prototype.getPublicType = function() {
  return this.type;

Animal.prototype.setPublicType = function(type) {
  this.type = type;

The private property, nickname, is not accessible outside the object.  It is accessed via the this pointer but it does not actually reside on the object.  The private property actually exists as a local variable in the createPrivateThis() function that is accessed via closure (only available inside the function).