The last few months at Lob, our engineers have started to create Lob’s new scalable API infrastructure. As we begin transitioning to this new infrastructure, we want to discuss the technologies we have decided to use.
At the infrastructure layer, we are hardcore AWS fans. Our entire infrastructure at Lob is built on AWS. Consequently, we view the platform as a major driver of productivity, especially for a company of our size. Below is a diagram of our system architecture and the AWS services we use.
The two driving factors behind our API design have been language independence and modularity. We want to give ourselves the flexibility of using any language or framework to write a given endpoint of the API. For example, on our address verification endpoint (which is pinged several times every second) Node.js is a logical choice because of its ability to handle a large number of concurrent requests. However, on our more computation-heavy endpoint for parsing and validating objects we may want to take advantage of the large number of libraries and tools available in a language like Ruby or Python.
By building our API in a modular way where each endpoint is its own internal API, we can pick and choose which language we want to use for current or future endpoints. That said, we don’t want to internally support too many languages so we usually try to stick to 1 or 2 preferred languages. Right now we predominately use Node.js and want to talk about some of the tools we have been using to build our custom MVC and APIs.
Node.js has been a great boost to productivity for the Lob team. The asynchronous nature of the language and its ability to handle a large number of concurrent connections fits well into the architectural needs of our company. The Node.js community and strong support for npm means there are usually several open source libraries for performing nearly any task. There are also a large number of productivity tools available to help developers write high quality code more quickly. Below you will find information on some the Node.js tools that help us build at Lob.
Hapi is a newer framework and is a “simple to use configuration-centric framework”. It has built in support for input validation, authentication, and other essential facilities that make it perfect for building an API . Aside from being a great API focused framework, it has built in support for view engines and makes working with Handlebars (our preferred engine) seamless. Hapi is also a very readable framework so the learning curve isn’t as steep as other fraweworks.
We back our current API offerings with a relational database storage rather than a NoSQL storage system which you will see in a large number of Node.js web apps. In order to interface with the relational database in Node.js we started using a great ORM called Bookshelf.js. Bookshelf.js is built on top of the Knex query engine and “extends the Model & Collection foundations of Backbone.js”. Bookshelf.js provides great support for defining models, inheritance, eager/lazy loading, and multiple database engines (MySQL, PostgreSQL, and SQLite3).
We currently use Mocha for testing our Node code. There are a large number of great testing libraries in Node, but below are a few reasons why we enjoy using Mocha.
We have been using Gulp.js in order to speed up the development process by automating common tasks. Below is an outline the workflow we are currently using with Gulp.
.png)
.png)
.png)