From bare metal to cloud to serverless
I’ve been away from my blog for a while and while I haven’t finished my series on the You Don’t Know JS books (I still highly recommend them for anyone interested in JS) My lack of routine posts was starting to make this site look like an ad page for someone else’s book series and I’ve not had much time to talk about other cool projects I’ve been working on and playing with. With that said today I’d like to talk about a relatively new concept called «Serverless».
What does Serverless mean?
What does that even mean? How can you have a website/application and not have servers? Well the fact of the matter is you can’t. The name might be slightly misleading but but it’s not without merit. Serverless doesn’t mean to imply that there aren’t any servers but rather that you don’t manage your servers. You consume other services managed and secured by someone else and operate your platform on top of their infrastructure.
How did we get here?
In the beginning there was Bare Metal
When I started my career everyone ran on what’s now often referred to as bare metal. All this simply meant was that you bought or provisioned a physical server somewhere. In early 2000’s for the company I worked for this meant having a multi-million dollar facility that we built from the ground up with several rows of cabinets, hundreds if not thousands of feet of networking cables, Firewall appliances, backup machines with robotic tape loaders, massive cooling units and a dedicated team of staff to build, monitor and maintain both the physical hardware and the software installations. At this same time there were many hosting companies that offered IaaS where you could rent, lease or co-locate your servers saving you varying amounts of the overhead involved in having your own hardware.
Then there was VPS
Next came VPS another IaaS offering where hosting companies realized they could build higher end servers and divide the resources of the machine into several virtualized instances and offer better pricing to the customer while still giving them the feel and control of having their own server.
Soon we wanted to live in the Clouds
This quickly evolved into cloud computing where hosts created API interfaces into their provisioning allowing for more robust flexibility and dynamic scalability. Also virtualization was moved to clusters of machines allowing for more reliability in the event of hardware failures.
Cloud computing led to a revolution in DevOps. Automation of software provisioning with API connections into server provisioning. This allowed application developers and maintainers to create robust environments where their infrastructure could be more reactionary to the state of their application. We were no longer dedicating teams of hardware and operating system specialists as sysadmins but we were either expecting our sysadmins to abandon their hardware expertise for software development skills or our software developers to become sysadmins. This was because these DevOps platforms were built using developer tooling like Ruby and Python.
In 2014 Amazon via AWS introduced a new product called Lambda. This was a new type of offering that was a Function as a Service (FaaS), but what does that mean? If you think of your website as a series of static content connected to and driven by a series of APIs FaaS means that the executable code behind your API becomes the unit of resource that you pay for. Lambda functions in combination with AWS API Gateway allow you to use the high availability of the AWS platform to manage your scale. AWS now becomes your security provider managing large amounts of infrastructure needs for you and freeing you from sysadmin and DevOps. Combining FaaS with one of the many popular DBaaS platforms means you can now start striping away the need for managing your own infrastructure.
How does it all come together
While there are many other players who have come to the Serverless party since Amazon’s release of Lambda as a FaaS platform AWS is still the most refined of the platforms. A few frameworks have come together over the last year or so, there’s the DEEP Framework, Serverless (formerly JAWS), PAWS, Sparta, Apex and more. The one I’ll be focusing on today is Serverless. This application framework was first and foremost designed for use with AWS but as they prepare for their 1.0 release they are currently working with Google Cloud Functions, Microsoft Azure, IBM OpenWhisk and others to bring Serverless to many more providers.
Drinking the AWS Kool-Aid
Until the other players catch up and the space and frameworks mature making a completely serverless application currently requires drinking a little bit of the AWS Kool-Aid. The more you embrace the AWS platform the more freedom you begin to have from operations management and worrying about scaling.
Architecture of an AWS based Serverless Application
What does the deployment architecture of a serverless application look like? While that may vary from application to application and while most of the pieces I’ll present to you can be swapped out for pieces from other providers this is what it could look like if you drank all of the AWS Kool-Aid
What are all these pieces?
- Route53: DNS server
- CloudFront: CDN and edge cacheCORS
- CloudFront logs: HTTP Access Logs
- S3 Bucket: Static file container
- Cognito: User management
- APIGateway: HTTP Proxy to API functions
- CloudWatch: Proxy and application logs
- Lambda: Functions to be triggered by events either HTTP or otherwise
- IAM: Identity Access Management
- SQS: Message queuing service
- DynamoDB: NoSQL based DBaaS
How do I get started?
As I write this blog post Serverless is in the process of preparing to transition from v0.5 to v1.0 which will see some breaking changes in the framework. V1.0 is currently in alpha and a series of blog posts have been published on migrating your project. Check out the amazing Getting Started guide, GitHub, or sample projects. Also there is a growing list of plugins, many of which I strongly recommend including;
- Meta Sync - Securely sync your the variables in your project’s _meta/variables across your team.
- Offline - Emulate AWS Lambda and Api Gateway locally to speed up your development cycles.
- CORS - Adds support for CORS (Cross-origin resource sharing).
- Webpack - Use Webpack to optimize your Serverless Node.js Functions.
- Serverless Client - Deploy and config a web client for your Serverless project to S3.
- Optimizer - Optimizes your code for performance in Lambda. Supports coffeeify, babelify and other transforms
- CloudFormation Validator - Adds support for validating your CloudFormation template.
- Prune - Delete old versions of AWS lambdas from your account so that you don’t exceed the code storage limit.
- JSHint - Detect errors and potential problems in your Lambda functions.
- ESLint - Detect errors and potential problems in your Lambda functions using eslint.
- Mocha - Enable test driven development by creating test cases when creating new functions
- Auto-Prune - Delete old AWS Lambda versions.
- Serverless Secrets - Easily encrypt and decrypt secrets in your Serverless projects
In the coming months I will likely publish additional blog posts about Serverless particularly as 1.0 moves out of Alpha. I’ve been using Serverless daily for a few weeks in doing initial prototyping for a new startup and I plan on expanding my use of Serverless to personal projects and even converting this blog to a Serverless Application that I will open source.