A 1 year review of Laravel Vapor

8 months ago, we wrote about how we rebuilt Fathom Analytics from the ground up and moved to Laravel Vapor. Thousands of people have read it, and it’s one of the most popular articles on the Fathom blog. Beaten only by Paul’s Make your website a black hole to big tech post. (I know you bought those page views, Paul).

But enough about Paul’s shady dealings, let’s talk about Laravel Vapor. A quick background for those of you who are unaware, Laravel Vapor is a serverless deployment platform for Laravel (the beautiful framework we use for Fathom) which allows us to focus on building our application and not servers. Before you get smart with me, yes, I did spend a few months working with servers for our custom domains solution, but that is a tiny part of our application and requires zero maintenance from us, because it runs on T3 burstable instances and it has global, multi-region availability. So now I’ve flexed on that, let’s calm down and get into the review of Vapor. We’re now at the mark where we’ve been using Laravel Vapor for almost 1 whole year (our 1 year anniversary will be 22nd August 2020).

How easy is Laravel Vapor to use?

Getting a project set-up on Vapor is a piece of delicious cake. A standard Laravel app can be deployed ridiculously fast, and a staging environment can be deployed in seconds too. The documentation is very comprehensive, and has everything you need to get going. And in terms of additional education, the Serverless Laravel course (that I ended up creating after the success of my Vapor article on this blog) is a great resource for learning by watching, as it covers a lot of advanced use cases, and over 500 people have bought it so far.

In terms of the technical side of things, it’s literally a case of adding some packages to your project, authenticating with Vapor and then pushing things up. As with any new tool, there is going to be learning, but you won’t have to spend time configuring servers, queues etc. which is a huge blessing. We don’t have to plan out how many servers we need to load balance, we don’t have to load test, we can just get going.

How much does Laravel Vapor cost to run?

One of the big things that people are curious about is how much Laravel Vapor costs to use. Ultimately, we cannot give a generic cost as it varies per project. Some people are spending $50-100 and other, larger companies are spending a few thousand dollars each month. At present, there’s no simple resource for calculating your AWS costs, but you can attempt it yourself from the AWS website. The pricing has been reasonable for us at a few thousand dollars a month, and we’ve not had to hire any full-time DevOps person to help. Remember, I’m really not a server guy. I know enough to get by but I don’t enjoy it. The fact that we run our enterprise grade infrastructure for a few thousand dollars is so awesome to us (which is less than the cost of hiring a top-level DevOps person). This is far cheaper than hiring a full-time staff member. We also have incredible global availability, and we sleep so damn well at night.

How reliable is Laravel Vapor?

One of the big things that people care about is reliability for production applications. Firstly, let’s talk about uptime. Our collector Vapor environment hasn’t gone offline at all in the last 8 months. We had some fun with SSL certificates on our dashboard (I will never use AWS technical support again) but our collector has been incredibly stable. We’ve had huge spikes, and it’s not gone offline once. We have a pretty steady amount of page views coming in, but occasionally a big customer will send out an email campaign, and we’ll be flooded. Zero negative performance impact, it’s been amazing.

And sure, we’ve had moments where we’ve had big SQS backlogs (due to the way I coded some parts of our application early on) but we’ve never lost data. This is huge - no customer of Fathom has ever lost even a single page view from being tracked. (Can our competitors say this? Nope!) The SQS backlogs are actually fantastic. Sure, there’s a slight delay in Current Visitors for our customers when these rare backlogs occur but the data still gets aggregated. I had a fun day a while back where we saw a backlog of 100,000+ jobs, due to a bunch of transaction lock-fights. Long story short, I fixed it all up, increased the concurrent executions for the queue and it was all processed rapidly. Don’t glance over that sentence. By simply adjusting a single line in my Vapor configuration file, I was able to instantly provision hundreds of Lambda containers to work through our backlog of jobs. Comparatively, if that was in a non-serverless environment, we’d have to roll out extra servers / workers manually to deal with this. Additionally, SQS can handle an unlimited amount of jobs, so that “but what if Redis gets full” stress is gone.

So overall, we’re incredibly amazed with Vapor and it’s passed the “is Vapor production ready” test with flying colours (no, this isn’t a typo, my American friends). Vapor has been able to handle thousands of our clients processing an incredible amount of page views with total ease.

How fast is Laravel Vapor?

In terms of speed, it’s been great. We initially ran our application behind a US based Application Load Balancer (easily set-up in Vapor), but whilst this was cheap to run, we wanted things to be faster globally, so we rolled out an API Gateway. This meant that we had an edge-optimized deployment and, sure, our servers are still in the US but the visitor’s connection to the edge, and then from the edge to our US server, is much faster. We typically see around a 100ms response time, but that’s with a conservative CPU / Memory configuration. You can get it faster by being smart with your requests.

One downside with speed, is that there can be small lags when connecting to RDS from Lambda. This ultimately comes down to Lambda and isn’t down to Vapor, but I still wanted to mention it. However, there have been recent breakthroughs on this and, on 30th June 2020, AWS released RDS Proxy support for MySQL and PostgreSQL. So I’m confident that the Vapor team will cook up something wonderful here.

Another comment on speed: If you want to get an idea of how fast our dashboard loads, check our demo. It’s nice and fast. And we’ve still got a ton of room for optimization (coming in soon). Our support system is full of comments on how fast our dashboard loads for our customers of all sizes.

Is Laravel Vapor scalable?

You never need to scale your HTTP layer. Vapor auto-scales, all the servers are kept up to date… it’s amazing. I check out the response times every now and then, in the Metrics section, but I’m completely uninterested in it as it’s all managed by the experts on the AWS team.

With your database, you can configure it as a Multi-AZ RDS instance, which means you can easily scale up / down, and perform a failover (which is usually pretty quick and doesn’t cause much downtime at all). In addition to that, you can also roll out Aurora for MySQL. At the time of writing this, we’re using an over-provisioned RDS for MySQL instance, but we’re going to be moving our entire application to DynamoDB in the next version because of Alex DeBrie and his incredible book.

With regards to key-based storage, we have Redis & DynamoDB. At present, we use a small Redis instance, and it’s been great for our needs. We have a multi A-Z, t3.micro, with 500MB of storage, and we’ve not even come close to filling it up. But with that said, I still feel uneasy about having a fixed size Redis cluster. We’re looking into using DAX on-top of DynamoDB on-demand. And yes, we use on-demand DynamoDB for some key-based storage. It’s more expensive than provisioned capacity for DynamoDB, but we are relaxed in the knowledge that we can put “infinite” data in with no limits.

For SQS, we’ve really enjoyed the scale. Vapor sets it up beautifully and you can even control concurrent executions if you desire. No complaints there.

With assets, all of them are deployed publicly to S3, and served up via Cloudfront automatically within the app. It’s done in a really smart way, which avoids unwanted caching, and we’ve been really happy with it. For us, one of the concerns was always “will Vapor cache our javascript file and not have the latest version?” but they bust the cache by using directories & dynamically injecting the Cloudfront URL. It’s a smart solution that we all take for granted now.

What do we love the most about Vapor?

It’s so funny to me that I went from being absolutely obsessed with the idea of Vapor, to putting Fathom on it, to then sharing my knowledge with the world in various articles & my course. So I’m going to attempt to summarize some of the reasons why Laravel Vapor is the best solution for us:

Everything is managed

We don’t have to update anything on the main servers. Occasionally, Redis wants us to apply updates, but we’ve had nothing on RDS outside of a certificate update, and DynamoDB is even easier. Don’t underestimate how good this feels. Not having to worry about server security / updates is a huge quality of life improvement. And we have zero DevOps costs or hires, which is bloody brilliant.

We can scale infinitely

We don’t have to over-provision our HTTP layer or queue, it will just automatically scale. A customer with 1-billion page views a month can sign up tomorrow and it’ll just work… quickly and easily. We currently run a fixed size RDS, but are planning on moving to 100% serverless with DynamoDB soon. Even with our RDS fixed that we currently have, we’re very comfortable with it at, and our costs are completely reasonable.

No VPC / Security group management

For the nerds at the back! In AWS, there’s so much going on. If you know, you know. At Fathom, we often joke that you need a degree to understand Google Analytics. Well, I feel the same way about AWS. So not having to worry about VPCs / security groups etc. is great, and it means that I bother Chris Fidao less on the weekends.

Simple deployments and CI

We use ChipperCI, but I also deploy from my local machine. Our CI is simply: tests, build frontend & vapor deploy production. It’s such a smooth process. Vapor also offers this incredible “rollback” feature, which I’ve used at least 20 times in an emergency. It’s really funny because when I first saw that feature, I remember thinking “Why is someone going to need this?”. That thought didn’t age well.

Infinitely scaling queues

One of the biggest worries I had with running an analytics product was “What if our Redis cluster filled up and we couldn’t store additional jobs?”. I know, it seems so silly, but it could theoretically happen. In our Heroku days, I would check our Redis instance regularly to make sure things were good. These days, we use SQS, and it scales infinitely, so we can throw billions of jobs at it without losing sleep. So if you’ve got 5,000,000,000 page views a day? Let’s dance.

Controlled concurrency

We have a fixed size RDS instance at the moment. If we had a smaller sized one, we would want to make sure that it’s not overloaded, as we don’t want it to crash. The beauty of Vapor is that you can set your maximum job-concurrency. So if we set our job concurrency to 15, and we had 100,000 jobs come in at once, you would only see around 15 jobs being executed concurrently, allowing the database to process them without crashing. Having a backlog is much better than having your database crash. And if you can’t stand the backlog, use something that isn’t RDS or scale your RDS up.

We only pay for what we use

For Lambda, we pay per-use, which is perfect for us. So we don’t need to provision for tons of requests, and keep our scaled up infrastructure running 24/7 at high cost. Instead, our infrastructure scales up and down based on demand.

What are the limitations of using serverless technology?

You should always be aware of limitations when choosing a technology. Limitations are not synonymous with “bad”, they are just pieces of information you should be aware of, so you can decide if you can go without them. Here are the limitations I’m aware of:

The filesystem isn’t shared between requests

If you’re wanting to do something like prepare a big CSV file, appending to the end of it between requests, then you’re not going to be able to do this. The reason being because Lambda invocations do not guarantee that you’ll receive the same “container” as the previous one. However, with this said, AWS recently introduced a shared file system for Lambda and the Vapor team have documented how to use it in your Vapor project.

The maximum execution time is 15 minutes

If you’re running a HTTP request or a background job, your request must execute in 15 minutes or less. This is because Lambda only allows that. An example of where this is inconvenient is when you are putting together a data export for a user. You could run it all in a single job, which may run for 30+ minutes due to the amount of data, but Lambda stops this. But limitations foster creativity. For Fathom, we actually needed data export functionality, and we have some huge customers with a lot of data. So I just chose to break each data type (Site Stats, Page Stats, Referrer Stats etc.) up into an individual job belonging to an “Export”. This way, we had 15 minutes per data type, and didn’t need a big, long-running job. For us, I actually scheduled the data exports to run in a maxed out Lambda environment, for maximum CPU & RAM, without affecting the spec of the main servers, saving us a ton of money.

There are no queue priorities and the queue isn’t FIFO

Vapor is built for scale and doesn’t use FIFO (First in, first out). A lot of us are used to horizon workers, where we can define the order of our queues by priority. With Vapor, it uses SQS, and there’s no default way for us to have an “emails” queue and a “default” queue inside the same queue. All jobs are bundled together. This can be a limitation for some people, but there are workarounds. For us, it makes no difference, because SQS can spin up thousands of concurrent workers to quickly process these jobs. Remember, Vapor is built for high scale, so this is perfect. Having said that, you should be mindful, because you do not want to overload your RDS instance with 1,000 concurrent connections.

Provisioned concurrency “blockers”

When you use provisioned concurrency, it can take time to provision. So if you deploy and notice an error immediately, you cannot quickly deploy a hotfix. You have to wait until the provisioned concurrency is done provisioning. I’m sure there are other ways to workaround it but that’s just something to be aware of. We use provisioned concurrency because if you know you’re going to hit the concurrency 24/7, you will save money.

And that’s all I’ve really got in terms of limitations. There aren't many, and workarounds are easy.

How is Laravel Vapor support handled?

One of the most important pieces for such a critical platform is support. How fast are the replies and who is handling support? Personally, I’ve been incredibly pleased with the support. The support team consists of Taylor Otwell (creator of Laravel - you may have heard of him) and Mohamed Said (Expert diver & Laravel team member). In addition to them being on the support team, these are the main people who build Vapor. I’m sure we’ll see James Brooks make his way into Vapor support one day, but I know that he’s super busy working on Laravel Forge & Nova. And they’ve just added Nuno Maduro to the team, which is very exciting.

I’ve sent in close to 100 tickets and I’m waiting for Taylor to tell me “You’re fired” as a customer. I’ve asked questions about the basics, along with some of my crazy ideas. I always operated on the premise that they would appreciate my ideas / feedback / questions because it would help them shape the product and docs. But what’s my feedback after 100,000 tickets have been sent in? I’m pretty damn pleased. Replies are usually within 24 hours, and I’m always left satisfied (that’s what she said).

I have noticed that more and more people are persuading their clients/bosses/CTOs to move to Vapor, and I can see a world existing where a demand for enterprise level support arises. I’m very curious to see what happens here, as I’ve spoken to a few people and they’ve made it clear that they would pay $200 / month for premium support. Arguably they could introduce tiers. You sell 100 of those support packages, and you’re at $240,000 / year, which could pay for the salary of a dedicated support worker. It’s a pretty interesting idea I think but it would obviously add more complexity to the business. One other idea I had is to clone Mohamed.

What’s the Vapor community like?

Vapor has only been out for about close to a year. But there is already a passionate community behind it. I don’t mean to boast but my Serverless Laravel Slack group has some of the smartest Vapor users in the world. We talk about concepts, feedback, ideas and help each other regularly. And if you head over to Laracasts or Laravel.io, you’ll get help there too.

Wrapping up

Overall, I have only good things to say. The team is improving on things and working super hard. We still use Vapor for Fathom Analytics, and we regularly reflect on how great it is. I did a few consulting gigs for companies using Laravel Vapor, and everyone has been very satisfied with the product.

The target audience for Vapor is so mixed, because it can range from an early company/freelancer, who just hates servers, up to companies that have huge budgets and don’t want a DevOps team or server issues. It’s very appealing from freelancers all the way up to big enterprises.

As I said above, I’ve been very happy with support, but the main complaint I’ve heard from others is that they wish they could pay for “premium” support for faster response times. You can’t offer 24/7 support on a $39 / month service, it’s just not financially feasible. I can’t even imagine us giving 24/7 support for Fathom, and we have far less complex questions sent our way. So the support area is very interesting to me because it makes me wonder how big the demand for enterprise support is. What percentage of Vapor users would pay $200 / month for premium support?

Encore

As always, I’m always happy to answer any questions you have about Vapor. If you’re a Fathom customer, you get to the front of the line with your questions. Only joking. But seriously, if you need privacy-focused website analytics, we’re a great Google Analytics alternative, and you should check us out. Thousands of sites are using our software, we’re big in the Laravel community and I’m positive that I saw “Sign up for Fathom Analytics” on Taylor Otwell’s todo list.

Looking for some Laravel Tips? Check out Jack's Laravel Tips section here.

You might also enjoy reading:

• What tech stack does Fathom Analytics use?
• We rebuilt Fathom Analytics from the ground up and moved to Laravel Vapor
• Building the world’s fastest website analytics