Laravel tips from Fathom

SingleStore is now faster on Apple Silicon

2023-01-13T00:00:00+00:00

Jan 13, 2023 PT -

The MacBook Silicon processors are incredible, there's no debate there, and various tools have offered fantastic emulation for arm64 and other processor architectures. The problem is that it's always been slow in some areas. Phase one of us running SingleStore on my MacBook M1 was made possible by work from Charlie Joseph, who managed to get SingleStore running on Docker via emulation. Phase two was made possible by Carl Sverre and his team via the SingleStoreDB Dev Image. And at this point, things were pretty good, but they still needed improvement, as emulation wasn't as fast as we know it could be.

Fast forward to 12th January 2023, and Docker releases BETA support for emulation via Rosetta. We knew this was coming, and I'd been optimistic that performance would improve, and it has. But let's stop with the talking and get to the benchmarks. After all, that's why you're here.

Benchmarks

A few notes for the benchmarks you'll be seeing below:

SingleStore uses something called a query plan cache. Long story short, the first time you run a query, there's a slight lag as SingleStore's engine decides on the BEST way to run the query. The benchmarks are run after that initial query plan cache has run because, in production, this initial query plan cache run is irrelevant.
The benchmark values are an average of 5 runs.
I profiled these queries via SingleStore studio.
I'm running on SingleStore 8.0.4.
My machine is a MacBook Air M1 2020, so many of you will see WAY better performance than me (e.g. if you're on a newer MBA or MBP).
The benchmarks are done in milliseconds, and I've rounded to 2 decimal places.
The benchmarks are not testing for concurrency because this is local development.

Now let's get to testing.

Basic queries

I've performed some basic CRUD queries below and some database creation to see how these behave.

Query	Before Rosetta	After enabling Rosetta	How much faster is Rosetta?
CREATE DATABASE test;	3,083 ms	2,954.75 ms	4.16%
DROP DATABASE test;	939 ms	740.33 ms	21.15%
CREATE TABLE users (id BIGINT AUTO_INCREMENT PRIMARY KEY, name TEXT, email TEXT);	121 ms	28.75 ms	76.24%
INSERT INTO users (id, name, email) VALUES (1, 'Jack Ellis', 'jack.ellis@google-analytics.com');	5.4 ms	2.66 ms	50.74%
SELECT * FROM users WHERE id = 1;	4.8 ms	2.5 ms	47.91%
UPDATE users SET name = 'Jack Google' WHERE id = 1;	6 ms	3 ms	50%
DELETE FROM users WHERE id = 1;	5.8 ms	2.25 ms	61.21%

Aggregations

Now we get into my favourite area, aggregations. I've loaded in 3,564,346 pageviews that belong to usefathom.com (our website). In reality, I wouldn't be using this much data locally unless I was debugging a specific data issue. But I know we'll see the most variance in the benchmarks if I use more data. Clearly, we're not going to get the same performance as when SingleStore is running on intel processors but let's see how things go.

Query	Before Rosetta	After enabling Rosetta	How much faster is Rosetta?
SELECT SUM(pageviews) from pageviews;	67.6 ms	7.4 ms	89%
SELECT pathname, SUM(pageviews) as x from pageviews GROUP BY pathname ORDER BY x DESC LIMIT 100;	125.2 ms	16.2 ms	87%
SELECT pathname, SUM(pageviews) as x FROM pageviews WHERE referrer_hostname = ‘https://laravel.com' GROUP BY pathname ORDER BY x DESC LIMIT 100;	133.6 ms	12 ms	91%
UPDATE pageviews SET utm_campaign = ‘boop’;	22,000 ms	3,593 ms	83.66%
DELETE FROM pageviews WHERE referrer_hostname = 'https://laravel.com';	610.33 ms	93 ms	84.76%
DELETE FROM pageviews;	488.5 ms	44 ms	90.99%

How do I enable Rosetta on Docker?

Easy peasy, simply:

Update to latest Docker
Go into Settings -> Features in development
Tick the “Use Rosetta for x86/amd64 emulation on Apple Silicon

Additional notes

Before enabling Rosetta, the Nodes page in SingleStore studio didn’t work and always showed 0% for both CPU and memory. With Rosetta, it works perfectly.
Performing things such as INSERT INTO pageviews SELECT * FROM pageviews is so much faster when Rosetta is enabled
Member of my SingleStore course, Franco Gilio, profiled and saw huge improvements too. His Laravel Nova searches went from 664ms to 166ms. Incredible.

Not bad

Overall, I’m very impressed by speed improvements. And Docker battery jokes aside, my battery life seems to be doing better. These benchmarks were tested by be manually, and they’re obviously not relevant for production, but I thought they’d be useful to do. I hope this post was helpful. And if you’re not in our community, make sure you check out SingleStore for Laravel, to learn how to use the fastest database in the world with Laravel.

How to have multiple unique columns in SingleStore

2023-01-04T00:00:00+00:00

Jan 4, 2023 PT -

You've used MySQL for many years and have multiple unique keys set up on your tables. Then your data grows, you say Goodbye to MySQL, and you move to SingleStore. But hold on a minute; SingleStore only allows you to use one unique key per table. And if you're using an auto-incrementing ID, which most of us are in the Laravel world, your one unique key is gone.

The simple solution

Before I get into this, I will give all credit to Carl Sverre. I told him how I was enforcing uniqueness via atomic locks, and he knew of a better way. Let's get to it.

Step 1: Modify your users table

Our users table is going to be the create_users_table migration with one modification. We have removed the unique() method on the email field, so the table won't enforce it.


Schema::create('users', function (Blueprint $table) {
    $table->id();
    $table->string('name');
    $table->string('email');
    $table->timestamp('email_verified_at')->nullable();
    $table->string('password');
    $table->rememberToken();
    $table->timestamps();
});

Step 2: Create your email_users table

So this is where the beauty comes in. We can't enforce email uniqueness on the users table, but we can enforce it on another table. Stay with me, folks.


Schema::create('users_emails', function (Blueprint $table) {
    $table->string('email');
    $table->primary('email');
});

Step 3. Creating a user

We now have two tables involved when creating a user. To keep this simple, let's imagine we have a static function called registerUser somewhere in our application. This is what we might do:


function registerUser($email, $password) {
    try {
        DB::transaction(function() use ($email, $password) {

            // We create our user as usual
            User::forceCreate([
                'email' => $email,
                'password' => Hash::make($password)
            ]);

            // Now, here's the beautiful part
            // If the email already exists in this table
            // it will throw an exception, enforcing uniqueness
            DB::table('users_emails')
                ->insert(['email' => $email]);
        });
    } catch (\Exception $e) {
        // Handle the exception
    }
}

Step 4: Editing a user

How easy was that? And now, let me show you what we'd do when editing a user.


function changeEmail($userId, $oldEmail, $newEmail) {
    try {
        DB::transaction(function() use ($userId, $oldEmail, $newEmail) {

            // Modify the email in the users table
            User::where('id', $userId)
                ->update(['email' => $newEmail]);

            // We have to delete the email because the primary key is immutable
            DB::table('users_emails')
                ->where('email', $oldEmail)
                ->delete();

            // And then we insert the new email of course
            DB::table('users_emails')
                ->insert(['email' => $newEmail]);
        });
    } catch (\Exception $e) {
        // Handle the exception
    }
}

How epic is that?

Step 5: Deleting a user

And now deleting a user is pretty similar to editing a user.


function deleteUser(User $user) {
    try {
        DB::transaction(function() use ($user) {

            // Delete the users email
            DB::table('users_emails')
                ->where('email', $user->email)
                ->delete();

            // Goodbye my lover, goodbye my friend
            $user->delete();
        });
    } catch (\Exception $e) {
        // Handle the exception. You wouldn't get a "unique" error here though.
    }
}

How cool is that?

My previous recommendation was to use atomic locks to enforce uniqueness, which is acceptable, but this is better. With our transaction approach, there are fewer queries, and things are faster. And uniqueness is enforced by the database.

You have duplicate data, but who cares? Databases are about trade-offs. Technology is about trade-offs. Life is about trade-offs. Throw in 10,000,000 email addresses, but you have the database enforcing uniqueness thanks to your transactions? I'll take that any day.

Should I use Laravel Vapor?

2020-03-25T00:00:00+00:00

Mar 25, 2020 PT -

A lot of people have heard of Laravel Vapor, and know that it's for Serverless Laravel deployment, but they aren't too sure what it does.

Laravel Vapor was built by Taylor Otwell and the Laravel team and is a serverless deployment platform for Laravel applications. It effectively marries our code to underlying AWS infrastructure with very little effort from us, which is great. A lot of people have avoided AWS, because there’s too much to learn, but we now have an easy route in.

The beauty of Vapor is that we can control AWS resources through its beautiful user interface. This means we always stay at the high level and, only occasionally, do we need to dive into AWS a little to play around with some configuration.

Vapor, to me, is a step up from Heroku. For those of you who haven’t used Heroku, it’s a beautiful GUI / platform that lets you provision dynos (servers), and attach add-ons, without doing any configuration yourself. But it’s expensive and they limit you to rigid tiers with certain things such as Redis / Postgres / MySQL, meaning it costs more when you need a small upgrade. Instead of allowing you to upgrade to the next AWS tier, they limit the tiers to force you into paying more. Comparatively, with Vapor, you get access to every single sizing option that AWS offers.

Laravel Vapor is the future of Laravel applications. With Vapor, they charge a fixed price of $39 / month and you get AWS prices, with zero markups, for everything it provisions. No markup, how wonderful. It ends up being MUCH cheaper for a lot of use cases.

But that’s the technical side of things, what is Vapor actually for? Why would I use Vapor instead of DigitalOcean?

Why should I use Vapor

You don’t want to spend your time managing servers

You don’t want the updates, CPU / RAM monitoring, server reboots, corrupted hardware, etc. You value your health and enjoy your sleep. Being on-call 24/7 is stressful and exhausting. 2 AM downtime notifications and whatever else comes with managing servers. Sure, it doesn’t happen all the time, but you (or a member of your team) need to always be available to put out fires. Using Vapor means you can utilize a series of AWS services that are all fully managed for you, meaning the AWS team gets the 2 AM calls. Your time is more valuable spent building your application.

You don’t want to hire outside help to manage your servers

When we used a multi-VPS set-up for Fathom Analytics, we were quoted $1,000 / month for DevOps support. We’d have someone who gets notified if our servers have issues and they’ll respond as soon as possible. That is expensive. We only had around 3-4 servers.

You have unknown or varying server load

Vapor is perfect for unknown or varying workloads, since you could process 400 million requests at 6 random times throughout the month, and then 1 million every other day, and you wouldn’t have to over-provision. You only pay for the requests you use.

You run a critical service and want as little downtime as possible

Because Vapor uses AWS Lambda, it means thousands of different servers are being used to process your web requests. This means that you won’t run into these server-related downtime issues on AWS Lambda. The same applies to the queue service (SQS) and the database. Sure, the database service can have downtime, but you can mitigate that very easily. We’ll be covering that in the course.

You like to tweak settings & experiment

In Vapor, scaling your cache or database is as simple as clicking Scale and then entering your desired size. This means you can tweak to your desire and monitor the outcome.

You want a platform that was made specifically to host Laravel applications

Laravel Vapor was built by Taylor Otwell and the Laravel team. Why is this important? Because they are Laravel experts, and the platform was built specifically for the framework your application uses

When you shouldn’t use Vapor

Now before we close up, I want to balance out this argument and keep it objective. If a hosting bill over $20 / month concerns you, or the $39 / month price tag for Vapor seems expensive, I’m going to go ahead and say Vapor is probably not for you.

Vapor is for people who hear "$39 for zero server maintenance" and reply immediately with "Sign me up now". When I first pitched Vapor to a client of mine and explained that you can set up applications on-demand, have zero server maintenance and high availability, they wanted to move right away. Businesses understand the human cost of server maintenance, and it’s thousands of dollars. If they can pay such a small amount to solve this problem, they would fly someone to Taylor Otwell’s house and hand him $3,900 for 10 years of Vapor, and it would be cheaper to do all of that than to pay a developer for a week of server set-up and then the infinite server maintenance that comes with that.

And that's my summary on Vapor and whether you should use it. If you're curious, I hear that someone has a course on how to use Laravel Vapor.

What is Serverless?

2020-03-22T00:00:00+00:00

Mar 22, 2020 PT -

Before we get ourselves too confused with definitions, let us address the elephant in the room. Yes, Serverless infrastructure uses servers.

Serverless just means that you don’t have to worry about software updates, security patches, monitoring, provisioning capacity, load balancing, uptime, and all other maintenance.

So how does serverless differ from what we’ve been doing for the majority of our careers? The easiest way to explain things is for me to compare a “traditional” set-up against a serverless set-up for a basic web application.

Let’s imagine we’ve got a basic Laravel application, and let’s imagine that it’s a chatroom. The features we have are register, login, logout, post message and perhaps some email notifications when you are tagged.

Traditional

With your typical, old school web application, you’d set-up LAMP or LEMP on a server. That server may be a dedicated server or a VPS. If you’re using shared hosting, this will all be set-up for you. But that’s the minority of cases when it comes to Laravel applications.

For this example, let’s imagine you have some sort of VPS from DigitalOcean or a dedicated server.

So as I said, the traditional set-up would be having it all on one server.

Linux
Apache
MySQL
PHP

So that’s how the traditional set-up looks. It’s the set-up that a lot of us came up using, and may still use.

This isn’t good though because we have all of our eggs in one basket. If that dedicated server has some software or hardware issue, we could lose data and be offline for hours. Imagine if we had to restore a 1TB backup to our database because our database was unrecoverable from an issue? It would be a headache. What about if your MySQL instance required more CPU or RAM, you might have to take everything offline to perform the upgrade. And that’s one of the reasons why this approach isn’t as common anymore.

Broken-out Traditional

With the ‘Broken out Traditional’ approach, we split our application up into pieces. Instead of having the database and the webserver together, we now have 2 servers. 1 is responsible for MySQL and the other is responsible for the webserver. This is better because it means we can control each server separately. We can switch out the MySQL server parameters easily in the event of maintenance/server upgrades without taking the web server offline.

So this is certainly a step up from traditional. There are obviously many other combinations where you can set-up your infrastructure in a custom way to add redundancy to your application, stepping up each time, but I’m not going to dive into them because we’ll be here for weeks. The downside of this is that you increase complexity/management as you scale-out. As the late Biggie Smalls once said, Mo Servers Mo Problems.

Serverless

We will jump right past the infinite other configurations and onto a serverless set-up (done with Laravel Vapor) because that’s what we’re going to be talking about in the course.

With serverless, we still use servers but you don’t manage them. Instead, we utilize services like:

Lambda (HTTP Layer / Commands / Queue workers)
RDS (Highly available database)
SQS (Queue)
SES (Email)
ElastiCache / DynamoDB (Cache)
Cloudfront (Content Delivery Network)
and more.

These are the AWS services that Vapor utilizes. Because they are services, we can provision them with certain settings, without having to worry about underlying server tweaks, updates, and management. Instead, a team of experts keeps the services online for us. There are thousands of servers running these services but, as far as we’re concerned, we’re serverless because we don’t care about the servers.

So that’s the difference. By choosing serverless deployment, we choose to focus on our applications and not on infrastructure. We choose to pay $39 each month for Laravel Vapor instead of $1,000 (or more) each month for someone to manage our servers.

Some people like DevOps. If that’s you, serverless might not be for you. But for those of us who want to spend our time writing code and making our applications better, serverless is the best thing since sliced bread.

Sure, we still have to manage services occasionally, but that’s rare and easy. We stay at a high level when we use these services, and never have to dive into server problems.

We choose sleep instead of 2 AM wake-up calls from uptime monitoring solutions telling us our website is down. We choose a managed database service where database storage is decoupled from the main computation. And all the other serverless goodies.

Serverless Laravel isn’t a trend, it’s the logical next step. Less time on infrastructure and more time building our applications.

Improve Laravel Vapor response time with prewarming

2020-03-22T00:00:00+00:00

Mar 22, 2020 PT -

For those of you who aren't aware, prewarming is used in Vapor to reduce cold starts. When you deploy your application, there won't be any Lambda containers that are "warm" and ready to respond to requests by default. This leads to guaranteed cold starts, which can add up to 2 seconds to response times, which is not ideal. Prewarming means that Vapor will send requests to a specified amount of containers on deployment to warm them up and will continue to send requests every 5 minutes to keep them warm. So prewarming is a great thing and you should always do it.

For those who already prewarm, you may be thinking "Am I prewarming enough?" or "Am I prewarming too much?". In addition to those concerns, you also might have heard the argument of "Why don't you just use a standard server since you won't get any cold starts?" etc. Well, I have news for you, and it's good news.

Let's consider a $5 / month DigitalOcean droplet. If it always online with 1GB of memory ready to go. Let's say that each Laravel request is 25MB. That means, in theory, if all memory is assigned to PHP requests (heh), the Droplet can handle 40 concurrent requests.

In Vapor, if you want to match that concurrency availability you would need to have 40 lambda containers warmed up. And guess what, it's not expensive to do.

Vapor pings your containers every 5 minutes when you use prewarming. There are ~730 hours in a month. 730 x 60 = 43,800 minutes. 43,800 minutes / 5 = 8,760. So there are 8,760 requests per month sent by Vapor per container for prewarming.

Let's use a 512MB Lambda container (far more than the 25MB each DigitalOcean request has) and consider warming 40 containers.

Post-Deployment Warming

Let's be pessimistic and assume that the post-deployment warm is 2 seconds and that we are responsible for the cost of the start-up. AWS doesn't charge us for this but I'm not sure who incurs the cost: Vapor or us. When you don't know the answer to something like this, and can't find it documented, you should always assume the worst case pricing scenario: So let's pretend we will be billed 2 seconds per cold start:

512MB for 2 seconds = $0.00001666
Request = $0.0000002
Data Transfer costs are $0.01 per GB, so they're negligible
Total cost per deployment for 1 container to be warmed: $0.00001686
Total cost per deployment for 40 containers to be warmed: $0.0006744

So each deployment will cost us $0.0006744. Let's say we deploy 500 times in a month (Freek Van der Herten productivity). That'd cost us 33 cents in a month.

Warming every 5 minutes

These warms are keeping your already-warm containers warm, so you won't have that 2 second cold start.

512MB for 200ms (estimate) = $0.000001666
Request = $0.0000002
Data Transfer costs of $0.01 per GB are negligible
Total cost per 5 minute "ping" = $0.000001866
Total cost per 5 minute "ping" for 40 containers = $0.00007464
Total cost for 40 containers to be warmed every 5 minutes for a month = $0.6538464

Isn't that incredible? So what does that mean in real talk? Let's assume an API with 200ms response times.

$0.6538464 / month = ready for 200 req/sec concurrency
$1.3076928 / month = ready for 400 req/sec concurrency
$2.6153856 / month = ready for 800 req/sec concurrency
$5.2307712 / month = ready for 1.6k req/sec concurrency

And sure, you will still have to pay for the actual request processing but the point is that you are fully warmed up and ready to handle incredible capacity for a ridiculously low price. Do you really think that someone who is handling 1.6k requests per second (5.7M req per hour) can't afford $5.30 to warm their containers up? That's the price of a coffee.

I'm confident that my math won't be exact but it's a solid estimate based on AWS' published pricing. I would advise running over the numbers for your own specific use case. All you need to do is use the AWS Lambda Pricing page to get the figures you need. Decide what kind of warming you want and go from there.

My main point here? Do not be afraid of setting your warming setting nice and high... because you're worth it :)

The Vapor team are actively working on improvements to their cold-start handling. It's not perfect, you'll still have cases where a few requests will encounter a delay (if they hit during the exact moment prewarming is occuring), but prewarming helps. They introduced provisioned concurrency (1 week after Lambda announced it), which is great, but it's way too expensive . The vapor team are actively working on new ideas to improve cold starts, and they move very fast.