2021 Year in review

This year in review is a no-nonsense write-up of some great things that happened, a bit of drama, and some things we did poorly. The purpose of a year in review is not to make sweet love with your achievements; it's to reflect and acknowledge your wins and losses truthfully.

The journey begins

On the 7th of January, we decided it was time to get serious about getting away from MySQL for our analytics data. We had been running into headaches, experimenting with many different database solutions, and our software was missing a crucial feature for an analytics platform (the ability to filter data by pathname, referrer, etc.).

After a few previous false starts, we spent the first quarter of the year working on a massive data upgrade. The result was beautifully simple, but the build-up required lots of exploration, testing and refactoring. But of course, we're a small team, so this had to happen alongside other things we were working on, so it took time. Amongst the work, I was able to tweet a tease of multi-user support, which still isn't live (more on that later).

The outcome was that we moved from an Amazon Web Services RDS instance, which we were overpaying for, to a hyper scalable, fast, SingleStore cloud database instance. We now had the world's fastest website analytics.

We also found out that GitHub was using Fathom for their new project in the first quarter of the year. This one took us by surprise, and we felt so incredibly humbled. Thanks, GitHub, that meant a lot to us.

Take me to your Version 3

Our customers' analytics were loading faster, but they weren't aware of the groundwork we had just laid. Behind the scenes, we were collecting analytics data that would soon be filterable, allowing our customers to perform much deeper dives into their data. This new functionality would come with Version 3, and, yikes, we still had a lot of work to do before that was ready.

In a previous blog post, I advocated for CTOs to buy their developers video games consoles after an exhausting data migration. But I did not follow my own advice, and we jumped back into work on Version 3 right away. We felt very behind with it, and we needed to ship something soon.

We wrote thousands of lines of code over Q2, implementing a gorgeous new design, content filtering, a graph that you could interact with, UTM parameters, an "all sites" overview and a Refs box. In addition to those pieces, we had introduced various new features all over.

While that was great, we also had to punt on some V3 features because we had so much built-up code that needed to be shipped. We were in an awkward position where we had all of this good stuff to ship, and we didn't want to wait for these other medium/large-sized projects to be complete before shipping.

The launch of Fathom V3 went beautifully. Customers were happy, and we started landing lots of new customers following the updates. And as we said before, we will never do a major version update again; we will only do a feature-by-feature approach, meaning we release things as they're ready. There will never be a Fathom V4. Doing a major version is just too stressful, and we end up depriving our customers of features that are ready to use.

Keep my visions to myself

One of the biggest things I've struggled with is when my hype goes bad. For the uninitiated, Paul (Fathom co-founder) has labelled me as Fathom's hype man. This is because I'm always tweeting about what we're working on, doing extensive write-ups about how we do things, and just generally tweeting in excitement most weeks.

The downside of this is that I often tweet about features we're working on, then something pulls us away, and folks are left thinking, "Hey, where's my feature?". I always tweet with good intentions, but I've learned that I should only tease a feature when it's 90% complete. Otherwise, it comes back and bites us.

A whole new… website

Paul worked incredibly hard building a brand new website. We used Jigsaw by Tighten, and it was effortless to get it set up. The Tighten team was also really kind with their assistance over Twitter (thanks, folks, you're seriously the best).

The relaunch was unbelievably smooth, as we were already using a CDN, so we just had to swap out the origin. Behind the CDN, we're running a Laravel Forge server with NGINX. Deployment takes seconds, the site is entirely static, and we're delighted. There are many very powerful CMS choices available, but all we wanted to do was add markdown files and keep things minimal.

To save you asking me on Twitter, I'll answer the questions you're thinking. Why didn't we use Laravel Vapor for our marketing website? Because there's no point. We don't need hyper scaling servers because everything is cached on our CDN, which is "serverless." Our marketing site has no dynamic content. Even the referral landing page uses a static template and Alpine.js to bind URL parameters to buttons.

We deleted our database

In July 2021, we officially said goodbye to MySQL. Before you ask, yes, we moved our analytics data to SingleStore back in March, but we still had other data left in MySQL.

So we moved everything away from MySQL, and in August we deleted our MySQL database. The beauty of this was that SingleStore could suddenly offer us rapid retrieval of data such as users, sites, goals, etc., because we were able to store that data in memory.

After completely ditching MySQL, we were jiving with databases, so we decided to re-structure over one billion database rows (with zero downtime) and improve query performance by 30%. And of course, I wrote about this. We may move back to sharding by Site ID ahead of some incredible changes we've been planning, but we'll see.

The API has finally landed

Our customers and potential customers have asked us for an API since we first opened our doors, but I always put it off. And that was because I wasn't confident that our database could handle high throughput. Well, now that we'd moved to a solid database solution, we were ready.

We worked hard on the API, with Paul designing custom documentation and me chipping away at the code. The end package was beautiful, and we were ready to launch early access. We also ran into a software product called Torchlight, which allowed us to offer beautiful syntax and became the new default for our marketing site. The beauty about this package is that it doesn't need javascript to run; it generates the output when we generate our static site. This leads to blazing-fast performance.

The API was well received, with folks building packages, fun tools, iOS apps, and so many other things. We still haven't launched this "officially" because we have an update landing in 2022, which is on a whole new level.

Dyn-no-mo-DB

After spending many hours learning, talking, sharing, and coding with DynamoDB, September was when we completely removed it from our tech stack. I couldn't believe it had come to this, but we did not need it anymore. We were already paying a few thousand dollars for a great database cluster, and we had a tremendous amount of memory (RAM) available on the cluster. We figured we could use SingleStore for everything, and that's what we did. This move saved us a huge chunk of cash.

In sleep he sang to me, in dreams he came

In October 2021, we open-sourced Phantom Analyzer. We also deleted the old site, and many of you are likely wondering why. Well, the motivation behind it was that it wasn't being used by a ton of folks after the initial launch, and I was motivated to reduce our attack surface area where possible. Having an additional application, which required maintenance, was running on our AWS, and utilizing our Lambda burst concurrency, just felt wrong. It was well-received and, at the time of writing, it nearly has 100 stars. We'll take it.

Multi-tenant analytics

Throughout the final two quarters of 2021, we were on fire. We shipped multi-tenant analytics, and it took off. This is an amusing example of how you can build something, imagine how people might use it, and then see how folks actually used it.

We had built this feature for people who have multiple domains for a single application. So, for example, imagine you have jacks-shop.usefathom.com and then pauls-shop.usefathom.com, and then 1,000 other shops. You don't want to create a site for each of them, as that would be a massive headache. So the idea was that multi-tenant analytics would allow you to have a single Fathom site, single tracking code snippet. Then we'd support multiple domains within the Fathom dashboard.

We didn't build this functionality off of guesswork. Paul had spoken to multiple customers about what they were looking to do with Fathom, and we put this spec together based on their needs.

The launch went well, and folks started using it for those reasons, but then one of our customers (Jon Henshaw) found a series of bugs. He used a single Fathom Fathom site for all of his websites, which weren't part of a multi-tenant platform.

So we fixed the bugs, and many people started doing the same thing as Jon. The way these people saw it, they wanted an overview of their portfolio of sites, within the main Fathom dashboard, with the ability to quickly filter through content. Amazing. This isn't the first time people have used things differently than we'd expect, and it won't be the last.

Industry shattering news

In November 2021, after over a year of on-off work and extensive collaboration with our privacy officer, an EU DevOps genius, and lawyers from the EU & Canada, we were ready to launch EU Isolation.

Long story short, there was a huge ruling (Schrems II), where Max Schrems sued Facebook, and it invalidated the EU-US privacy shield. So this meant that folks could no longer casually pass EU residents' data (IP & User-Agent in the case of privacy-first analytics) to US-controlled servers.

Many folks wouldn't attempt this, but we have a lot of customers in the EU who rely on us for GDPR compliance, so we set up three availability zones in the EU, all owned by a German company called Hetzner. We then introduced an EU-controlled CDN on top of our analytics endpoints, and this meant globally faster analytics, along with actual GDPR compliance for our EU customers. The beauty of this was that we then utilized AWS' DNS hosting to provide our CDN with a low-TTL, health-checked endpoint, which meant we could easily remove an unhealthy server from the DNS record if it fell offline.

There are now a handful of analytics companies claiming to be "privacy-first" and "GDPR compliant" who are, quite frankly, lying to their customers and putting them at risk.

We were so fortunate to work with so many amazing people on this. We invested a lot of money into getting this solution off the ground, and it paid off.

For EU companies serving an international audience, there is no better solution than Fathom. Some companies chose to go all-in on EU-owned infrastructure, but what about their American customers? Why should US website owners have to tolerate slower page load times? Why should US developers accept slower API load times? We don't think they should.

Folks in the United States should not be neglected and punished with slow load times. Many American website owners take privacy very seriously, even when they're not compelled by law to do so, and we refuse to give them (and their website visitors) a slow experience.

Anyway, that's enough hype. We're very proud and are committed to innovation.

It's alright

We took time to reflect, calm down, and pay down technical debt throughout December. It's been an incredible ride this year, and we're appreciative of everyone who has ridden with us.

We've recently brought on extra professionals to work with us, and we're currently looking to expand our development team. We are not happy with the speed that we've been moving, and having a single developer has meant that things have moved slower than we'd like, especially when I'm busy with other things.

Interestingly, we haven't hired anybody to help us with support. We don't attract high maintenance, budget customers because we're not a budget service, and our support workload has been manageable throughout the year. This means that we can invest in other areas that will help improve Fathom as a product.

I can't say what else we're doing in 2022, but I can say that you're going to be happy that you're using Fathom.

Looking for some Laravel Tips? Check out Jack's Laravel Tips section here.

You might also enjoy reading:

• 2020 Year in Review
• The Journey to EU Isolation
• Why we ditched DynamoDB