Note: I am not currently affiliated, nor have I ever been at any time, with Meteor Development Group, Apollo, or Tiny. These are my own personal thoughts.
If you only got into web development in the past couple years, you might quite possibly have never heard about Meteor.
About the Author
But when it first came out in 2012 (before React or Vue even existed!) it was the hottest thing around for a while. Not only that, to this day Meteor still does many incredible things that the rest of the ecosystem has yet to catch up with.
Before we move on: this essay is a reflection of my own personal experiences with Meteor and its community, and is not meant to be an exhaustive or impartial history of the project.
Take yourself back to 2012. It’s the aftermath of the financial crisis, but Barack Obama is president of the U.S. and assuring the world that there’s hope. “Call Me Maybe” is topping the charts. And tech founders are to be admired (or at the most made fun of), not yet feared for their over-sized power.
Deep in Silicon Valley, a team of brilliant engineers currently enrolled in YCombinator has a realization: building web apps is too damn hard!
Out of this realization is born Meteor: a unified architecture on top of Node.js that bundles client, server, and database all in one neat little package that you can run (or even deploy for free!) with a single command.
What’s more, it was all reactive and in real-time! Can you imagine never having to write a
fetch() or worrying about updating your client data cache? And all that using the same familiar MongoDB API syntax both on the server and client!
Yep, it turns out Meteor was already tackling a huge chunk of the problems of modern web development in 2012.
A Lone Pixel-Pusher
Way back in the early ’10s I was earning a living tweaking drop shadows in Photoshop. As strange as it seems today, this was the tool of choice when it came to designing websites. This being the heyday of iPhone-inspired skeuomorphism, your paycheck depended on your ability to simulate a realistic leather texture for that call-to-action button.
Like many other UI designers though, I felt disillusioned that my beautiful creations didn’t always survive the rocky transition from Photoshop to browser in one piece. This, along with a generous dose of Silicon Valley kool-aid (I had just been to the Valley for the first time in the summer of 2011, and it made a huge impression on me), pushed me to try and strike it rich on my own rather than keep playing second fiddle to unappreciative developers.
I wasn’t starting from zero either: long before pushing pixels I had obtained my computer science diploma, even if I wasn’t quite top of my class. For example, I was once suspected of plagiarism because I was incapable of explaining what my own code did (my defense: if I was going to copy somebody else’s code, I would’ve copied something that works!). But web development had to be easier than C and Java, right?
I went through Michael Hartl’s Rails Tutorial but got overwhelmed with too many new concepts at once: routes, models, controllers, views, auth… All this combined with having to learn a new language meant Rails never quite clicked for me.
DoSomethingWithTheDOM() parts massaged into a sensible API.
The first project I built with jQuery was a little tool called Patternify, which still exists to this day! I had a ton of fun, but at some point I started being frustrated once more: playing around in the browser was fine, but to achieve anything big you needed to involve a server and a database at some point.
Meteor made a big splash right from the get-go, with one of the biggest Hacker News launches of all time. As one commenter put it:
My first impression of this: wow. If Meteor is all it appears to be, this is nothing short of revolutionary.
And revolutionary it was. Once my friend Sean Grove pointed me to it (he was in the same YCombinator batch as the Meteor guys), Meteor immediately appealed to me.
At the time, I was trying to build a “Hacker News for designers”, something that for whatever reason nobody has ever quite succeeded in doing. With its real-time capabilities, Meteor seemed like the perfect for this kind of web app. I got to work, deciding to make the project open-source in the hopes of attracting
like-minded contributors, and Telescope
Throughout this whole process I got to know the other members of the budding Meteor community, one of which was Tom Coleman, creator of Meteor’s first package manager (Meteor launched without any kind of first-party package manager and only supported regular npm packages much later, which seems hard to imagine today).
I was fresh off writing a somewhat successful design e-book so I approached Tom with a proposal: we’d combine his deep knowledge of Meteor and my design and marketing skills to write a Meteor tutorial book and establish ourselves as the de facto way to learn the framework just as it was taking off.
And guess what: we did exactly that! Discover Meteor launched in 2013 and thanks in no small part to a big boost from the Meteor team quickly became one of the main resources to learn Meteor, just as we had hoped. The book quickly became a big part of both our lives, and was also a big financial success (you can read this Gumroad case study to learn more).
We even had a podcast and a t-shirt, and I’m not sure how many other programming books can say the same. And did I mention Discover Meteor was also translated into dozens of other languages by volunteers (we made all the translations freely available)?
Fun fact: to this day I’ve still only met Tom in person once, for the Discover Meteor launch!
Things Cool Down
At first, it seemed like everything was great in Meteor land. Tom eventually went to work for the Meteor Development Group (the company behind Meteor, also known as MDG) itself, the Meteor community was growing, and the book was doing great.
It soon became clear that the community was splitting into two camps: those who appreciated Meteor’s clear value proposition of simplifying web development by providing an all-in-one environment, and those who wanted more openness towards the npm ecosystem to avoid being left behind by the rest of the webdev community.
Here’s the catch: the contents of the book clearly targeted the first camp, but we as programmers were firmly in the second. Were we supposed to keep advocating for practices we didn’t follow ourselves? Or should we rewrite the book from scratch to match what we actually did, in the process risking killing the very simplicity that made the book appealing in the first place?
We could never decide, and Discover Meteor slowly got out of date. And once Tom left the Meteor Development Group and stopped using Meteor altogether, it was clear that our adventure had come to an end.
7 Principles, 10 Years Later
1. Data on the Wire. Don’t send HTML over the network. Send data and let the client decide how to render it.
It’s since become apparent that you often do need to send HTML over the network, and things seem to be moving back towards handling as much as possible of your HTML compilation on the server, not on the client.
It’s now become commonplace to reuse components on both the server and client but that wasn’t the case 10 years ago. And to this day Meteor still handles many aspects of server/client code sharing better than most other frameworks.
3. Database Everywhere. Use the same transparent API to access your database from the client or the server.
The ability to write MongoDB commands (with some security constraints) from your browser was a big innovation at the time, but that paradigm never got popular beyond the borders of the Meteor community.
4. Latency Compensation. On the client, use prefetching and model simulation to make it look like you have a zero-latency connection to the database.
Also referred to as Optimistic UI, Latency Compensation is the idea of immediately showing the user the result of their actions by simulating the server response. While a nice idea in theory, in practice the complexity of updating the client cache and handling error states often makes it more trouble than it’s worth.
5. Full Stack Reactivity. Make realtime the default. All layers, from database to template, should make an event-driven interface available.
While many frameworks embraced client-side reactivity (starting with, well, React), Meteor’s full-stack reactivity has never been replicated quite the same way. While invaluable for chat apps, games, and other real-time apps, it can often become a performance-hungry liability for more “traditional” web apps.
6. Embrace the Ecosystem. Meteor is open source and integrates, rather than replaces, existing open source tools and frameworks.
If anything, Meteor didn’t go far enough in this direction, since it still had its own build tool, package manager, etc. Today using the ecosystem is just a no-brainer for any new project.
7. Simplicity Equals Productivity. The best way to make something seem simple is to have it actually be simple. Accomplish this through clean, classically beautiful APIs.
Meteor was, and still is, a pioneer in terms of simplicity and ease of use.
Tom wasn’t the only one who wanted to explore new skies. MDG itself had ventured into the GraphQL space, and soon pivoted to become Apollo, the company responsible for Apollo Client and Apollo Server among many other key building blocks of the GraphQL ecosystem.
So what went wrong? I’m not privy to any inside information so this is just speculation on my part, but I think the team realized that the amount of work required to make Meteor work on a large enough scale to repay their investors was just too much. As it existed, Meteor could work as a niche tool for a specific kind of project, but it could never establish itself as a dominant force in enterprise software, which is where all the real money is.
Achieving this would not only have required years of additional effort, but also throwing away most of the past five years of work, alienating their current community in the process. So just like there was never a newer, better Discover Meteor 2.0, MDG also never launched a Meteor 2.0 to address its current flaws and instead wisely chose to focus on Apollo and move on.
From Telescope to Vulcan
While others had moved on, I on the other hand wasn’t out of Meteor land by any means.
But what’s interesting is that although I had intended for Telescope to be a Hacker News clone, people had started using it for all kinds of different community-based apps, tweaking the templates to match their use case. This was such a cool direction that I decided to pivot the project to become a full-fledged general purpose web framework, and thus Vulcan.js was born.
Looking back, building Vulcan on top of Meteor instead of starting from scratch was not a good idea to say the least. I was hamstrung by all the same issues that Meteor was plagued with, without any of Meteor’s advantages since I had eschewed the traditional Meteor building blocks in favor of React & co.
Then again, there was no way I could’ve built something like Vulcan from scratch without standing on Meteor’s shoulders, so maybe it had to be this way? And despite its relative obscurity, Vulcan has been used by others to build startups, was used at a time by popular community LessWrong, and has been used by myself to build multiple sites, including Sidebar (a design newsletter with over 80,000 subscribers) and an AirBnB-like apartment rental platform.
In 2019, MDG sold Meteor to Tiny, a Canadian company that also owns Dribbble, Creative Market, and many other companies you probably know.
This marked a turning point for the framework: rather than try to conquer the world, Meteor would now focus on pleasing its existing community and growing at its own pace. This leaves us with a paradox: technically speaking, Meteor is the best and most stable it’s ever been; yet there is little interest in learning it from people who aren’t already using it.
And maybe that’s fine. After all, as the creator of a very-niche-but-beloved framework myself, I’m in no position to be casting any stones.
Still… I can’t help but think about what could have been. What if Meteor had taken over the world like we all hoped it would? What if the problems that still plague web development to this day (state management, reactivity, data fetching) had been dealt with once and for all a decade ago?
The Meteor Legacy
At this point you might be forgiven for thinking that Meteor burned too bright and fizzled out in the night sky without leaving a trace. But you’d be mistaken. Meteor did make a lasting impact, in more ways than one.
One of the new front-end frameworks that quickly outgrew Meteor’s own popularity was Vue, created by Evan You. And where did Evan work before creating Vue? You guessed it, MDG.
Or, you might’ve heard of a tool named Storybook. It was created by Arunoda Susiripala, who was by far the most active open-source Meteor contributor (in fact a running joke was to talk about “the Arunodas”, plural, because his output was too much to be the work of a single individual).
Today, Storybook is maintained by a company called Chromatic, which was co-founded by a certain Tom Coleman. Remember him?
And like I mentioned previously, the original Meteor team itself went on to create Apollo.
A New Direction for Web Apps?
As I reflect on Meteor, an old debate has come back from the grave to once more rock the web development web: SPAs vs MPAs.
While I’ve always been bothered by the downsides of SPAs, I always thought the gap would be bridged sooner or later, and that performance concerns would eventually vanish thanks to things like code splitting, tree shaking, or SSR. But ten years later, many of these issues remain. Many SPA bundles are still bloated with too many dependencies, hydration is still slow, and content is still duplicated in memory on the client even if it already lives in the DOM.
Yet something might be changing: for whatever reason, it feels like people are finally starting to take note and ask why things have to be this way. Remix has taken the React community by storm by offering a back-to-basic, performance-first approach, while Next.js is embracing React Server Components to solve some of the same issues.
Even tools that are more on the static side of things are evolving, with a new crop of options like Astro, Capri, or Slinkity letting you control which components to hydrate or not.
To say nothing of up-and-coming front-end frameworks like Marko, Qwik, and Solid, which are all about giving you better control over rendering and being more efficient. While over in Deno land, Fresh also embraces so-called “island architecture”.
A lot has changed in 10 years: between political upheavals, a devastating pandemic, and of course worsening climate change, the naive optimism of the early 2010s seems almost quaint. And we’re also more conscious of the fact that access to fast internet connections and modern devices is not always equally distributed throughout the population.
In this new context, Meteor’s all-real-time-all-the-time approach can seem a bit wasteful and excessive.
After all it’s been 10 years, but there’s yet to be another web framework that burns quite as bright as Meteor did at its zenith.