Node and Scaling in the Small vs Scaling in the Large
Over the past few weeks, I’ve been taking whatever spare moments I can find to think about what technologies we’re going to use to build the initial release of BankSimple. Many people would probably assume that I’d immediately reach for Scala, what with having co-authored a book on the language, but that’s not how I approach engineering problems. Each and every problem has an appropriate set of applicable technologies, and it’s up to the engineer to justify their use.
(Incidentally, Scala may well be a good fit for BankSimple, in no small part due to a bunch of third-party Java code that we need to integrate with, but that’s a whole different blog post, probably for a whole different blog.)
One of the most talked-about technologies amongst the Hacker News crowd is Node, a framework for writing and running event-driven JavaScript code on the V8 virtual machine. As part of evaluating what tech to work with, I considered Node. Yesterday, I expressed some general skepticism about Node, and the framework’s author, Ryan Dahl, requested that I write up my thoughts in more detail. So here we are.
I’m certainly not here to disparage Ryan, who’s a nice guy and a superb programmer; he knows more about low-level C than most of us could ever hope to, and without so much as a neckbeard to show for it. Nor am I here to question the enthusiastic community that’s quickly grown up around Node; if you’ve found a tool you enjoy working with and are committed to growing with it, more power to you.
Rather, the purpose of this post is to question two of the Node project’s stated goals, goals which seem to me to be at cross purposes.
What’s Node For?
The About section of Node’s homepage states:
“Node’s goal is to provide an easy way to build scalable network programs.”
Then, a couple paragraphs on, we’re told:
“Because nothing blocks, less-than-expert programmers are able to develop fast systems [with Node].”
So, is the purpose of Node to provide an easy way to build scalable network programs, or is it to allow less-than-expert programmers to develop “fast systems”?
While these goals may appear related, they’re very different in practice. In order to better understand why, we need to make a distinction between what I’m calling “scaling in the small” and “scaling in the large”.
Scaling In the Small
In a system of no significant scale, basically anything works.
The power of today’s hardware is such that, for example, you can build a web application that supports thousands of users using one of the slowest available programming languages, brutally inefficient datastore access and storage patterns, zero caching, no sensible distribution of work, no attention to locality, etc. etc. Basically, you can apply every available anti-pattern and still come out the other end with a workable system, simply because the hardware can move faster than your bad decision-making.
This is a great thing, actually. It means we can prototype haphazardly using whatever technologies we hold dear, and those prototypes will often take us farther than we expected. Better still, when we hit a roadblock, getting around it is trivial. Moving forward just means spending several minutes thinking about your problem and picking an implementation technology with slightly better performance characteristics than whatever you were using before.
This is where I believe Node fits in.
If you look at who’s flocking to Node, it’s largely web developers who have been working in dynamic languages with what we could politely call limited performance characteristics. Adding Node to their architectures means that these developers have gone from having essentially no concurrency story and very constrained runtime performance to having some semi-sane concurrency story – one rigidly enforced by the Node framework – running on a virtual machine with comparatively respectable performance. They slice off a painful bit of their application that’s suited to asynchrony, rewrite it in Node, and move on.
That’s awesome. That kind of outcome definitely meets Node’s secondary stated goal of “less-than-expert programmers” being “able to develop fast systems”. However, it has very little to do with scaling in the larger, more widely-understood sense of the term.
Scaling In the Large
In a system of significant scale, there is no magic bullet.
When your system is faced with a deluge of work to do, no one technology is going to make it all better. When you’re operating at scale, pushing the needle means a complex, coordinated dance of well-applied technologies, development techniques, statistical analyses, intra-organizational communication, judicious engineering management, speedy and reliable operationalization of hardware and software, vigilant monitoring, and so forth. Scaling is hard. So hard, in fact, that the ability to scale is a deep competitive advantage of the sort that you can’t simply go out and download, copy, purchase, or steal.
Herein lies my criticism of Node’s primary stated goal: “to provide an easy way to build scalable network programs”. I fundamentally do not believe that there is an easy way to build scalable anything. What’s happening is that people are confusing easy problems for easy solutions.
If you have an easy problem that was handily solved by moving from one piece of extremely limiting technology to a bleeding-edge piece of slightly less limiting technology, consider yourself lucky, but it doesn’t mean you’re operating at scale. Twitter certainly had such an easy win when, while at a fraction of the scale the service now operates at, one of their engineers rewrote their in-house Ruby-based message queue in Scala. That was great, but it was scaling in the small. Twitter is still fighting an uphill battle to scale in the large, because doing so is about much, much more than which technology you choose.
Growing Up Node
What concerns me about Node is that it’s going to be difficult to grow with as engineers move from scaling in the small to scaling in the large. (No, I’m not making the “callbacks turn into a pile of spaghetti code” argument, although I think you hear that time and again because it’s an actual developer pain point in async systems.)
The bold thing about Node is that everything is asynchronous, right down to file I/O; here, I admire Ryan’s commitment to a consistent and clear thesis for his software. Engineers who deeply understand their system’s workload may find places where Node’s model is a good fit, and may be a good fit effectively indefinitely; we won’t know that until Node sees long-term mature deployment. Most systems I’ve worked on change, though. Workloads change. The data you’re moving around changes. What was once well-handed by an asynchronous solution is suddenly now better served by a threaded solution, or vice versa, or you’re faced with some other out-of-the-blue change entirely.
If you’re deeply invested in Node, you’re stuck with one way of approaching concurrency; one way of modeling your problems and solutions. If it doesn’t fit into an event-based model, you’re hosed. If, on the other hand, you’re working with a system that allows for a variety of concurrency approaches (the JVM, the CLR, C, C++, GHC, etc.), you have the flexibility to change your concurrency model as your system evolves.
At the moment, Node’s core premise – that events necessarily yield performance – is still in question. Researchers at UC Berkeley found that “threads can achieve all of the strengths of events, including support for high concurrency, low overhead, and a simple concurrency model”. A later study that builds on that work shows that events and a pipelining approach perform equally well, and that blocking sockets can actually increase performance. In the industrial Java world, it comes up periodically that non-blocking I/O isn’t necessarily a better fit than threads. Even one of the most-cited documents on the subject with the blatant title of Why Threads Are A Bad Idea ends up concluding that you shouldn’t abandon threads for high-end servers. There just isn’t a one-size-fits-all concurrency solution.
In fact, taking a hybrid approach to concurrency seems to be the way forward if the academy is any indication. Computer scientists at the University of Pennsylvania found that a combination of threads and events offers the best of both worlds. The Scala team at EPFL has argued that Actors unify thread- and event-based programming into one tidy, easily comprehensible abstraction. Russ Cox, formerly of Bell Labs and now on the Go programming language project at Google goes so far as to argue that threads vs events is a nonsensical question. (Note that none of this even begins to touch on the distribution aspect of scaling a system; threads are constructs for a single computer, and events are constructs for a single CPU. We’re not even talking about distributing work across machines in a straightforward way, as you can in Erlang, but that’s worth thinking about if you’re babysitting a rapidly growing system.)
The point being: seasoned engineers are using a mix of threaded, event-based, and alternative concurrency approaches like Actors and, experimentally, STM. To them, the idea that “non-blocking means it’s fast” sounds, well, a bit silly; it’s scalability urban mythology. The guys that are getting paid the big bucks to deliver scalable solutions aren’t up at night feverishly rewriting their systems in Node. They’re doing what they’ve always done: measuring, testing, benchmarking, thinking hard, keeping up with the academic literature that pertains to their problems. That’s what scaling in the large necessitates.
Conclusion
For my investment of engineering time, I’d rather build on a system that allows me the flexibility of mixing an async approach with other ways of modeling concurrency. A hybrid concurrency model may not be as straightforward and pure as Node’s approach, but it’s going to be more adaptable. While BankSimple is in its infancy, we’re going to be faced with the happy problems of scaling in the small, and Node might be a reasonable choice for us at this early stage. But when we do have to scale in the large, I’d rather have a variety of options open to me, and I’d rather not be faced with the prospect of a big rewrite under pressure.
Node is a lovely bit of code with an enthusiastic community, a whip-smart maintainer, and a bright future. As a “bridge technology” that offers immediate solutions to early scaling problems in a way that’s particularly accessible to a generation of web developers who largely come from dynamic language backgrounds, it makes sense. Node more than meets its secondary stated goal of bringing reasonable performance to developers of intermediate experience who need to solve network-oriented problems. Node is, for a certain breed of programmer, familiar and fun, and it’s undeniably easy to get started with. People in the Node community are having a good time reinventing the familiar wheels of web frameworks, package management, testing libraries, etc., and I don’t begrudge them that. Every programming community reinvents those things to their norms.
Once we’ve made clear what Node is a good fit for, though, it’s important to remember that there are no panaceas for problems of significant scale. Node and its strictly asynchronous event-driven approach should be viewed as a very early point on a continuum of technologies and methodologies that comprise scaling in the large.
Approach popular solutions cautiously. Everybody may be talking about a hot new technology, but very few people are actually working at a scale at which those technologies are going to be put through their paces. Those who are tend to be heads-down in numbers and research, working with tools and techniques that have been maturing over time. If you invest your time in a new technology, be ready to learn and grow with it, and maybe to jump ship when you find yourself constrained.
This stuff isn’t easy.
—Jul 27, 2010
Something New
Imagine, for a moment, a bank that doesn’t suck.
A bank that doesn’t gouge you with fees.
A bank that doesn’t treat you like crap.
A bank that cares about design, but gets out of your way.
A bank that puts your money to work automatically.
A bank that’s building a platform for the future of personal finance.
I’ve imagined that bank, usually while on hold with my current bank after they’ve screwed something up. I want to be a customer of that bank. More than that, I want to make it happen.
So, I am.
Starting today, I’m joining BankSimple as a co-founder, with the role of Chief Product & Technology Officer. In a nutshell, I’m going to make sure we build something that’s simple, beautiful, and works really really well.
Walking away from Twitter wasn’t an easy decision. Working there has been a life- and career-changing experience. I’ve learned all sorts of lessons, made great friends, and worked on something that millions of people now use every day.
But millions of people also need a better bank.
It’s a different sort of problem than Twitter, and it’s going to be a very different sort of company. But I feel ready to take on the challenge that BankSimple presents, and I’m joining on with two brilliant people who’ve put a ton of thought into building a better bank.
If BankSimple sounds like something you want, here’s how you can help make it happen:
- Sign up to be first to know when we launch.
- Join the team. We’re hiring interns for the summer, and we’ll be hiring programmers, designers, customer support, operations, and many other positions down the road.
Or, just talk to me about it. Tell me what you want from it. I’m at alex@banksimple.net for all things work-related, henceforth.
I’m excited. I’m energized. Today is the start of something great.
—May 17, 2010
The Moderate’s Position on iPad Openness
When Apple’s iPad was announced in January, I immediately posted a reaction. Now that the device is out, it seems worth revisiting the subject.
Do You Feel Like A Hypocrite Yet?
First thing’s first. Back in January, I wrote that the iPad wasn’t something “I want to bring into my home”. But right now, there’s an iPad in my home. I didn’t make a promise not to buy one, but I certainly implied that it wasn’t appealing to me. So what changed?
Since January, developers have been posting previews of their iPad apps. Some are designed expressly for the device. Some are existing Mac OS X or iPhone apps that have been reformatted for the iPad’s paper-like dimensions. Regardless, even a few screenshots or a blurry demo video on YouTube is enough to show you that there’s something big going on here. Look at Things or Twitterrific. Familiar, but different, compelling; superior. That’s what got me into the Apple Store this past Saturday.
By the end of the day, I was convinced. Human-computer interaction has found a sweet spot on the iPad. It’s all the power of desktop computing, plus the valuable constraints of mobile devices, minus the limitations of both. It just makes sense. Use one for a couple hours and your desktop or laptop will seem clumsy, arbitrary, and bewildering. It is, simply, how (most) computing should be.
You can be as cooly aloof as you like about the device, but it won’t change the fact that it’s a fundamental step forward in computing. Many consumers can surely afford to sit this initial release out until the costs come down and the quality goes up. But if you work in tech, you should spend some time with an iPad. If it doesn’t change the way you think about what you do, you’re either a genius or an idiot.
Claims I Stand By
I made a number of claims back in January. I’ll revisit them now, in light of the reality of the situation.
Do I still think I wouldn’t have learned to program on an iPad? In the small, no; the device launched with a PHP IDE in the App Store on day one, amongst other code-related applications. But in the large, the device still discourages the kind of tinkering that’s possible on desktop computers, and I don’t think it should. One could learn to program on or for the device, but it’s not as easy and affordable as it could be. More on this below.
Do I still think the device isn’t for creative uses? In the narrowest sense, you can use it to create things: sketches, modifications to images, notes and other written works, music. But in the larger sense, most of what the iPad can be used to create at this point in time are not “first class” creative works. This is one of the odd things about our particular place in the history of art and technology: a program that allows someone to create music from pre-recorded loops is itself a first-class creation; the music that is pieced together from those loops may be enjoyable, but it has an innately lower creative value unless re-contextualized (for example, released by a hot record label, with no mention made of the software used to create the song). But ultimately, I’ll concede that the iPad will eventually become a tool for the creation and/or performance of first-class creative works.
Do I still think the iPad is “cynical”? I think it’s cynical of Apple to have a device that’s so closed from the get-go, but the goals of the device are so endearingly humanistic that you can’t really call it anything but hopeful. I think I was probably wrong on this count, but it’s something you can’t really understand until you’ve interacted with the thing.
All in all, I stand by what I had to say back in January: that the iPad is a beautiful, important, transformative device released under a confusing regime of questionable ethics. That said, I think three simple changes would make a world of difference towards assuaging people’s concerns about the iPad and Apple’s direction.
The Moderate Platform on iPad Openness
There’s been a lot of confusion about what a call for a more open iPad means. Cory Doctorow’s screed certainly didn’t help things, as it’s at the fiery, incoherent end of the spectrum of people who would like Apple to make some changes in policy and procedure. I don’t speak for folks like Cory, and I’d prefer not to be lumped in with them. I’m squarely between that “left-wing” rhetoric of openness-as-we’ve-known-it and the “right-wing” of Apple loyalists-cum-apologists who think that everything with iPad/iPhone is just dandy as is.
Here is my position, restated as clearly as I possibly can:
- Apple should not charge to put applications you’ve written onto your personal iPad (or iPhone, for that matter). If you purchase one of these devices, you should be able to install software of your own creation on it without any intervention or approval on Apple’s part, other than creating a free developer account. Essentially, take today’s iPhone/iPad developer program, and make it free.
- Apple should lift restrictions on running interpreted code on its mobile devices. Let people run Basic, Python, and Ruby interpreters on iPad and iPhone; we could make amazing tutorial applications for these and other programming languages. Let them run emulators for obsolete devices so intellectual property for them that has passed into the public domain can live on. Given that users can do much of this by virtue of Safari containing a JavaScript interpreter, the restriction seems arbitrary and backwards.
- Apple should remove the concept of private APIs from its developer offerings. Give developers the same tools that Apple’s own programmers get to use. If an API is still too unsafe or experimental for developers to make use of, don’t ship it, or gate it to development versions. Don’t restrict third parties from taking full advantage of the device and its software.
That’s it. That’s all I want from Apple to make the iPad and iPhone more open. I don’t think it’s crazy or unreasonable.
Yes, it’s a call for Apple to give up some revenue from developer program fees. As I explored back in January, we’re talking about something on the order of 0.001% of the company’s annual revenue. Apple has almost certainly spent more appeasing Greenpeace and the other environmental advocacy groups that have lobbied for changes to their manufacturing and shipping processes. 0.001% of revenue to enable a new generation of programmers and remove the most fundamental objection to Apple’s developer policies? That seems worth it.
What’s Not Being Argued For
Here, conversely, is what I am not arguing for:
- I am not arguing that the software behind the iPad should be released under any open source license, beyond what’s required to be released back to the open source community (something Apple is reasonably good about).
- I am not arguing that the iPad should be more modifiable by users at a hardware level, or that it should ship with any other standard ports, plugs, or interfaces.
- I am not arguing that Apple should abandon all use of DRM. I have faith that as various parts of the media industry get comfortable with the idea of unrestricted content, DRM will fade away, as it has from part of the iTunes Music Store catalog.
- I am not arguing anything about the ability to run background processes. I’m sure this issue will be addressed in an upcoming OS release, and Apple’s mobile products are more than usable without this functionality. (See, however, the item about doing away with private APIs above.)
- I am not arguing that Apple must make changes to its developer support procedures. I think they know that if they don’t do right by developers, developers will leave for other platforms. Apple gets this, and they appear to be trying to improve things. I don’t think developers have an innate right to sell their applications in Apple’s proprietary, controlled marketplace. That’s simply not how commerce works.
Finally, there’s the issue of the App Store. I’m on the fence about it. My hunch is that Apple should follow Palm’s lead and allow users to install applications from the web, albeit after prompting for the user’s consent and warning against whatever security issues might arise despite the platform’s sandboxing. I also think that Apple should operate their App Store on an open, published API, and allow alternative app stores to be opted into by users. However, I don’t feel strongly enough about these positions to make them part of my “platform” above.
Aren’t You On Hiatus?
Am I blogging again? I’m not sure. This is, uh, a blog post, certainly. I didn’t really plan to write anything else in the short term, and I’m still focused on other projects. This is more of a special feature, let’s say. See you later this year.
Post-Script: April 14, 2010
After a little less than two weeks with an iPad, I’ve decided to sell it. Currently, it makes more sense to sell an iPad to someone outside the US who won’t be able to buy one for another month (at least) than it does to pay Apple’s $65 restocking fee. The iPad is still a fascinating device, and I’m glad I got to play around with one, but it needs work, and I’m waiting until the next version to get really invested in it.
The iPad is too heavy; that extra half-a-pound makes a big difference in many configurations. The screen is too bright for low-light conditions even with the brightness cranked all the way down. Generally, it’s hard to get comfortable with the device, even with a decent case. Typing more than a tweet is un-ergonomic and painful. Plus, most of the really nicely designed software for iPad currently doesn’t handle over-the-air/cloud sync, including Apple’s own iWork apps. That makes juggling an iPad, a mobile device, and a personal computer a major hassle. (Yes, Dropbox is great, but it’s not currently a solution to this problem. That’s up to developers, and to Apple.)
There are things I’ll miss about the iPad. I love having my entire library of academic papers with me at all times. The Kindle experience on it is far better than Amazon’s own hardware. I sold my Kindle 2, a decision I don’t regret despite not holding onto the iPad; the Kindle is simply not adequate for anything more than the most basic texts, and the iPad makes that abundantly clear. The Instapaper app is great, and I’ll definitely miss using it. Beyond those good points, I found myself reaching for my laptop and phone often enough that owning a largely redundant $800+ device for occassional use seems frivolous. I offered the iPad to my wife, but she had much the same reaction of “I’m not sure I really need this or would use it day-to-day, save for travel”.
It’s still hard to swallow owning a device that’s subject to Apple’s increasingly aggressive developer-hostile policies, no matter how carefully you paint those policies as “preserving the integrity of the platform” or whatever helps you sleep at night. Apple has absolutely no reason to listen to me or any other critic, as they’re busy making buckets of money and, for the most part, making users happy. Since I first wrote this post, though, Apple has moved even further away from being “moderately open” with their new restrictions on the programming languages that can be used on their platform. I don’t expect that trend to reverse, and that makes it harder for me to spend my money on Apple’s closed products without feeling guilty.
I’ll probably buy an iPad or something like it again in the future. In a year or two, they’ll be cheaper, lighter, and generally better. For now, it’s fun to play around and soak up all the creative interaction design that’s happening on the iPad, but it’s just not a purchase I can justify.
—Apr 05, 2010
The Emerging Languages Conference
Simply a pointer to an event that’s likely of interest to readers of this blog.
If you like programming languages and you care about how they’re evolving, set aside July 21st and 22nd of this year. O’Reilly’s Brady Forrest and I have put together the first Emerging Languages Conference, co-located with OSCON, which returns to Portland, Oregon this year.
The event is designed to get language designers, researchers, implementors, and enthusiasts in a room for two days to share experiences, successes, disasters, ideas, and hopes for the future. In the morning, folks working on interesting young languages (or language-related tech, like runtimes) will present 30 or 15 minute talks about their projects. In the afternoons, we’ll break into to freeform “unconference”-style sessions where attendees can focus on whatever floats their digits: compiler design, type systems, interpreters, source-to-source translation, etc. etc.
It’s my hope that this event will connect people who didn’t know they had common goals, in the process helping to shape the future of programming languages. I think it’s going to be a big deal.
We’re going to keep tickets under $200 for attendees, and that price includes an as-yet-undetermined level of access to OSCON (at the very least, you’ll get to see the keynotes). If you’re giving a talk, the event is free, of course. We’re trying to secure sponsorships to fly in folks from faraway lands who are on a tight student/academic budget.
Check out the blog, follow the event on Twitter, join the Google Group, and hold tight for registration information. Hope to see you there!
—Apr 05, 2010
