diego's weblog

there and back again

Category Archives: science

the importance of Interstellar

iDo not go gentle into that good night,
Old age should burn and rave at close of day;
Rage, rage against the dying of the light.

                                                    Dylan Thomas (1951)

Over the last few years a lot of movies -among other things- seem to have shrunk in ambition while appearing to be”bigger.” The Transformers series of movies are perhaps the best example. Best way to turn off your brain while watching fights of giant robots and cool explosions? Sure. But while mega-budget blockbusters focus on size, many of them lack ambition and scope. Art, entertainment, and movies in particular, given their reach, matter a lot in terms of what they reflect of us and what they can inspire. For all their grandiose intergalactic-battle-of-the-ages mumbo jumbo, Transformers and other similar movies always feel small, and petty. Humans in them are relegated to bit actors that appear to be props necessary for the real heroes (in this case, giant alien robots) to gain, or regain, inspiration and do what they must do. And always, always by chance. Random people turn into key characters in world-changing events just because they stumbled into the wrong, or right, (plot)hole.

Now, people turned into “the instruments of fate (or whatever),” if you will, is certainly a worthwhile theme and something that does happen. But stories in which the protagonists (and people in general) take the reins and attempt to influence large-scale events through  hard work, focus, cooperation, even -gasp!- study, became less common for a while. Art reflects the preoccupations and aspirations of society, and it seems that by the mid-to-late 2000s we had become reliant on the idea of the world as reality TV – success is random and based on freakish circumstances, or, just as often, on being a freak of some sort. This isn’t a phenomenon isolated to science fiction — westerns, for example, declined in popularity but also turned “gritty” or “realistic” and in the process, for the most part, trading stories of the ‘purity of the pioneering spirit’ or ‘taming the frontier’ with cesspools of dirt, crime, betrayal and despair.

Given the reality of the much of the 20th century, it was probably inevitable that a lot of art (popular or not) would go from a rosy, unrealistically happy and/or heroic view of the past, present, and future, to a depressing, excessively pessimistic view of them. Many of the most popular heroes in our recent collective imaginations are ‘born’ (by lineage, by chance, etc) rather than ‘made’ by their own efforts or even the concerted efforts of a group. Consider: Harry Potter, the human characters in Transformers (and pretty much any Michael Bay movie since Armageddon), even more obviously commercial efforts like Percy Jackson or Twilight along with other ‘young adult’ fiction and with pretty much all other vampire movies, which have the distinction of creating ‘heroes’ simultaneously randomly and through bloodlines, the remake of Star Trek turned Kirk joining Starfleet into something he didn’t really want to do; the characters in The Walking Dead; the grand-daddy of all of these: Superman… and, even, as much as I enjoy The Lord of The Rings, nearly everything about its view of good and evil involves little in the way of will and intent from the main characters. Characters talk a great deal about the importance of individuals and their actions, but in the end they’re all destined to do what they do and the key turning points are best explained as either ‘fate’, simply random, or manipulated by people of ‘greater wisdom and/or power’ like Gandalf, Galadriel, Elrond and so on. Good and evil are defined along the lines of an eugenics pamphlet in a way that gets to be creepy more often than not (the ‘best’ are fair-skinned, with blue or green eyes, and from the West, the ‘worst’ are dark-skinned, speak in hellish tongues and are from the East, along with an unhealthy obsession with bloodlines and purity of blood, and so on; Gandalf “progresses” from Gray to White, while Saruman falls from being the leader as Saruman the White into shrunken evil serving Sauron, the Dark Lord… as “Saruman of Many Colours”… you get the idea).

All of which is to say: I don’t think it’s a coincidence that in this environment good Science Fiction in general and space exploration SF is always relegated a bit, particularly in movies. There is nothing random about space exploration: it requires an enormous amount of planning, study, effort, hard work, and money. You can’t inherit a good space program. It has to be painstakingly built, and supported, across decades. When a not-insignificant percentage of society flatly discards basic scientific theories in favor of religious or political dogma while giving an audience to Honey Boo Boo or Duck Dynasty, it’s not illogical for studios to finance another animated movie with talking animals than to push people beyond their comfort zones.

Even so, there’s always been good SF, if perhaps not as frequently as SF fans would like. And over the last 20 years we have started to see  Fantasy/SF stories that combine a more “realistic” view of the world, but mixed in with the more idealistic spirit of movies like The Right Stuff. In these we have characters succeeding, or at least ‘fighting the good fight’, through exertion of will, the resolve to change their reality. And even if there’s an element of ‘fate’ or chance in the setup, the bulk of the story involves characters that aren’t just pushed around by forces beyond their control. Nolan’s Dark Knight trilogy, Avatar, Serenity, most of Marvel’s new movies: Iron Man, Captain America, The AvengersWatchmen. In books, the Already Dead series and the Coyote series, both of which could make for spectacularly good movies if ever produced. In TV, Deadwood, which is perhaps the best TV series of all time, was a good example of the same phenomenon — it felt realistic, but realistically complex, with characters that weren’t just swept up in events, and that exhibited more than one guiding principle or idea. We got ‘smaller’ movies like Moon that were excellent, but large-scale storytelling involving spaceflight that wasn’t another iteration of a horror/monster/action movie is something I’ve missed in the last few years.

What about last year’s Gravity? It was visually arresting and technically proficient but fairly mundane in terms of what actually happens. It’s not really inspiring — it’s basically the story of someone wrecking their car in the middle of the desert and having to make it to the next gas station… but in space, the focus on experiencing a spiritual rebirth, and in case we were confused about the metaphor the see the main character literally crawl out of mud and water and then slowly stand and start to walk. Bullock’s character in Gravity is also one of those guided by circumstances, frequently displaying a lack of knowledge about spaceflight that even the original monkeys that flew in the early space missions would have slapped their foreheads about.

Which brings me to Interstellar. No doubt it will be compared to 2001: A Space Odyssey (with reason) and with Gravity (with less reason). Interstellar is more ambitious than 2001 in terms of science, matching it or exceeding it in terms of story scope and complexity, while leaving Gravity in the dust.  2007’s Sunshine shares some themes and some of the serious approach to both science and fiction (… at least the first 30 minutes or so, afterwards it shares more with Alien) as well as with the (in my opinion) under-appreciated Red Planet (2000) and even some elements of the much less convincing Mission to Mars. It also reminded me of Primer in terms of how it seamlessly wove pretty complex ideas into its plot.

We haven’t had a “hard” SF space movie like this for a whileKey plot points involving gravitational time-dilation, wormholes, black holes,  quantum mechanics/relativity discrepancies… even a 3D representation of a spacetime tesseract (!!!!). 2001 was perfect about the mechanics of space flight, but Interstellar also gets as deep into grand-unified theory issues as you can probably get without losing a lot of the audience, and goes much further than 1997’s Contact. There are some plot point that are weak (or, possibly, that I may have missed an explanation for, I’ll need another viewing to confirm…), and sometimes there are moments that feel a bit slow or excessively, shall we say, ‘philosophical’, although in retrospect the pauses in action were effective in making what followed even more significant.

Comparisons and minor quibbles aside, Interstellar is spectacular; the kind of movie you should, nay, must watch in a theater, the bigger screen the better, preferably on IMAX.

The movie not only has a point of view,  it is unapologetic about it. It doesn’t try to be “balanced,” and it doesn’t try to mix in religion even as it touches on subjects in which it frequently is mixed in the name of making “all points of view heard.” Interstellar is not “anti religion” … and it is not pro-religion either. There’s a fundamental set of circumstances in the plot that allows the movie to sidestep pretty much all of the usual politics and religion that would normally be involved. Perhaps someone can argue whether those circumstances are realistic (although something like the Manhattan project comes to mind as an example of how it can actually happen). But the result is that the movie can focus almost exclusively on science, exploration, our ability to change things, either individually or in groups.

This, to me, felt truly refreshing. Everything that has to do with science these days is mixed in with politics and/or religion. This also helps the story in its refusal to “dumb things down”…  its embrace of complexity of ideas, even if less focused on a lot of specific technical details than, say, Apollo 13 was, which is a natural result of having the Apollo data at hand.

How many people, I wonder, know by now what NASA’s Apollo program really was? Sometimes it seems to be relegated to either conspiracy joke material or mentioned in passing to, for example, explain how your phone is more powerful than the computers that went to the moon. Somehow what was actually attempted, and what was actually achieved, isn’t remarkable anymore, and the true effort it took is less appreciated as a result. With that, we are making those things smaller, which gives us leeway to do, to be less. It makes “raging against the dying of the light” sound like a hopelessly romantic, useless notion. It justifies how approaching big challenges these days frequently happens in ways that makes us “involved” in the same way that Farmville relates to actual farming. Want to feel like you’ve solved world hunger? Donate $1 via text to Oxfam. Want to “promote awareness of ALS”? Just dump a bucket of ice water on your head. Want to “contribute in the fight against cancer”? Add a $3 donation while checking out of the supermarket. No need to get into medicine or study for a decade. Just bump your NFC-enabled phone against this gizmo and give us some money, we’ll do the rest.

I’m not saying that there is no place for those things, but recently it seems that’s the default. Why? Many commentators have talked about how these days we lack an attitude best described by Kennedy’s famous line “Ask not what your country can do for you, as what you can do for your country”. But I don’t think the issue is not wanting to do anything, or not wanting to help. I think the issue is that we have gotten used to being scared and feeling powerless in the face of complexity. We’ve gone from the 60’s attitude of everyone being able to change the world to feeling as if we’re completely at the mercy of forces beyond our control. And we’ve gone overboard about whatever we think we can control:  people freaking out about the use of child seats in cars, or worrying about wearing helmets when biking, while simultaneously doing little as societies about the far greater threat of climate change.

When education was a privilege of very few, very rich people, it was possible for pretty much everyone to accept a simplistic version of reality. That was before affordable mass travel, before realtime communications, before two devastating world wars and any number of “smaller” ones. Reality has been exposed for the truly messy, complicated thing it is and always was. But instead of embracing it we have been redefining reality downwards, hiding our collective heads in the sand, telling ourselves that small is big. Even heroism is redefined — everyone’s a hero now.

Interstellar is important not just as a great science fiction movie, not just because it is inspiring when it’s so much easier to be cynical about the past, the present or the future, but also because beyond what it says there’s also how it says it, with a conviction and clarity that is rare for this kind of production. It’s not a coincidence that it references those Dylan Thomas verses more than once. It’s an idealistic movie, and in a sense fundamentally optimistic, although perhaps not necessarily as optimistic about outcomes as it is about opportunities.

It’s about rekindling the idea that we can think big. A reminder of what we can attempt, and sometimes achieve. And, crucially, that at a time when we demand predictability out of everything, probably because it helps us feel somehow ‘in control’, it is also a reminder in more ways than one that great achievement, like discovery, has no roadmap.

Because if you always know where you’re going and how you’re getting there you may be ‘safe’, it’s unlikely you’ll end up anywhere new.

diego’s life lessons, part III

Excerpted from the upcoming book: “Diego’s life lessons: 99 tips for survival, fun, and profit in today’s baffling bric-a-brac world.” (see Part I and Part II).

#9 make the right career choices

Everyone will have seven careers in their lifetime, someone said once, and we all repeated it even if we have no idea why.

The key to career planning, though, is to keep in mind that while the world of today ranges from complicated to downright baffling, the world of tomorrow will be pretty predictable, since as we all know it will just be a barren hellscape populated by Zombies.

So the question is: post-Zombie Apocalypse, what will you need to be? Survival in the new Zombie-infested world will require the skills of any good D&D party: a Healer, a Warrior, a Thief, and a Wizard — which in a world without magic means someone to tinker with things, build weapons, design shelters with complicated spring traps, and knowledge of how to brew a good cup of coffee.

Clearly you don’t want to be a Healer (read: medic/doctor), since that means no one will be able to fix you — you should have friends or relatives with careers in medicine, however, for obvious reasons. Being a Thief will be of limited use, but more importantly it’s not really the kind of thing you can practice for without turning to a life of crime as defined by our pre-Zombie civilization (post-Zombies, most of the things we consider crimes today will become fairly acceptable somehow, so you may be able to pull this off with the right timing).

That leaves you with either Warrior or Wizard, which translates roughly to: Gun Nut or Hacker. And by “Hacker” we mean the early-1980s definition of hacker, rather than the bastardized 2000s version, and one that is not restricted to computers.

So. Your choices for a new career path are as follows:

  • If you’re a Nerd, become a Hacker.
  • If you’re neither a Nerd or a Hacker, just become a Gun Nut, it’s the easiest and fastest way to post-apocalyptic survival. This way, while you wait for Zombies to strike you won’t need to worry (for example) about a lookup being O(N) or not, or why the CPU on some random server is pegged at 99% without any incoming requests.
  • If you’re already a Gun Nut, you’re good to go. Just keep buying ammo.
  • If you’re already a Hacker… please don’t turn into an evil genius and destroy the world. Try taking up some activity that will consume your time for no reason, like playing The Elder Scrolls V: Skyrim or learning to program for Blackberry.

NOTE (I): If you’re in the medical profession, just stay put. We will protect you so you can fix our sprained ankles and such.
NOTE (II): there is also the rare combination of Hacker/Nerd+Gun Nut, but you should be aware that this is a highly volatile combination of skills which can have unpredictable results on your psyche.

#45: purchase a small island in the Pacific Ocean

As far as having a permanent vacation spot, this one really is a no-brainer. Why bother with hotels when you can own a piece of slowly sinking real estate? Plus, according to highly reliable sources, you don’t need to be a billionaire.

True, you will have significant coconut-maintenance fees and you’ll probably need a small fleet of Roombas to keep the place tidy, but coconuts are delicious and the Roombas can help in following lesson #18.

NOTE I: don’t be fooled by the “Pacific” part of “Pacific Ocean.” There’s nothing “pacific” about it. There’s storms, cyclones, tsunamis, giant garbage monsters, sharks, jellyfish, and any number of other dangers. Therefore, an important followup to purchase the island is to buy an airline for it. You know, to be able to get away quickly, just in case.

NOTE II: this is actually an alternative to the career choices described above, since it is well known that Zombies can’t swim.

NOTE III: the island should not be named Krakatoa — see lesson #1. Aside from this detail, owning a Pacific Island does not directly conflict with lesson #1, since the cupboard can be actually located in a hut somewhere in the island (multiple cupboard hiding spots are also advisable).

#86 Stock up on Kryptonite

Ok, so let me tell you about this guy… He wears a cape and tights. He frequently disrobes in public places. He makes a living writing for a newspaper with an owner that makes Rupert Murdoch look like Edward R. Murrow. He has deep psychological scars since he is the last survivor of a cataclysmic event that destroyed his civilization. He leads a secret double life, generally disappearing whenever something terrible happens. He is an illegal alien. Also, he is an ALIEN.

Does this look like someone trustworthy to you? Hm?

That’s right. This is not a stable person.

Add to the list that he can fly, even in space, stop bullets, has X-ray vision, can (possibly) travel back in time and is essentially indestructible. How is this guy not a threat to all of humanity?

Lex Luthor was deeply misunderstood — he could see all this, but his messaging was way off. Plus there were all those schemes to Take Over The World, which should really be left to experts like genetically engineered mice.

The only solution to this menace is to keep your own personal stash of Kryptonite. Keep most of it in a cupboard (see lesson #1) and a small amount on your person at all times.

After all, you never know when this madman will show up.

santa claus conquers the martians

Part 2 of a series (Part 1Part 3)


I’ve had a busy week, and have been trying to sit down and put together a followup to my response to the NYT’s article on data centers.

I write the title, and I soon as I do, my mind goes blank. I read the title again. What the hell was I thinking? I am looking at the screen, white space extends below the blinking cursor, mirrored by something somehow stuck in my head, alternating on/off, rumbling lowly like an idling engine: I swear I had a point.

So naturally I start to think that this, perhaps, should be the new title. Which, in the expected recursion path that would follow naturally ends up in another meta-commentary paragraph (also with a simile close to its ending), which I decide not to write. Recursion upwards, probably to conform with an implicit image of happiness we may or may not feel (or is in this case is really quite unwarranted and even more, even worse: unnecessary) but we should generally imply anyway, because these days if you’re not explicitly happy something must be wrong, and therefore it must be fixed. Neutral has become a bad state to be in, apparently, long after being “with us or against us” became a common way to think about nearly everything. No, recursion has no direction except, perhaps, into itself, but it now occurs to me that years of looking at function call stacks have trained me (hopelessly comes to mind, but that’s also not happy) to think of recursion as up or down, rather than, say, horizontally from right to left.

Fascinating, I know.

– oOo –

I will eventually get to Santa Claus and the Martians, but for the moment, back to the article.

The series was titled “The Cloud Factories”, and right there it broadcast ever-so-subtly that it was to be something intended to get worked up about.

“Factory” can mean “the seat of some kind of production” but in this case the weight of the word is in the manufacturing angle. This doesn’t quite feel right, though. A factory is where things are built, sequentially, or at least mostly sequentially, and a cloud is anything but built, and the process is anything but sequential. A cloud emerges, and if we switch to the definite article and the proper noun with all its implications and uppercaseness, it’s also true that The Cloud is an emergent phenomenon. Metaphors are often misapplied, can be incorrect, but it’s not that often that a metaphor involving an overloaded term (“cloud”) is both misapplied and incorrect in the exact same way for nearly all the meanings of the term. This takes some skill.

So, yeah, the point of the title of the series was not to be accurate as an analogy, but to evoke. Specifically, an image. Much like the factory in which they make Itchy & Scratchy cartoons in The Simpsons has chimneys and dark dense smoke coming out of them, as does every factory in The Simpsons, regardless what it’s for. The “factories” in the “The Cloud Factories” seem to intentionally or not (but can this really be unintentional?) transmit the idea of dirt we associate at a reptilian level with “factory”. Dirt. Pollution. Guilt by association. Then — the title of the article, the first of two so far, drops the subtle imagery: “Power, Pollution and the Internet.” Strangely enough, beyond the title the word “pollution” appears exactly once in the entire article.

Pollution and the Internet. How could one not react to that? What I wrote a week ago was pure reaction, if nothing else to the reactionary tone of the article, but by now I have accumulated enough in my head to maybe add something else to this topic, which, perhaps predictably, has a bit less to do with the contents of the article itself (not that that topic is exhausted by any means) but on what is one possible way to look at its main thrust through the lens of discourse on technology nowadays, how we use metaphors and analogies to convey something that we haven’t yet internalized, and the factors at play in sustaining a reasonable and reasonably deep conversation in an environment that doesn’t lend itself to that. And if all of this in retrospect looks obvious, consider this the admittedly convoluted way in which I am creating a reminder, a mental note: something to pay more attention to.

On to it, then.


Action — argument (paraphrasing, summarizing): “That which powers our online services and more generally the Internet is really a hidden pollution machine run by people fearful of reducing waste, even though the means to do so are readily available.”

Reaction — counterargument (now really summarizing): “Not true.”

That the argument isn’t true may be indeed true, and yet to not just agree with the counterargument because, for example, you respect whoever made it but to understand it requires a degree of experience and training and knowledge that is well beyond what most people could get to because, quite simply, they have their own jobs and lives. Indeed, if it’s not your job and it’s not your life (and for most of the people for whom this is a job, it’s also our life), you really shouldn’t bother. The modern world, and to some degree the very basis of our progress is that we use things that we can’t build, and in many cases can’t even understand. We travel by plane even though many people have no idea how it works, let alone are able to build one.

And that is just fine.

We trust the plane, though, don’t we? Well, now we do, but 150 years ago the thought that you could pack tons and tons of baggage and instruments and hundreds of people into a tin can and by pushing air at unimaginable speed through smaller tin cans attached to the larger tin can with bolts you would get the thing to fly was unpopular indeed.

Bear with me for a minute here. I’m getting somewhere. Promise.

As I was writing a week ago I was typing frantically and in the process of switching windows I entered “action reaction retraction” into Google, and the last result visible before I had to scroll said “Robert H. Goddard. The New York Times.” which seemed intriguing enough, and following there were notes on a retraction that seemed almost too appropriate. Really? was the thought, so I went to the Times archives and found the quote, but in the process lost the bizarre way in which I stumbled on to it. I spent almost an hour yesterday, I kid you not, going through the browser’s history to see what I’d done, and I still can’t remember why I was typing that except to think that I must have read this before, and further googling just for the quote shows that it’s been mentioned a few times in the last several years. Sarah Lacy included the quote in her followup, along with her own thoughts regarding an earlier Times story on Tesla motors which shows if not a pattern at least some concordance of mistakes all going in the same direction, or misdirection.

The quote was a retraction from the Times in which it acknowledges:

“Further investigation and experimentation have confirmed the findings of Isaac Newton in the 17th century and it is now definitely established that a rocket can function in a vacuum as well as in an atmosphere. The Times regrets the error.”

This was triggered by Apollo 11’s flight, when, one presumes, a 50-year-old takedown of rocket pioneer Robert Goddard on the very pages of the Times might have come to their attention:

“That Professor Goddard, with his ‘chair’ in Clark College and the countenancing of the Smithsonian Institution, does not know the relation of action to reaction, and the need to have something better than a vacuum against which to react — to say that would be absurd. Of course he only seems to lack the knowledge ladled out daily in high schools.”

The Times regrets the error. This reminded me of what we could call the case of Catholic Church v. Galileo. At least the Vatican actually apologized to Galileo directly, although in fairness to the Times, it took the Vatican closer to 400 years to get to that point.

The reason I bring up the quote again is that there’s a certain tone of mischief detectable in it, since no one can possibly believe that they are seriously a) realizing just now that rockets actually work in a vacuum and b) that the way to correct for this is to say that “this confirms the findings of Isaac Newton.” Points for whoever wrote it: it was funny.

And just to be clear: this isn’t about giving a pass to the Times, but to try to figure out why this seems to be a recurring problem from which the Times seems far from exempt, even when we may be inclined to think they are exempt from it.

The question is, then, why would they, the nebulous they but that nevertheless is actually people, talented as they may be, would have originally thought that trashing Goddard, someone with enough credentials to presumably give him the benefit of the doubt at the very least, was a good idea?

Perhaps because in doing that they were reflecting, ahem, the times — the prevailing sense of what was or wasn’t possible in the age. The “truth” as they saw it, because truth and facts are two different things. To top it off, in this particular case a giant rocket traveling at some 11,000 meters per second was, as an undeniable fact, still very much in the future, but when the rocket was actually up there, actually carrying three people and countless gizmos and measuring devices and chemicals of all kinds, you didn’t have to know anything about physics to realize that there was something to this seemingly crazy idea of rockets in space after all.

Back to trusting airplanes at last: We trust the plane because we see it. We feel, down to our bones, the effort of the engines as it takes off and lands. If someone started to argue that the typical turbine was somewhat wasteful, I don’t think I’d be alone in thinking Well, while I’m inside the plane and on the air, I’d prefer a little waste to not being, you know, alive.

So is there something to the idea that, in the popular imagination, not seeing is disbelieving, to invert the well-known dictum?

More importantly, given the complexity and sheer scale of the systems involved in running the Internet, what would it take to “see” when what we’re talking about can’t, ever, actually be seen?


That’s a line from Batman Begins uttered by Mafia mob boss Carmine Falcone while he is explaining to a young Bruce Wayne why he should just stop acting all flustered about crime and go home. It’s a critical line not only in the film but in the overall story arc of the trilogy, since within it we find Bruce Wayne’s drive to become Batman. Bruce agrees with Falcone’s thesis but not his solution, decides to understand, disappears into the underworld, then returns, seven years later, as Batman.

Understanding — not fearing — takes knowledge, and knowledge takes a long time and effort to develop.

Convincing people that flying rockets in space “only” required that we actually fly a rocket in space. What would be the equivalent for getting people to accept that how data centers work is not some perennial waste, where secret gerbils run mindlessly within wheels, most of the time doing nothing at all, wasting energy and in the process laying waste to the planet as well? Well, one way would surely be to getting everyone to spend the equivalent of Wayne’s “seven years in the underworld” which in this case would be not only getting a degree in computer science but spend a good amount of time down in the trenches, seeing firsthand how these things are actually run.

That this is an impractical solution, since we can’t have the whole planet get a CS degree or work in a data center, is obvious. It leaves us with the alternative of using analogies and metaphors to express what people still haven’t internalized, and probably will never be able to internalize, in the way that they have the concept of a rocket or an airplane. Before planes flew, the idea of them also had to be wrapped in analogies and metaphors, usually involving birds. The concept of a factory would have undoubtedly required some heavy analogies to be explained to people in, say, the 16th century. We grasp at something that is known to make the unknown intelligible.

The analogies we choose matter, however. A lot. Which is why I keep talking about planes not factories. A modern commercial jet is a much more apt analogy for the type of “waste” involved in running a modern data center.

There is waste and pollution involved in running a jet, as anyone can plainly see. Sometimes the waste is obvious (empty seats), sometimes it’s not (unnecessary circuitry), but generally people don’t doubt that the good people at Boeing et. al. are always doing their damnedest to make the plane as efficient, safe, and effective as possible. The same is true of Internet infrastructure.


You may or may not agree with the plane analogy, there may be better ones, there are more things to discuss and there certainly is a need for us in the industry to engage more broadly and try to explain what’s going on as long as everyone in the world doesn’t have a CS degree (a man can dream).

So for all the faults I could find with the article, I think it was good that it triggered the conversation, and herein lies our second conundrum.

This “conversation” — it will require effort to be carried out.

A brief detour: reading Days of Rage a commentary in the latest issue of The New Yorker, which references Santa Claus Conquers The Martians while talking about the “Muslim Rage” of recent days over a YouTube video no one had actually seen, certainly not before the protests. I agree with a lot of the article, except on one point:

“The uproar over “Innocence of Muslims” matters not because of the deep pathologies it has supposedly laid bare but because of the way the film went viral.”

Psy and Gangnam Style was viral. This video wasn’t. If anything, from what we know, it seems to be quite the oppositeof viral, since apparently it was simply an excuse used by people in power to rile up the unhappy (there’s that word again) masses so they could have something to do: “Angry? Unemployed? Bored? Feel you have no future? Here, go burn an embassy.” And how irrationally angry you have to be to somehow find that looting and burning and killing either solves a problem or makes up for anything or is even, just, a remotely justified way to react. How displaced you have to be from yourself and disconnected to what surrounds you. I can hypothesize, only. At points in my life I’ve had little or no money but never felt in a way that would ever lead me to react in that way. Not that this is about money, I know, it’s just one of the factors (probably), but one that I can try to relate through. But I digress.

SCCTM is indeed an actual movie and the reason I bring it up is that I had seen it years ago in an MST3K episode, and when remembering that it occurred to me that what happened in the Middle East was a more, perhaps the most, extreme version of a pervasive phenomenon, that of reacting to what our perception is of something rather than to the thing itself.

Mind you, this isn’t one of those “things were better in my time” type of arguments. While there was a time decades ago when in-depth roundtables in media were more common fare, this happened in an environment in which the amount of raw data to process was far, far less than it is now. We are overwhelmed by data but lacking in information. This isn’t a matter of access to technology, either. I’d bet a lot of the people doing the burning and killing in Benghazi had cellphones. We all do.

This, deep in the weeds of this post (essay?), is what triggered the topic in my head. The end of the chain of associations: that what we’re often doing these days to handle all the information that we’re exposed to would be tantamount to MST3K dispensing with the actual viewing of the movie and simply skipping to the part where we make fun of it. It wouldn’t be the same, would it? Context is critical, but we react in soundbites and generate storms of controversy over a few words which can’t possibly have context attached, because there’s simply no space for it, anywhere.

Twitter and to some degree Facebook are often blamed, unfairly I think, with a supposed devolution of our society into people trapping their thoughts into contextless cages 140-characters in size. I don’t think there’s any question, though, that we humans are and have always been lazy if we can get away with it, and that the deluge of information leave us with little time to reflect on it, so the mind recoils and defends itself with quips and short bursts, and Twitter (and Facebook) are a good mechanism for that. It just so happens that this constant jumping around topics superficially is both a) effective as a dopamine release mechanism –read: addictive– and b) the perfect way of thinking of yourself as informed and on top of everything and yet truly involved in nothing. Why isn’t Twitter or Facebook to blame, then. Let me give you a Twitterless example: sad advertisement on TV, people starving, a catastrophe somewhere. Text a number and give $3. Done. Back to watching Jersey Shore, or 60 minutes, or whatever.

Twitter, Facebook, all of them, are not the proximate cause. They are an effect. A reaction.

The environment we live in has fundamentally changed because there is readily available, quite simply, more data about everything, a large part of which is a barrage of trivia and gossip — which is to be expected since they are, ahem, trivial to generate. If Lindsay Lohan having a traffic accident is enough to generate massive news coverage and the cascade of reaction that follows, topics that are deeper and more complex and are more difficult to grasp will find it hard to compete.

It’s something new, or relatively new in historical terms, and I don’t think we know how to handle this deluge yet. We are drinking from a seemingly limitless flood of information but we haven’t yet figured out how to close the faucet every once in a while. We don’t necessarily drown in it but this flood that is constantly rushing around us leaves us with no time to reflect on any one point.

Information overload! Pfft. This isn’t a new idea! I bring it up not only because I think that we are increasingly using (creating) media that is suited to how we are trying to deal with it, and the edifice we construct with all of it is not well-optimized to transmit complex ideas (this, also, is not at all original), and so it seems critical that we have to work hard at finding the right metaphors and analogies, the right tools to talk about how the machinery of the Internet works. Tools and machinery, here, somewhat ironically encapsulating the point.


Analogies matter, metaphors matter, and we need to find better ones to talk about what the Internet is (for example, a “global village” it is not, and this term has luckily fallen by the wayside, but the many reasons why will have to wait for another time). We also have to contend with a shifting media environment in which a conversation like this can get all too easily lost in the noise, not because, as a cynical interpretation would have it, people only care about Snooki or the Kardashians or whatever, but because until we figure out how to live and engage with complexity when soaking in data there will only only surface and precious little depth.

And if there’s an additional meta-point to “Power, Pollution and the Internet,” something else that is important beyond the specifics in the article, it is that we as an industry have left a void that can be filled with anything, and if we don’t engage and try make what we do more comprehensible for everyone who, rightly, doesn’t have the time to understand it because they’re busy running the rest of the world, then we in the industry have no one to answer to for it but ourselves.

Part 2 of a series (Part 1Part 3)

diego’s life lessons, part II

Excerpted from the upcoming book: “Diego’s life lessons: 99 tips for survival, fun, and profit in today’s baffling bric-a-brac world.”  (part I is here).

#6: sign up for every personalization app available

“Big data” is all the rage these days. We are told that a few machines running Hadoop can figure out the deepest triggers in your psyche from only a few Facebook ‘Like’ actions. Companies crop up every other day promising to figure out what kind of ice cream flavor you’re going to like because you buy a certain color of bath towel. And they’re right, of course. A human being’s life can absolutely be defined by a linear regression on a few loosely correlated data points. It doesn’t matter if what they’re correlating is a couple dozen people in Tennessee having burgers, while you live in Sweden and like to unwind watching The Seventh Seal over and over. People are people, it doesn’t matter who they are, where they are, why they are, or how much they are. Humans are just not that interesting or different from each other. If you need proof, go look for videos of stake boarding puppies on YouTube and count the views.

It used to be these were websites, but now they’re just apps. They follow you, track your every move, and give you suggestions about all the things that you should be doing. By signing up for every one of these apps, you not only ensure that you will be doing as little thinking as possible, you will be ahead of the curve on this I-don’t-need-to-think-or-do-anything trend. Soon, apps will shift from giving recommendations and predictions to just telling you where you should go. The inevitable next step is that the app will do it for you — you won’t need to move a finger. An app will tell you that you should be having dinner somewhere (eventually even skipping the part where it tells you), check you in, tweet for you about how delighted you are with the dinner you’re supposed to be having, all from the comfort of your home while you’re eating canned beans. Then you can hide in a cupboard in peace (See rule #1).

In time, the apps will even date and have their own, um, applets. Which will let them achieve the app singularity and just bypass humans altogether (for the most part. We will still have to keep the phones plugged so the apps can run. We’ll basically be tasked with providing electricity for them, something that The Matrix got right).

So get on board. Before an app does it for you.

#18: raise your own tiny robot army

Wake up call: Do you really think that people are getting Lego Mindstorms just for the kids? We have been warned time and again of the consequences of a decaying civilization, impending apocalypse, and whatnot. What we have not been told is what happens when someone else in your block has a robot army. Trust me, you need one.

A tiny robot army comes in handy in countless ways. Whether you’re engaging in hostilities with the jerk from the apartment right above yours that won’t stop playing loud music, or just escorting your dog while you take him for a walk, a robot army will always come in handy. In peacetime, the robots can be deployed to do various chores, like washing dishes, cleaning your house, or carrying bags from the supermarket. And every once in a while, you can use them to invade a nearby property if you’re suspicious that they may be threatening your way of life, by, for example, hiring a new gardener to change the shapes of their shrubbery. In a pinch, if, say, your Internet connection is down, they may be dressed in hilarious tiny costumes and made to enact Shakespeare plays for you. You haven’t really experienced Hamlet until you’ve seen it performed by a tiny robot army.

And I know what you will say: “I’ll call Aquaman!” But, really, Aquaman is useless outside of water, and he isn’t real. Batman only works Gotham, so unless you live there, you’re on your own. Get a robot army ready, or suffer the consequences.

#50: be friendly with squirrels

Godzilla has battled many enemies: Mothra, Space Godzilla, Megalon, Mechagodzilla. All worthy foes. But Godzilla never had to face a horde of angry squirrels.

Squirrels are a force to be reckoned with. They have multiple powers: blinding-fast movement, jumping, crawling up surfaces, high-speed sniffing, some can even fly. They are fearsome foes of cabling: squirrels can chew through pretty much anything, given enough time, including but not limited to fiber optic cables, coax and the wiring system of your car, which I have personally experienced.

Some people have said that squirrels are just fancy rats with fur coats, but nothing could be further from the truth. They’re highly advanced creatures, as you can see from the following photographic evidence:

That’s right. Anti-tank weaponry. Tiny violins. Lightsaber fights. Squirrels can handle them all. They even had the forethought, thousands of years ago, of preparing for today’s Jurassic Park-like experiments with plants.

Naturally, squirrels cannot be true friends of mankind, since we compete for the same lightly-forested Internet-enabled high-garbage-density habitats known as the suburbs. So “friendly” is the best you can hope for. If your interests happen to match theirs, they can be a powerful ally, and, in case you ignored Lesson #18 above and don’t have a tiny robot army, you may be able to entice them to fight for you by giving them a box with assorted lengths of wire and some nuts.

#76: always have a miniature EMP device handy

Ever have that problem where your neighbor keeps construction going all the time? Or been at the movies and some jerk doesn’t stop talking on the phone? What about that meeting in which people just won’t stop playing Angry Birds on their iPads?

All of these problems have one solution: a miniature EMP pulse generator. This wonderful device will wipe out all circuits within a reasonable radius, returning your immediate surroundings to something like, say, Victorian-era England. What a time that was, when you could hold a world-wide empire that controlled hundreds of millions of people from a tiny island thousands of miles away from nearly everything and no one really worried about pesky things like human rights, child labor, and such. On the other hand you had to be constantly at war for all sorts of reasons, which really put a cramp on the Queen’s croquet schedule, but hey, nothing’s perfect.

Speaking of croquet: I don’t get it. Cricket, either. And who named these things anyway? Football, basketball: those are sports you can understand just by hearing their name. But croquet? Sounds like a side for breakfast, not a sport.

Anyway, back to the portable EMP. Procuring this device may be slightly tricky, and customizing it properly is not for the faint of heart. The best way to do it is to build it yourself: spend a few years becoming a nuclear physicist and then following that up with mechanical and electrical engineering degrees. With any luck, civilization will still be around by the time you are done.

Once you have your device, be careful how you use it, since it will likely also wipe out some or all of your own devices — not ideal, but in the end a small price to pay for peace and quiet.

not with a bang but a whimper

This is the way the world ends

This is the way the world ends

This is the way the world ends

Not with a bang but a whimper.

–T.S. Eliot, “The Hollow Men”

While apocalyptic fiction stories are a dime a dozen these days (what with the anything-with-zombies craze that’s been building up for a while now…) the genre isn’t new by any means. In western culture, we could trace it back even to Revelations in the bible. We’ve been worrying about our impending doom for a long, long time.

If eschatology isn’t new, perhaps our awareness of its human scale is. In the last half-century the advent of nuclear and biological weapons has changed how we think about the end of the world as we know it: we’ve grown less focused, I’d say, on the extraordinary events that we imagine as prerequisites for such a thing to occur, and more focused on the impact on individuals and everyday life that this would have. Simultaneously with the cold war we got a much clearer picture of how thin the edge of the razor on which humanity’s existence stands really is. Nuclear war, bioweapons, a super flu, an asteroid, Yellowstone, you name it. That is — not only we know the world can end, we also know that there’s multiple sure-fire ways in which it can happen. The question becomes, then, more often than not, what happens to us, shifting the view away from the pyrotechnics to the quiet that remains. A whimper, not a bang.

Two of the earliest, fullest realized examples of this is that I can think of are Richard Matheson’s I Am Legend (1954) and Nevil Shute’s On The Beach (1957). Granted, I Am Legend was ahead of its time in terms of the cause of the end of the world but I’d argue its focus nevertheless is no longer from “how could this happen?” or “why?” but rather “now what?”

More recently there are many others I could mention, but a great example is Cormac McCarthy’s The Road (2006) which had a (surprisingly) fairly faithful adaptation in the 2009 movie. Incidentally, The Road contains one of my favorite passages in any kind of fiction:

What is it? she said. He didnt answer. He went into the bathroom and threw the lightswitch but the power was already gone. A dull rose glow in the windowglass. He dropped to one knee and raised the lever to stop the tub and then turned on both taps as far as they would go. She was standing in the doorway in her nightwear, clutching the jamb, cradling her belly in one hand. What is it? she said. What is happening?

I dont know.

Why are you taking a bath?

I’m not.

The subtlety and quiet power of that moment, all that is said and all that isn’t, get me every time. But I digress (if only slightly).

To this list I now have to add Will McIntosh’s Soft Apocalypse. I won’t spoil the story, and I’d recommend avoiding reviews and just reading it. What I found interesting is that it deals with a fairly complex set of causes, albeit in sketch form — not unlike the “Why are you taking a bath?” paragraph from The Road. It communicates as much in silences, in what it doesn’t say, as in what it does. In the specifics, the only weak point I can remember is that it references the frog-in-boiling-water urban legend, but I’ll even accept that as something a character thinks he knows, but isn’t true.

In any case, it’s a great book. Highly recommended.

the wrong metaphor

“The end of television and the death of the Cable TV bundle.”

Such is the title of this article in the Atlantic published a few days ago. Every time I see one of these articles pop up, I roll my eyes and wonder when we’re going to stop treating markets, technologies, products and services as if they were living things, and therefore as if they could die and disappear from one moment to the next.

Because, the thing is, they don’t.

AOL still has something like three million dial-up subscribers. Three million people paying hundreds of dollars a year! Vinyl is still around. So is radio. Windows, which is quickly becoming less relevant, sells three hundred million copies a year or more. Books have been around for hundreds of years. Even print newspapers, under relentless pressure from digital media, also sell hundreds of millions of copies every day. And on, and on, and on.

Technologies that have reached mass adoption can’t be “killed” by other technologies that may replace them. Very large markets have a momentum of their own, and even as the generation of people that grew up with them passes away, there is usually some level of replacement as some people in the new generation carry it forward. Even products, which are specific instances of a technology or specific solutions to a certain market need, take a long time to die off, and can really only be “killed” by whoever is making them. Even then, many long-time users will hang on to them for as long as they can.

I think we have trouble seeing this accurately because of the multi-generational timespans involved. Not unlike the trouble we have in reacting to long-term, multi-generational challenges like global warming.

A more accurate way of describing this process would be to say that markets, technologies, and products fade away, rather than “reach their end” or “die”. Some fade away very slowly, others a bit faster. If the technology allows it, there can be tipping points were a transition to another dominant technology happens fairly quickly, perhaps in a few years. One example of this would be Facebook vs. MySpace, where the switching costs involved are so low-cost (from the perspective of the user) and so strong, as far as self-reinforcing feedback loops go, that they can happen in a few years. Even so, MySpace is still around, and the situation isn’t as clear-cut as we’d like, with Facebook “killing” MySpace, because if anyone was primarily responsible for MySpace’s fast decline, it was MySpace itself.

This is important because how we talk about something matter as much as what we think of it. What we have to avoid is the wrong metaphor becoming the basis for an inaccurate mental model.

Ok, I get it. A headline that says “TV will fade away faster” is less catchy, and sounds stranger, than “TV is dead.” Mass media doesn’t do nuance and subtlety well. But if we can’t expect it to be gone from the headlines, we should keep in mind that this isn’t how the world really works.

diego’s life lessons

Excerpted from the upcoming book: “Diego’s life lessons: 99 tips for survival, fun, and profit in today’s baffling bric-a-brac world.”

#1: Hide in a cupboard

We start the series with perhaps the most important of all lessons: you should spend most of your life hiding in a cupboard.

The ever-growing focus on “safety” in our society, while laudable in its pervasiveness and intrusiveness, doesn’t go far enough.

Life is full of dangers: sharks, tigers, spiders, sharp-edged furniture, volcanoes, you name it. Consider: just being alive guarantees that at some point you’ll be dead! This is unacceptable. In my own scientific analysis, cupboards are the safest place to be. Here’s a few reasons:

  • Cupboards are cool and dry, which coincidentally match the conditions required by most dried or canned foodstuffs to be appropriately stored. Good nutrition is important.
  • Tigers mostly confine themselves to wildlife areas (zoos, your backyard)
  • Sharks need water and can’t really travel far without it, so you won’t find them more than a few feet away from the bathtub.
  • Spiders don’t like beer. Logic dictates, therefore, that they avoid the kitchen.
  • Volcanoes exist in remote areas with weird names, like Krakatoa or Eyjafjallajökull, which is clearly not near your cupboard. Unless you live in Krakatoa or Eyjafjallajökull, in which case I suggest that first you move to another country, and then hide in your cupboard.

All in all, cupboards are excellent locations to retreat to, whether you want to avoid watching election results, ride out the apocalypse, or disrupt your phone’s cell signal so you can play Angry Birds in peace.

Note: In case of other dangerous situations (e.g., a highway nearby) I also recommend wearing a helmet and kneepads when in the cupboard, just in case.

#20: Own and use regularly at least one Windows PC

Maybe you switched to Mac a long time ago. Maybe you’re truly enlightened and run your own Ubuntu Beowulf Cluster in your basement. Whatever the case, not using Windows regularly is a crucial mistake. Specifically, a 2- or 3-year-old Windows PC running the latest version of Windows. (This ensures Windows will run, but just barely.)

A near-death experience will give you a new appreciation for life. Skydiving with a broken parachute, swimming with sharks in blood-soaked water, fighting a Kraken, all of these are good options for that. But using a Windows PC for a few minutes will achieve the same result with less than half the chance risk of death or injury.

And if you’re a real daredevil, having three or four Windows PCs and attempting to network them is guaranteed to get your heart pounding. True, it’s unlikely you will actually succeed at networking them, but the experience is what really counts. Another option for thrill-seekers is to start using the latest version of Office, sight-unseen, when faced with a non-negotiable deadline. If you do this, make sure to turn on Clippy, aka “Office Assistant”. He will be to you what Wilson the volleyball was to Tom Hanks in Cast Away.

Additionally, the original Minesweeper experience requires Windows, and if you haven’t played Minesweeper on a 200×200 grid, you haven’t really lived.

A corollary to this rule is that you should buy a new Windows PC at least once a year. You will engage in the thrilling process of figuring out whether you should get an AMD Phenom or an Intel Core 2, or find out exactly what the difference is between an nVidia GTX 550Ti, 560, 560Ti, 570, 580, 590, 670, 680, 690, or if you want to go retro and get one of the GT line, or GTS line, or the 4xx line, or even decide that what you really want is one of the many fine  ATI cards. (Like choosing one of the hundreds of types of cereal at a supermarket aisle, choosing video cards in the PC world is a wonderful experience that is guaranteed to keep you entertained for days.) Once you order it and get it 6 to 8 weeks later, turn it on just to experience the blast that is the instantaneous update process, as gigabytes of mandatory updates download and install. Later (much, much later) peruse all the pre-installed software and offers. Sign up for as many offers as possible, including, if possible, AOL Dial up, and then attempt to cancel them. Spend some time talking to technical support, rebooting the computer, unplugging it and replugging it. When you’re done, return it. No need to specify a reason. The people at the return center already expect it. In the process, you will help the economy by keeping the service industry humming along.

#47: Avoid nuclear detonations

An important rule to follow — being far away from nuclear detonations when they occur is a must if you want to keep on commuting, enjoying non-fat decaf soy chai lattes, and generally breathing. You may be familiar with nuclear explosions from that documentary by James Cameron about killer robots that will take over the earth in the near future (the one he did before going to Titanic to find Kate Winslet’s necklace), as well as countless home movies made by the US Army of houses being blown away and generally left a complete mess. The sheer forces of destruction, surface-of-the-sun temperatures, and blinding flash of light (not to mention radiation) are bad enough, but here’s what they don’t usually tell you: nuclear detonations have a side effect called an EMP, which wipe out electrical equipment far beyond the actual blast radius.

That’s right. No TV. No internets, which means no Wikipedia, or videos of animals doing funny things. No phone (for AT&T iPhones, same lack of ability to make calls, however). No blender. No ice. No ice! If there’s a measure of how far civilization has come, it’s the unregulated, unlimited flow of ice in the dwellings of common folk. Without ice, you will lose the ability to produce many common cocktails, and you won’t be able to create any ice sculptures. And who wants to live like that? In a cocktail-less world with no ice sculptures? Seriously.

In short: if you see a very large, very bright mushroom cloud in the distance, board the nearest plane that works and get away from it. Preferably not traveling to Krakatoa or Eyjafjallajökull (see rule #1).

#68: Aliens do not come in peace

Less a “life lesson” than a straight-up fact of the universe, it’s something that should nevertheless be always kept front and center. When you find yourself (as we’re often wont to doing) in a typical Iowa cornfield in the middle of the night, after having run out of gas, and a shiny spaceship lands in front of you, the rule is simple: DO NOT TRUST THE ALIEN.

Here’s a handy guide of how to respond to various first-contact situations:

  • If the alien majestically walks down from his/her/its spaceship, extends their hand/leg/tentacle and says/whispers/grunts “We come in peace”, shoot him/her/it.
  • If the alien is a tiny crab-like thing that wants to attach to your face and has acid for blood, shoot it.
  • If the spaceship looks like a car and the alien looks human, shoot it twice. Especially if they claim not to be an alien. Those are the most dangerous.
  • If the alien has a bizarre mask and dreadlocks, distract him by placing a cardboard cutout of Arnold Schwarzenegger from the movie Commando to your side, then shoot it. Naturally, this requires you carry said cardboard cutout with you at all times, preferably on the passenger seat for easy access.
  • If the alien is some sort of gelatinous blob that would not be affected by shooting, just run. Gelatinous blobs are never fast.

The one exception: when you find the alien in your backyard shed, and it likes Reese’s Pieces. In this case, attempt to confirm it’s peaceful by  verifying the alien is pliable to cross-dressing and wearing Halloween costumes. Then place it in your garage so he can build some intergalactic phone equipment, and start preparing for unnamed government agencies to descend on your property, by, for example, heating up the coffee and getting some donuts. It doesn’t hurt to be polite.

cargo cult troubleshooting

I recently started listening to Hypercritical (fantastic show btw). During last week’s  show (#75 “Just A Dinosaur”) Dan Benjamin & John Siracusa discuss the problem of corrupted binaries on the app store that Marco Arment first brought up. The discussion starts around minute 50 of the podcast I think. In the process of talking about that, they referenced previous download problems from Apple that Dan had and how the feedback he received was a whole host of measures that included disabling packet flooding on your router, port scan detection, among other things, that I call “cargo-cult troubleshooting”.

This is not, btw, not a criticism of Dan or of the people that offered help, but rather an attempt to codify a particular behavior we engage in that is all-too-common in solving problems with complex systems. I’ve seen this before, I’ve done it before (I think we all have at one point or another) and it seems interesting to figure out why we do it.

Why “cargo cult troubleshooting”? Wikipedia has a good article on Cargo cults. Briefly (and avoiding the religious overtones), we could say that Cargo cults attempt to reproduce the observed conditions under which something happened thinking that it’s those conditions, and not an external factor outside of your control, that made it happen.

A walk down troubleshooting lane

To look at why this happens, let’s start with this particular example — let’s go through this specific problem-solving process and then look at some possible root causes.


  • Downloads for software updates from Apple fail repeatedly to validate (and therefore install) from every machine in your house.


What we know with a high degree of certainty is:

  • The files must be either incomplete (ie a broken download) or the download completed but was somehow corrupted, therefore failing the validation against their signature.
  • As a first step, you verify (as Dan did) that downloading from a different geographic location produces no errors. Therefore the problem is location-specific. This rules out a widespread Apple problem.
  • Because you also have multiple machines at home, you can verify that it’s not machine-specific. This rules out problems in one machine, or a failing disk drive.
  • Rebooting router, machines, etc., has no effect, so the problem isn’t related to the state of the machines on your end.
  • It’s Apple-signed-binaries-specific. Other downloads of any other type work fine for you, including other downloads from Apple, such as visiting their website (in essence, a download of HTML, CSS, JS, images and other data, I am assuming that apple.com and other Apple properties work, and this is an important clue). Even more, it seems to be at the very least Mac-specific, so that iOS installs work. I am also assuming, probably correctly, that iOS app install/update was unaffected, even when using the same network. iOS apps are also signed and distributed by Apple using the same infrastructure, so this is another important clue.
  • Searching online shows many other people, in several different locations, reporting apparently the same problem. The “apparently” will be something we’ll come back to later, but for the moment let’s assume it is the same problem.

The easy part of the diagnosis is over. Before starting to fiddle with all settings everywhere, let’s see how much farther we can go in identifying the cause. We’re left with a few components that could be the root cause:

  • Apple’s servers/process related to Mac downloads,
  • Apple’s CDN (probably Akamai)
  • more broadly, the route between your house and the servers
  • your local network
  • your ISP
  • your router

Start with the possibility that the downloaded binary is complete but corrupted/invalid. We know that TCP sockets, which underlie HTTP connections, have error correction built in. A valid TCP connection will deliver the same data at the receiving end that was sent at the sender end, so the file that is arriving at your machine (if complete) will be what the server sent (short of an incredibly sophisticated man-in-the-middle attack). Additionally, since we know that other downloads work, in particular other non-signed-binary downloads from Apple work, the network route end to end is fine, and so is your ISP, at least as far as full downloads are concerned.

So if the file is downloaded fully but still broken it means that the server is sending a full, but corrupted or incorrectly signed file. This is the first possible cause.

Now, as far as the download being broken or incomplete — Is it possible that due to bizarre settings or a bug your router is bailing out (or your ISP blocking traffic) after downloading some amount of data, therefore leaving you with an incomplete file? Unlikely, perhaps, but not impossible. The fact that it only happens with Apple binaries makes it even less likely (a bizarre Apple-specific setting in your router perhaps? ISP rate-limiting?). Similarly the likelihood that this could be a widespread problem and be a characteristic of some modem that somehow only affects Apple’s CDN servers is also low to nonexistent. However, let’s say this is the second possible cause.

The rest of the scenarios involve either Apple’s signing process failing or one of Akamai’s or Apple’s servers involved in the storage having a corrupt image, disk, or software problems that then serves out invalid binaries to a location for a specific time. This is the third possible cause.

So we have whittled it down to three possible causes:

  1. Apple or the CDN is serving a complete but corrupt (or incorrectly signed) binary. In the case of an incorrectly signed binary, this can be only Apple’s, and not the CDN’s, problem
  2. Your ISP or router is consistently interrupting only signed-binaries, and only from Apple (keep in mind we already decided that the possibility that a complete file was corrupted en-route short of an attack taking place was nonexistent because even in the rare case of a bizarre (or even unheard of) router malfunction, the probability that this bug was affecting just apple signed binaries was similar to that of an elephant suddenly levitating due to quantum fluctuations around the elephant)
  3. Apple or the CDN are serving incomplete (and therefore broken) downloads

Let’s look just at the second possible cause vs. the other two as a unit for a moment. Occam’s razor comes to our aid: (paraphrasing) the simplest explanation is most likely the correct one. Is it more likely that some bizarre setting in your router (or ISP) is corrupting binaries only for a particular type of binaries from a particular company in a particular location, or is it more likely that everything works fine (as it does for all other cases) up to the server source, and that it’s Apple or the CDN that is just serving a broken file?

The latter is more likely, and a “simpler” explanation — although it may seem unlikely for reasons I’ll touch on later (the “I broke it” vs “it broke” issue). For the moment, suffice it to say that we automatically assume that Apple would not let this happen, but if you remove that assumption (which is incorrect, again, more later) then things become more clear.

While rate-limiting from your ISP or router, or some other router configuration related exclusively to Apple’s servers is possible, if unlikely, it is pretty much impossible that “normal” content not be affected.

This is a critical point that I mentioned above: None of the reports mention problems navigating to Apple’s site, or downloading any other type of content from Apple (such as trailers, movies, music, etc.). Even more, many if not most of these users are likely to have iPhones, iPads, and iPods, all of which also require signed content downloads, and are served from the same infrastructure, and therefore under the same conditions, as other Apple updates. If all data coming from Apple, including its website, was failing to load, that would be a much simpler (and fundamental) problem, which perhaps could, for example, involve DNS settings.

This leaves us with the first and third options, specifically pointing at Apple and not merely the CDN component. Why? CDN storage being at fault could be a culprit but only if this is a rare, random, and quickly fixed situation. Akamai, and all CDNs, have sophisticated infrastructure that will take out “bad” machines out of rotation quickly. Apple (which as far as I’m aware uses Akamai for many things) no doubt has that type of infrastructure too.

This leaves us with the most probable cause: a repeatable problem that persists for a single location and happens over a span of time, in Apple’s signing process or the file generation/copying that surely follows it. This would point to some bug in custom software on the server side being involved, in which Apple is signing binaries and only randomly corrupting them which ends with a complete file that doesn’t pass signature checks. Given that the problem seems to be limited to some locations consistently, we could also guess that there’s something about those locations by themselves or in interaction with the signing or copying process that is breaking the binaries, perhaps an older part of the infrastructure that is not easily solved because it hasn’t been migrated to new systems for example, or some difference in the environment (network time issues are common) that creates locally valid but globally invalid signatures.

The result of the analysis says that there’s no need to fiddle with settings or call your ISP, since that won’t solve the problem. You can only wait for Apple to fix it (perhaps report it to them) and in the meantime get the binaries from another location, like Dan is doing.

Is this absolutely the right diagnosis? I don’t know, of course. Based on the data I have so far, this seems reasonable, and I do know that if I was faced with this problem, I would either download the software from the location that works, or just sit quietly and fume (yeah, more likely the first option :)). I wouldn’t waste a minute fiddling with router settings. Maybe, if I was feeling somewhat desperate, I would reboot the broadband modem, hoping that by doing that I may get assigned another IP by the ISP and perhaps, maybe get assigned to a slightly different geographic location by Apple where things may be working.

In any case, the specifics of this case aren’t what interests me, what interests me is how solutions that are highly unlikely to affect the true root cause of a problem are accepted, and then spread, online and offline.

Where does cargo-cult troubleshooting come from?

Cargo cult troubleshooting leads to solutions that are closer to “stand on one foot and whistle quietly” than something that actually goes at the root cause of the problem, that is, they don’t actually fix the problem at all. But if so, how do these things get started, and then spread, in the first place?

As for how they get started, the most likely source is variables out of your control. Let’s look at this example. Update fails repeatedly. You start trying to fix it and as long as you keep trying things, the likelihood that (if the problem is in Apple’s end) it will be fixed by them increases significantly. So you do thing #785 and suddenly it works! Only you didn’t fix it. Apple did. Because there’s a giant variable (or more accurately set of variables) that you don’t control on Apple’s side along with all the infrastructure in between, you can never really know what fixed it, especially if you’re trying things for a long enough period of time (say, 1-2 hours at least). Unless you show it is repeatable, which we almost never do. That is: propose that switching feature X breaks Y. Switch X off. Show that Y now works. Switch X on. Show that Y now doesn’t work. Do this three or four times. But that’s not what we usually do. We usually just get something working, are happy that the pain is over, and move on.

There’s an interesting aside to this in terms of why we assume that the problem is on our end first, rather than the other. It’s what I call the “I broke it vs. It’s broken” mindset, of which I’ll say more in another post, but that in essence says that with computer systems we tend to look at ourselves, and what is under out control, as the source of the problem, rather than something else. This is changing slowly in some areas, but in a lot of cases, with software in particular, we don’t blame the software (or in this case, the internet service). We blame ourselves. As opposed to nearly everything else, where we don’t blame ourselves. We say “the car broke down,” not “I broke the car.” We say “The fridge isn’t working properly” as opposed to “I wonder what I did to the fridge that it’s no longer working”. And so on. We tend to think of Google, Apple, and pretty much anyone else as black boxes that function all the time, generally ignoring that these are enormously complex systems run by non-superhuman beings on non-perfect hardware and software. Mistakes are made. Software has bugs. Operational processes get screwed up. That’s how things are, they do the best they can, but nothing’s perfect.

The propagation of a cargo-cult solution

So that’s perhaps a valid theory for how non-solution solutions get started, but then they have to spread. Wouldn’t the fact that hundreds of people are saying in forums “this works” mean that it does? Not necessarily.

First, other people trying to solve the problem are also affected by variables out of their control, and they may experience similar results when trying multiple things in sequence.

Second, the people involved in first trying to identify the solution (let’s call them “Patient Zero”) are usually nerds. Take me, for example. I may have already been tinkering with my equipment, and perhaps in a rare case or two mucking around with, say, the MTU settings or blocking filters leads me to “unbreak” something that I actually broke — but I don’t remember changing. But we tend to forget that most people don’t look at a router settings console in their entire lives. So then I post my “solution” as something to try and someone tries it and it works, it seems to confirm what I said, but either because of external variables, or because of… rebooting.

This is the third way in which “solutions” propagate as valid — just by rebooting. Many if not most of the “solutions” involve rebooting. Rebooting your machine, the router, disconnecting and reconnecting things, reinstalling OSes or firmware. Rebooting/Reinstalling/Powercycling is like the utility knife of Cargo Cult medicines, and one that in many cases in fact works. Low memory, dead sockets hanging out for some reason, and subtle bugs, you name it, there is still a need to reboot devices. So in another small amount of cases, rebooting actually does fix a problem. Myself, as Nerd Patient Zero, know this, and probably was the first thing I tried. But this is not true of everyone, and the least technically sophisticated people are the least likely to just start restarting things for no apparently no reason, because they don’t know that there’s a possible correlation between how long something has been running and possible corruption, misuse in resources that leads to resource starvation, leaks, etc. There’s a reason tech support starts by asking if you have rebooted something. They’re not trying to be obnoxious, they just now that often this is enough to solve state-related problems, and a lot of people don’t think of trying that. The fridge, after all, doesn’t have to be rebooted to be happy, and even the original “Windows Experience” (By which I mean not some fancy Microsoft Marketing term, but “Reboot every day at least, reinstall every 3 months if you want to have a speedy machine”) is not something that normal people remember to do all that often.

The fourth way in which things propagate is through the game of telephone that are Internet forums. A user may think they have a corrupt binary problem but they actually have another problem. Perhaps the download can’t start at all, instead of failing to authenticate. No matter. “This sounds kind of like what I’m seeing”. Even if “kind of like” is not really something that should apply when debugging this type of problem, they don’t know that. They change the setting (and in the process reboot the router, which perhaps was really the problem) and boom, it works! Or — they try a number of things in sequence, then Apple fixes the problem on their side, and presto! In flood the reports of success with the cargo-cult solution.

Finally, and this is perhaps a major reason, we share a strong cultural memory from mechanical and electrical devices in which seemingly ridiculous solutions actually worked. For example, the Apple III was infamously so poorly designed that in some cases when there were issues people were advised to lift the machine an inch or two from the desk and let it fall, which would solve the problem. This was because the action would re-seat the cards, which had been loose. Similarly, in some older TV sets hitting the TV on the side would fix the problem, because of  more “mechanical” reasons, such as loose components, etc.

One of my favorite moments from Armaggeddon is when they are trying to restart the engines of the shuttle go get off the asteroid, and Andropov, the astronaut they picked up at the space station, gets frustrated with the lack of progress, goes down to some kind of engine room where Watts (co-pilot) was frantically and apparently randomly pushing buttons, shoves her to the side and as he shouts “This is how we fix problem in Russian space station!” he  starts banging on some pipes with a wrench. This being a typical Michael Bay movie, the solution works and everyone’s happy ever after. With complex software and hardware systems, however, the equivalent of hitting the equipment with a wrench can’t really solve the problem.

We will only, occasionally, just think it does.

graphs aren’t social

At Ning, we often hear the following question from people outside the company: how do we see the ‘social graph’ from our perspective?

They don’t ask what people enjoy doing on the service. They don’t ask for stories of users, of finding long-lost friends, or of newfound friendships. They don’t ask what makes people on Ning join ten different social networks, as they often do, even if this flies in the face of conventional wisdom about how people are sick of joining social networks.

No. They ask about the social graph.

I am honestly amazed at this question — is that really what they care about? An abstraction? I am amazed, and at the same time I understand where they are coming from.

When I started at what would become Ning some three and a half years ago, I had spent a few years thinking about social networks, ad hoc group formation, and self-organization. I was completely immersed in the terminology with which we try to make sense of phenomena that is sometimes hard to grasp at an intuitive level. Groups of people are not just the sum of the individuals that compose them, and this creates a chasm that is hard to cross–for we, as individuals, project out of ourselves and into the world when we look for answers.

What I hear when I listen to someone talk about the social graph, what I think of when I read about the social graph as if it was a real, tangible thing –in fact the only real and tangible thing–, is a mathematician, talking to someone proudly showing off her new car, pointing at a wheel and asking: “how’s that circumference working out for you?”

The notion of a circle, how it is unequivocally described, what its characteristics are, is undeniably of great value to anyone interested in mathematics or geometry, and even to anyone interested in designing a car or producing a wheel. And it is of no value to that same person when using a car. Not in the mechanics of the action, mind you, in which the device is at work, but in the actual use, and even enjoyment of what the abstraction, applied to something real, can do.
Likewise, the notion of a social graph is interesting to us in nerd-dom as part of our continued understanding of what it is that we’re actually doing, but it holds almost no sway in determining whether something has value for people.

Hundreds of social networks have come and gone. They all had ‘social graphs’ as a key part of their functionality. If this mythical graph is of such importance, why, then, did so many of them fail?

A thousand factors, to be sure, but one of the most important surely must be that they didn’t connect, in some meaningful way, people to each other, and to the service.

I choose words carefully when I say people and not users. Semantics matters. It’s easy to fall into abstractions to handle numbers or notions that seem too big to grasp, but we should resist the temptation. Because it’s when we start designing our software for abstractions, rather than for people, that things go off-course.

Don’t get me wrong — I live and die with abstractions. You will have to take my little diagrams and sourcetrees and formulas and tools from my cold, dead hands. But these are the things I build software with, not for. And that’s how it should be.

We must see past the abstractions and speak to what people really use, and crave, and communicate with. Let’s use the abstractions and obsess over them, but then translate that into something real and meaningful.

Graphs aren’t social. They’re just graphs.

And we should never lose sight of that.

let’s go north!

Northwest Passage now open. On the plus side, you can now travel by sea between Europe and Asia. I sense tourism opportunities. Who said climate change was all bad?


Get every new post delivered to your Inbox.

Join 383 other followers

%d bloggers like this: