☞ Long-term thinking in the world of software
Ephemerality vs. Impact
Here’s a partially-baked Grand Unified Theory of software and society, intended as a provocation. So please bring on the criticisms!
The news is filled with a constant refrain of dysfunction and bad choices and ethical lapses in the realm of tech companies. So, why is the tech world so prone to this? While there are no doubt many causes for these lapses, I’d like to posit one such reason: a mismatch between the goals of tech leaders and the nature of software.
Simply put, technologists want to make a lasting impact on the world, a dent in the universe, if you will. They want to create change, but also a legacy.
But here's the problem: tech, and especially software, is inherently transient. Code, in many ways, is fragile and delicate: it can easily fail or become buggy or be rendered obsolete. An online service rarely operates the way it should even just a few years after it is created. Try to emulate an old website and you will be battered by difficulties. Want to see what an old page on the Internet looks like? The Internet Archive might help a bit, but if the webpage is interactive, it might not even be possible.
Since code and the software that we build is inherently ever-changing, there is a race to make sure it can have a big effect. Because if you are trying to make a difference through software, you end up trapped by the conclusion that your software product must affect as many people as possible as quickly as you can before it becomes obsolete. Therefore, tech companies scale massively, even though they are built on top of this fragile and ephemeral substance. And, as a result, we are left with the warped priorities of tech startups and their cultures. They move too quickly, break things, fail to maintain what they've built, and leave the world strewn with the results of these fast and ill-considered decisions.
While large software projects can evolve over time to address these issues, fundamentally there is a specific problem here: tech suffers from long-term thinking on top of a short-term medium.
So what do we do? Do we just resign ourselves to the fact that short-term thinking—and all the dysfunction that comes along with it—is a necessary feature of the world of software? Or should people abandon the tech world for physical things, like infrastructure, or anything else that might last a bit longer?
Neither of these paths is necessary. Instead what is required is a shift in perspective: recognizing the role and niche of different types of software. Some software can and should exist for the long-term, but in order to do so in a healthy way, it requires a culture of maintenance, rather than simply innovation. For example, there is the group known as The Maintainers, which focuses on sustaining “our human-built world.” There is what Maciej Cegłowski, the creator of Pinboard, has referred to as “artisanal SaaS”: a small and slowly-growing software company, built for the long-term. These might be the kinds of perspectives required to make sure that long-term thinking is embedded in the world of technology and the companies that exist around it.
But other software should be inherently evanescent, and that's okay too. As Alan Perlis, a computer scientist and one of the more aphoristic people who worked with computers in their early days, noted, “Is it possible that software is not like anything else, that it is meant to be discarded: that the whole point is to see it as a soap bubble?” (this might not be the right mental framing, but it is nonetheless clear that software is truly different from other creations and constructions.) Clearly, some code is needed only for a short amount of time (even websites might be best thought this way). Companies devoted to this are just fine, and should be humble in their understanding of their code's lifetime.
Sometimes software should stand the test of time, and we need to change our creative culture to sustain it. But many other times, software can still have an impact, even if it is designed from the outset to be smaller and more transient. And that doesn't make it any less valuable (and there are likely lessons from pace layering that might be relevant). Software and tech don’t have to be big. And when we are able to think both long-term and smaller-scale, we might be able to avoid a lot of the problems of our current tech culture.
I talked about “rethinking the science of science funding” on Lux’s new podcast “Securities.” Please feel free to give it a listen.
A few links worth checking out:
The James Webb Space Telescope and a Quest Every Human Shares: “We stagger upward under the weight of our knowledge of our own mortality. In the face of the ultimate abyss that is destiny, we can find honor and dignity in the fact that we played the cosmic game to win, trying to know and feel as much as we can in the brief centuries allotted to us.”
From the always-fascinating Daystrom Research Institute subreddit, a contrarian take: “The Borg have a PR problem, or, why the Borg are great and we should all join up”
Whole Earth by John Markoff: a fantastic new biography of Stewart Brand.
Until next time.
Perhaps a slightly more optimistic view from a systems perspective.
Computer science is the discipline of breaking down problems into smaller and smaller parts. Each level of abstraction has its own specialty but someone needs to know how to combine these parts. And someone, in turn, combines *those* parts. Turtles all the way up.
Within each level, there are small details and large inter-dependencies. So maybe each level of abstraction needs different kinds of minds (and time frames) to become robust?
This is right on, as is D. Schmudde's comment.
Code suffers from a sort of entropic decay, but not all systems/software decays at the same rate. In the churn of pursuing product/market fit, that decay rate is devastatingly high unless the system is designed for iteration (and then is merely "high").
Before I did much coding myself, I was dismayed at how often developers were "refactoring" – not just old code but code they had written 2-3 months ago! But due to a weird career arch that led to me coding, I now find myself refactoring constantly. Always Be Refactoring! But there is a "right way" to refactor when the rapid iterations just keep on coming. We can distill/generalize down to the right set of primitives with the right affordances and interfaces – legos, in a sense – with which we can build + modify quickly. Moving legos creates far less entropy than moving around the lower-level constructs.