September 13, 2010

The Decentralization Epidemic

In a previous entry penned some three years ago, I discussed paradigm shifts (in molecular biology and elsewhere) and more generally our inability to properly comprehend contemporary events for which we are present with the same analytical rigor we are able to apply to the past. (Schrödinger's Cat was also mentioned, but that's mostly because quantum mechanics is awesome.) In this post, I will cover a new paradigm shift of sorts that seems to be all the rage in fields as disparate as politics and computer science: decentralization.

At the outset, two articles appearing in the August 14th issue of The Economist must be considered. (And yes, I am a couple weeks behind due to the twin malaises of exhausting travel and overwhelming laziness, but bear with me.) The first is an article on latest developments in artificial intelligence and the second is one on the transformation of Britain underway since the Conservative–Liberal Democrat coalition took power. So, what does artificial intelligence based on hive algorithms and the devolution of power from 10 Downing St have in common? Everything, really. Allow me to explain.

Centralization's Heyday: 1900–1980

Before we can get to decentralization, however, we must pause first on what led up to it, namely, centralization. The history of the 20th century, in many ways, serves as a time line of this phenomenon and its unraveling. By the turn of the (last) century, power in western Europe had been consolidated among a few countries; and one assassination, two wars, and much bloodletting later, the trend would only exacerbate, of which the catastrophic centrally planned economy of the Soviet Union was only the most extreme example. Almost all western governments experienced centralization to a great extent in the post–World War II era.

The history of computers, meanwhile, has no era of consolidation of which to speak; computers, unlike humans or nation-states, were born centralized. Mainframes, time-sharing, and punch cards, though seemingly eons ago on a timescale defined by Moore's Law, were a way of life even as late as the time of my (much regretted) appearance on this planet.

Crumbing of the Old Regime: 1980–2000

Centralization would begin to face challenges towards the end of the millennium. Thatcherism would take hold across the pond and Reagan would ride a wave of small government rhetoric to victory twice (even while massively expanding spending, but that's a story for a different site). By 1989, the Berlin Wall was coming down as the Soviet Union disintegrated and in 1996, Clinton would eventually famously declare that the era of big government was over.

On the computational side, the advent of the personal computer marked a shot across the bow on centralization's hegemony. By the end of the century, small machines sitting in home offices and packing quite the computational punch were traversing the Internet—itself a testament to decentralization, composed of a massive array of individual nodes.

And yet, what happened in these two decades wasn't a paradigm shift. It was a reduction in centralization that had most likely gone awry, certainly in the computational realm and probably also in the governmental one. It wasn't, however, a recasting of ideas. Even the Internet, seemingly the epitome of decentralization and a product of this time, relies on thirteen root nameservers situated around the world to work. And while there is caching, that's not really decentralization so much as it is optimization.

The Advent of Decentralization: 2001–present

What would come in the new millennium, however, would be a radical reshaping of approaches, not merely reworking of problems along the same axes. Some of this, no doubt, was a consequence of a change in the geopolitical and computational landscape. With the problem of rogue nations isolated to a few spots (e.g., North Korea, Somalia) and state-sponsored terror of the 1970s and 80s giving way to stateless actors led by fanatical clowns hiding out in caves, mutually assured destruction could no longer be a viable theory of operation. Similarly, the explosion of mobile platforms has made decentralization both viable and commercially attractive in ways it was not before.

To deal with these problems, the old approaches guided by centralization are simply untenable. Whether it is mimicking knowledge acquisition by reconceptualizing intelligence as a hive mind instead of a human one, or pushing more functions from London to localities rather than cutting budgets while retaining the same distribution of power, these are fundamental shifts in approach, not technique.

And those are only two examples; countless others exist, especially in the computational arena. AJAX, for instance, pushes more computation onto the client to deal with, among other things, network latency. Similarly, BitTorrent, although not totally decentralized, allows content to be downloaded directly from a number of hosts rather than following the traditional server-client model. And this is not to even mention technologies such as OpenID, OAuth, and ant colony optimization (which could provide for much more efficient packet routing).

* * *

So, what's the point of all this rambling, you might ask (and rightfully so, considering it's largely nonsensical)? Well, mostly it is just to observe this fact: these times, they are a-changing.

But less facetiously, it is to note this: much of the radical agenda stemming from the British government today is one of realignment, not reduction, fundamental change, not superficial alteration. On the other hand, while some of that sentiment may now be fomenting in the States within the tea parties (if the lunacy and racism can even be sufficiently disaggregated to uncover a coherent message), the mainstream political machine seems to be moving in the opposite direction—more centralization, that is. Each of the major pieces of legislation, from the health care bill to financial reform bill to the credit card legislation, envisions a larger, not smaller, role for the federal government.

We could, I suppose, argue about the merits of each of these reforms passed in the past congressional term, but to do so is likely an exercise in futility more pronounced than life itself. No one even knows everything that is in those monstrously dense bills, let alone how it will all operate once implemented. Anyone claiming an educated opinion, therefore, is certainly a fool and most likely also a liar. But that collective ignorance doesn't have to stop us from considering these developments in light of the broader theme of decentralization that seems to have happened upon us in the last decade. Is more centralization what we need now? Or is it more decentralization?

I frankly have no idea. But whichever way we go, what we do need is a paradigm shift—some new ideas for old problems. The rebranding of tired old solutions to pesky old problems has become the mainstay of both major parties in the States; and the prospect of any of these attempts succeeding looks particularly grim. The problems confronting us today cannot be solved without fundamental reconceptualization, much as AI was a fool's errand until swarm algorithms. So, will someone please get on this? Paradigm shifts don't come easy. Thanks in advance.