This is an amazing article, IMHO, required reading for any technologist. Just one of the many excerpts I found:
“But now, with the prospect of human-level computing power in about 30 years, a new idea suggests itself: that I may be working to create tools which will enable the construction of the technology that may replace our species. How do I feel about this? Very uncomfortable. Having struggled my entire career to build reliable software systems, it seems to me more than likely that this future will not work out as well as some people may imagine. My personal experience suggests we tend to overestimate our design abilities.”
And another: “These possibilities are all thus either undesirable or unachievable or both. The only realistic alternative I see is relinquishment: to limit development of the technologies that are too dangerous, by limiting our pursuit of certain kinds of knowledge.”
It is interesting that this view mirrors some of the views of commentators on the left like Clive Hamilton and George Monbiot in the sense that the proposal is to limit growth or “progress” as a means unto itself, and instead re-evaluate our priorities and determine a clearer direction forward.
Often such perspectives are written off as being “anit-progress” or “Luddite” views. This is inaccurate IMO. These views are informed by the information we have about how technologies and societies are developing today projected forward, and on our poor history at dealing with some issues. Ignoring these warning signs is to our folly.
I, at present, don’t share Joy’s hope. As much as it pains me to say that. Without a drastic alteration of the current status quo, as I see it the current power-driven, so-called “free-market” neo-liberal doctrine, these forces will be unleashed sooner rather than later. Who’s going to stop them?
I am not a revolutionary, but I do recognise that without true accountability to the people to whom their decisions affect, corporations do not have a conscience. They do not share the concern that Joy expresses. And I personally doubt that the boardrooms are suddenly going to be filled with people that will take these concerns to heart.
The past 100 years, and particularly the last 30, we have seen power concentrate in the hands of an elite and powerful few. Technology is only increasing this power because it is being used to serve these interests, often at the expense of those without the means to enable technology to their benefit. Investment banks can afford to pay their traders hundreds of thousands of dollars a year and still make profit orders of magnitude larger. This whole exercise is driven by technology and how it facilitates financial market transactions. Governments are stepping aside and allowing corporations to lead the agenda.
IMO, we need to see a significant shift in the current direction really soon, else it will be too late. As Joy puts it “We are being propelled into this new century with no plan, no control, no brakes. Have we already gone too far down the path to alter course? I don’t believe so, but we aren’t trying yet, and the last chance to assert control – the fail-safe point – is rapidly approaching.”
I do believe that the massive change required is possible, I just don’t see the required changes actually coming to be within the current power structure. But I do agree we need to try to effect change – a possibility will only become a reality if we try and attain it. And as such I do agree with Joy that the dialog has to broaden and rapidly advance, and cross the divide into the body politic.