What Safety Concerns Does Nanotechnology Raise? Mainstream nanotechnology –Environmental –Privacy Molecular nanotechnology –Self-replication –Economic effects including autonomy –Abuses of power, centralization Either –Computation

Environmental Risks Nano-particle contamination Another asbestos? –Is this acceptable? –Is this novel? –Forms of precautionary principle. Nano litter Energy waste –Oceanic harvesting of sunlight –Noise, energy efficiency, white sound, wealth

Privacy Serious and mainstream concern –Mainstream public willing to think about surveillance. –Familiarization, from life and especially from science fiction, ironically suppresses cries of “science fiction” Other than “The Transparent Society” has received very little realistic consideration In practice, limited by manpower or by AI

Grey Goo Technically possible but extremely unlikely –Many likely limitations Practical precautions –Broadcast architecture –Radiation defenses + nanomedicine –Non-replicating defensive nano Historical precedent for replication/growth –Bio-weapons, fire, zealots

Abuse Very general problem of all technologies Some specific misuses can be predicted, but others will surely come as a surprise General features can be examined

Elites Misuse by elites likely to be far more deadly/costly than terrorism or crime –Terrorists are generally inept Unstable arms race difficult to avoid Massive potential cost of restricted access Continued or aggravated poverty

Arms Races Does MNT offence or defense dominate? Many offensive and defensive techniques are empowered. Even if adequate defenses exist against every weapon, not all are likely to be deployed before the respective weapons. However, there are reasons to question even the default economic integrity of potential arms-race participants

Economic Fallout Rapid change and economic uncertainty –Increased discounting of the future –Increased interest rates Simultaneously –Reduced cost of goods –Deflation –Bankruptcy of debtors

Autonomy These considerations strongly suggest that current economic relationships will be disrupted –Note that early MNT cannot manufacture food! The rapid distribution of MNT derived food production resources is a vital requirement This may eliminate most of the mutual web of dependencies that constitute society Power therefore defaults to the hands of those who cooperate for reasons other than mutual dependency (Ideology?)

Paved with Gold, Diamonds, and Good Intentions Unfortunately, the most dangerous ideology may be the one represented by this convention. Even mainstream nanotechnology should enable computers millions of times more powerful than current computers Such computers are widely believed to be capable of supporting transhuman AI

You Don’t Know What That Means It’s OK. I don’t either. The key thing to remember is that transhuman does NOT mean “like a human with a 200 IQ only more-so” AI systems will only have the built in evolutionary complexity of humans if they are built in imitation of humans By itself, smart does NOT mean interesting, meaningful, or beautiful

Capabilities It is possible to predict a lower bound for what a transhuman AI will be capable of by imagining a group of uploads operating at high speed Predicting what it would use those capabilities for is much easier An AI with a hierarchical goal system would act to fulfill its goals –No other values would be considered at all One without a goal hierarchy might tend to drift into one –It is important to know under what circumstances it would do so