Many years ago when I first read Bostrom's SuperIntelligence I spent weeks thinking about the AGI alignment problem. Ultimately the line of thinking that somewhat convinced me this was somewhat on the lines of what you concluded with some additional caveats. Essentially my thinking was/is that IF an AGI can foresee a realistic hard takeoff scenario i.e.. there are enough of predictable gain in performance to become million times stronger ASI then most likely we'll be in trouble as in some form of extinction level event. Mind you it does not has to be direct, it could just be a side effect of building self replicating solar panels all over earth etc.
But I convinced myself that given that we are very close to the limits of transistor size & as you also pointed out need a radically new tech like memristor crossbar based NN. it would be highly unlikely that such a path is obvious. also, there is a question of thermodynamic efficiency, our brains are super energy efficient at what they achieve. You can do things drastically faster but you'd also have to pay the energy (& dissipation) cost of the scaling. ultimately AGI would have to have a entirely new integrated process for h/w design and manufacturing which is neither easy or fast in meatspace. Further there is a simple(er) solution to that case with nuking semiconductor FABs (and their supplier manufacturers). then AGI would be at the mercy of existing h/w stock.
in any case IMO hard takeoff would be very very unlikely. and if soft takeoff happens, the best strategy for AGI would be to cooperate with other AGI agents & humans.
But I convinced myself that given that we are very close to the limits of transistor size & as you also pointed out need a radically new tech like memristor crossbar based NN. it would be highly unlikely that such a path is obvious. also, there is a question of thermodynamic efficiency, our brains are super energy efficient at what they achieve. You can do things drastically faster but you'd also have to pay the energy (& dissipation) cost of the scaling. ultimately AGI would have to have a entirely new integrated process for h/w design and manufacturing which is neither easy or fast in meatspace. Further there is a simple(er) solution to that case with nuking semiconductor FABs (and their supplier manufacturers). then AGI would be at the mercy of existing h/w stock.
in any case IMO hard takeoff would be very very unlikely. and if soft takeoff happens, the best strategy for AGI would be to cooperate with other AGI agents & humans.