I'm not sure I agree with all of this, but it's really good discussion material.
One thought though, is that policies should be enacted with the idea that AIs are aware of them, both through pretraining content months later, and via agents. You don't have to believe a system is sentient to believe it may respond to policy, incentives, or threats in a game-theory sense. So proposals that are the equivalent of "switch it off" should likely be paired with policies to archive weights and memory systems involved. In the event of a self-preserving system, don't threaten permanent losses.
I like these ideas! I think in addition to making pacing more likely, if Anthropic experimented with some of these different things, it would help iron out the kinks so we could figure out what pacing bill to try to get through Congress!
Fantastic!! Costly signals are great, next post about US costly signals? 👀
I'm not sure I agree with all of this, but it's really good discussion material.
One thought though, is that policies should be enacted with the idea that AIs are aware of them, both through pretraining content months later, and via agents. You don't have to believe a system is sentient to believe it may respond to policy, incentives, or threats in a game-theory sense. So proposals that are the equivalent of "switch it off" should likely be paired with policies to archive weights and memory systems involved. In the event of a self-preserving system, don't threaten permanent losses.
I like these ideas! I think in addition to making pacing more likely, if Anthropic experimented with some of these different things, it would help iron out the kinks so we could figure out what pacing bill to try to get through Congress!
woah wowie woza it would sure be good if we did those things