Discussion about this post

User's avatar
Feifan's avatar

Fantastic!! Costly signals are great, next post about US costly signals? 👀

BBZ's avatar

I'm not sure I agree with all of this, but it's really good discussion material.

One thought though, is that policies should be enacted with the idea that AIs are aware of them, both through pretraining content months later, and via agents. You don't have to believe a system is sentient to believe it may respond to policy, incentives, or threats in a game-theory sense. So proposals that are the equivalent of "switch it off" should likely be paired with policies to archive weights and memory systems involved. In the event of a self-preserving system, don't threaten permanent losses.

2 more comments...

Ready for more?