Transitioning into AI safety? Design your own adventure

Or: How to transition into AI safety efficiently, enjoyably, and quickly

This week, I’m helping out with BlueDot Context Week, “[a] 4 day residential program for people who want to understand the AI safety field and find their highest impact path.”

(h/t Jack Douglass for this beautiful cover image)

Some notes from the field:

1. I like Context Week, and Harry has been great to work with! I think it seems similar in spirit to Praxis, though with the following differences:

   1. Context Week doesn’t target a specific field (unlike Praxis, which focused on debaters); and,

   2. The program focuses more on collaborative seminar-type learning / reading sessions, whereas Praxis’s schedule was packed with speaker sessions.

2. In regards to the selection process, I think Context Week might have benefited from considering “yield rejections” / selecting for a specific kind of person. (That is: the kind of person whose career bottleneck is mostly lack of information)

   1. If I had asked the participants to design, from first principles, a thing that would maximally advance their AI safety careers, I find it unlikely that many of them would have converged upon the shape of Context Week. (Some would have! But not all. Some of these people have more context than I do, lol)

      1. For example: someone we’ll call Participant A expressed interest in stress-testing his preconceived notions about what the D.C. policy landscape looks like, and connecting with people in AI policy to determine what he should apply to, what jobs he should consider, and etc.

      2. In Participant A’s case, I think the Context Week selection committee would have done well to: spend most of the interview with Participant A designing a weeklong trip to D.C., and connecting him with people at various D.C. organizations to schedule 1-on-1 chats during that week + ask for invitations to any relevant socials taking place that week. Participant A has already done a very decent amount of reading, and I anticipate he will leave Context Week with the same D.C. uncertainties he came in with – it’s hard to resolve them from Berkeley!

3. This gets to a broader point I’d like to make about career transitions: if you’re looking to transition into the field, you can (and probably should) choose your own adventure. The fellowships are not the only way in! You don’t have to wait for permission from MATS or Pivotal or ERA or Astra to join the field. You can find a side door; you can design your own program.

Here’s how.

What I would do if I were transitioning in today

A roundup of things I wish I’d known, and been agentic enough to do.

Part A: Projects

Photo by Kelsy Gagnebin on Unsplash

The key points:

1. If you’re currently in the field, make a list of projects you think should exist in the world, and send it to anyone you know who is interested in transitioning in.

2. If you’re currently not yet in the field, and want to transition in, do a bunch of projects! Ask friends for suggestions, ask Claude for suggestions, ask yourself what your biggest unanswered questions are, and spend a day or two answering them. Write up what you learned. Make it into a vibe-coded site or artifact. Share it with people in the field and ask them for feedback.

Expanding upon the above:

In my view, the fastest way to both learn about and transition into the field is through doing projects, rather than doing a lot of reading. Of course, some baseline level of reading is helpful, but it’s more likely to stick if you do something with it.

For example:

This biosecurity memo is one of the first things I worked on related to AI safety. I spent a day or so doing a bunch of reading about AI bio, the history of bioterrorism and bioweapons, what the most dangerous biological agents could be, and the state of AI capabilities + arguments regarding trends. That initial project piqued my interest in AI biosecurity as a field, and was a precursor to the much more comprehensive Bioweapon.ai, as well as the inspiration behind my research on open-weight model risk (Uncensorable.ai).

Doing that project was interesting, fun, and likely much more effective than swimming around aimlessly in a bunch of random articles would have been. I had a clear sense of direction: I wanted to resolve the question, “should we be worried about AI biorisk?” And with that in mind, I knew what was relevant to read and what specific information I should look for in the articles I read.

The added benefit of this kind of approach is that you: address your actual uncertainties and learn a lot in a compressed timeframe; your retention is likely better because you applied the information you read about in service of answering a question; and, these artifacts will come in very handy for what I suggest in Part B.

So: What are some unresolved questions you have about the field? Are you confused about why people are saying AGI might be coming in 2030 (timelines)? Are you confused about the current frontier of mech interp? How time-horizon evals are designed, by which orgs in the space, and how they could be made better?

Spend a day (or even just a couple of hours) resolving your uncertainties! Then do a write-up, with clear reasoning transparency. Then:

Part B: The Side Doors

Photo by Mick Haupt on Unsplash

Consider skipping the fellowships!¹ They’re super competitive, you’re typically forced to choose from the research agendas of the mentors they happen to recruit (rather than the thing that would be maximally useful / impactful, since people have different views and bets on this²), applications are kind of hard and boring, and being rejected isn’t pleasant.

Also, you shouldn’t wait for the next cycle of [insert program] to start getting involved, nor should you treat a fellowship acceptance as the permission you need to enter the field. You do not need to wait on permission – and, importantly, you do not have all the time in the world.

So, if not the fellowships, what should you do instead?

After you’ve done your projects, sit down for 20-30 minutes and answer the question:

If I were to design a thing that would maximally accelerate my transition / help me join the field, what would it look like?

Things worth doing could include:

• A trip to SF/DC/Berkeley/London/etc to chat with people from the orgs you’re curious about. (Strong recommend booking the calls before you book the flight! Also ask about social events they’d recommend you go to.)

  • For this: Apply to BlueDot rapid grants.

• A visit to a hub office. For example, you could consider visiting Constellation, LISA, or Mox! Coworking with AI safetyists for a couple of weeks could teach you more about the field than a month of reading. (Note: All these hubs have their own rules about guests.)

  • Again, I recommend applying to BlueDot rapid grants to cover some of the costs of this, e.g. flights.

  • LISA says: “If you're looking to visit temporarily as a guest or to a trial a workstation, please contact us at info@safeai.org.uk.”

  • Mox has day passes, and if you know a Mox member, you can also visit for free as their guest!

  • For Constellation, you can similarly visit for free as the guest of a member. Ask around for intros :)

• 2–3 months of independent work. Consider applying to: Coefficient Giving, BlueDot, and the Long-Term Future Fund, which all fund early-career people doing the kinds of projects described in Part A.

  • Your write-ups would be very useful artifacts to include in your grant application, and you can write about what you’re looking to explore next and why this funding would be useful to you / enable you to continue your work.

  • (BlueDot Career Transition Grants are different from Rapid Grants, which are capped at $10k.)

• Cold-emailing 10 researchers whose work you engaged with, attaching your project, and asking at least one specific question (and/or asking for guidance on what to do next).

  • Also: Consider publishing your write-ups on EA Forum and/or sharing them as comment-enabled Google Docs with the above 10 researchers. (Commenters will give you other things to consider, point out where your analysis might be wrong, etc.)

You might have noticed that a couple of the things above (grant applications, cold emails) are in bold! Why is that?

If I were to transition in today, I’d personally do roughly the following:

• Figure out which org(s) I want to work at full-time, and why.

  • (It would probably be good to make a list of top 3-5 orgs.)

• Apply for a career transition grant.

• Send a bunch of cold emails to said org(s) with my projects from Part A, and ask to hop on calls with people from the org(s).

• Ask for an unpaid internship³ / ask someone on call to advise me while I work on a project they think is important (would also probably solicit ideas on call!).

  • Some orgs will say no to official unpaid positions for legal/capacity reasons – maybe ask for an advised project first.

• After working on the project: send it, ask for feedback, ask for career guidance, ask if there are any open positions at the org I might be able to apply for / if they’d consider me joining.

(I might actually do this when I’m in full-time job-search mode, which will very likely be soon.)

Some final notes on fellowships (because they can genuinely be very useful):

• Fellowships do provide useful feedback loops and mentorship for people who can’t yet self-evaluate; “just do projects” assumes some amount of taste. So, if this is you, and you’re not comfortable with choosing your own research directions, you can either apply to a fellowship or ask people you know for project ideas.

  • (I got the bio memo project idea from a friend in the field, actually!)

  • I also hope more AI safetyists will publish their project ideas. (I’ve got a forthcoming post on this soon.)

• There are other benefits of fellowships, e.g. connections with other people ~new to the field.

  • I think these can be replicated in other ways, but I do want to be clear that there are many valid reasons to choose a fellowship over a more unstructured path. Again, you know you best! But just be clear with yourself about why you’re opting for the fellowship path, and make sure you’re being intentional about it rather than just doing the most obvious thing because you don’t think there are other options available to you.

Your option set is infinite; pick something, and start this week.

Now go save the world!

1. There are obviously some caveats to this advice; you know your own individual situation best, and indeed, there are various reasons why I chose to apply to a fellowship earlier this year rather than pursuing my own thing. But I think your option set is bigger than you think. Keep reading for a more in-depth explanation / what I think you should consider instead.

2. By all means, if there is a mentor you think is working on something that you specifically have a strong interest in working on, do apply to the fellowships. The point I’m trying to make here is that you don’t need to be boxed in to the fellowship options. You have a lot of options; choose wisely.

3. (Funded by a career transition grant, hopefully)