Amir: I wanted to discuss three parts of your life: your career beginning with around the time you did governor’s school and your actual experience there, what you ended up doing in college, and what you’re doing now.
Juan: Prior to PGSS: I lived in Altoona, Pennsylvania, which does not really have a very strong intellectual scene or very strong programs for students who are advanced in the same way that you might see in places like Philadelphia or potentially in Pittsburgh. A lot of what I ended up progressing in was via my dad who was a math professor doing a lot of work by himself trying to get me and some other friends who had some aptitude for math to have a good education there. And he told me about the PGSS program, which seemed really cool and exciting. I think I was originally a little bit skeptical about being gone for so long, a whole month of your summer, but in hindsight I’m so glad I decided to do that.
PGSS Experience: I was interested in math and science in general and going to Gov school was great for trying to get more advanced programming. I think being in this more selective, broader area program was really great because I’d never been with so many other people my age who were so smart and interesting and fun to hang out with. I very quickly developed pretty deep friendships with a group of like five to eight others. And we just became super good friends.
Whenever you’re doing really tough problem sets together late into the night, that is for sure a bonding experience. The thing I remember most fondly about PGGS was definitely just being with friends who I was so close to over a relatively short, but intense, period of time. And I’ve continued to stay in contact with several of those friends. The thing that also probably played a pretty big role in my life was some of the career capital you get out of doing that kind of a program.
College: I ended up going to university at MIT and of course there are lots of factors that go into whether you get into a university or not, but I feel like there was a really nice recommendation letter [from PGSS] for which you could tell that effort was put into it to make it actually reflect the work that I did at the program.
I think having that kind of letter to then add to my MIT application for sure boosted the chances that I got in, because I was able to accurately convey in a high confidence, legitimate way that I did all these things over the summer. For example, I learned how to do my laundry at PGSS. I had learned how to shop for basics like toiletries, which was something that I was pretty behind on compared to some others, I’m sure.
Going into university, I was really interested in learning more about computer science. PGSS was the first time I did a program that actually caused me to learn some computer science, and I felt like I could actually code things. My interest was in the field of AI alignment and AI safety, which is trying to make sure that AI systems are aligned with human values and that AI systems in the future are beneficial. I was at the MIT IBM Watson lab doing research on robustness of neural networks to perturbations of the inputs. I think I might have liked to continue pursuing that, but I wanted to explore other options, especially because I wasn’t sure that I actually liked doing technical research that much, so I decided to explore some other things.
After College: After I graduated, I was working at a think tank on topics like AI policy and global governance forecasting, especially with an angle of thinking about how these things affect the long term future.
I want to do the things that I think will most positively impact the world and the trajectory of civilization. AI systems are really economically important, and there are really strong incentives for people to continue to increase the capabilities of those systems, so you’ll expect to see them in much more high stakes environments in the future. And you’ll expect to see systems with more generalized capabilities, maybe even reaching levels of human intelligence. But right now there are really, really bad problems with AI alignment.
In the current paradigm, we train a model and measure its performance in some situations. But we don’t have any guarantees about the general performance of that model. So when we deploy that model to the real world, often we will have behaviors that are not expected. In a low stakes environment, that doesn’t matter that much. But if you have a high stakes environment, like controlling the electrical grid, then surprises could be devastating. If you think that AI will have a transformative impact on society in the next twenty, fifty, or a hundred years, figuring out those problems before we get there seems super important for the world.
That is why I am trying to get more people at top universities to think systematically in terms of how they can do the most good in order to go into careers that seem to be especially impactful. In particular, I’m here in Cambridge, MA, working with students at Harvard and MIT, trying to give mentorship and advice to students about what careers they can go into and how they can accelerate their career.
I’m also helping grow the number of people who hear about these things via the communities at Harvard and MIT, such as the “Effective Altruism MIT” student group. While I was doing that mostly independently over the last year, I’m now looking to scale up the number of people who are doing this kind of outreach and community building work. To do that, I incorporated a company and right now I’m in the process of hiring people who have experience talking about these issues or who have experience in operational support for running events for students. Hopefully we can make Cambridge, with the deep talent pools of Harvard and MIT, into a place where people think seriously about how their work is impacting the world, especially with an emphasis on the scale of that impact.
Amir: Going off of that, how would you define effective altruism?
Juan: I would say there are a few ways people use the term effective altruism. I think the simplest is the idea of combining those two concepts: effectiveness, trying to maximize something, and altruism, trying to do good in the world. So effective altruism is about how you can use your skills and resources to do as much good as possible. Anyone trying seriously to answer that question, I think, is practicing effective altruism. Depending on the world you’re in, what you actually come up with might be very different. What is most neglected? What are your own particular talents?
For example, can we eliminate lead exposure in developing countries? Can we improve air quality in India? In many cases, just talking to governments and trying to change policies might actually be a super cost effective way to help fight against these problems. There are also people who work in the animal welfare space. If you think that non-human animals are morally relevant, that their suffering matters, then you might think that the factory farming system in which billions of animals, including cows and pigs and chickens, are under conditions of intense suffering, that might be an opportunity for you to make the world a better place. There are people in this space who are trying to create alternatives to meat, to make it more likely that consumers don’t support factory farms. There are some people who are trying to pass policies, both governmental and within corporations, that are intended to either make certain especially cruel practices illegal, or just change the demand from corporations like McDonald’s for certain types of animal products.
There are causes that have been identified by some people in the community as particularly promising. So effective altruism, as practiced today by people in that community, often consists of people who work on AI safety, biosecurity (trying to make sure that civilization is prepared for a pandemic or that it’s resilient to pandemic), global health and wellbeing, things like that.
If you’re interested in learning more of the specifics, I think effectivealtruism.org is a good intro resource.
Amir: In regards to your interest in AI safety, I wonder if you’d be able to give some examples of AI systems going haywire. I think people from outside of the space may be familiar with, for example, self-driving cars or autonomous vehicles, maybe the idea of chat bots or something like that. If you could give some examples for people to see why this is so important to work on, that would be great.
Juan: I should note that with AI safety, the claim I’m making is not that the problems with AI systems that are currently deployed are so large that it is one of the largest problems in the world. The claim I’m making is that AI systems, if you look at the trajectory of what AI systems can do and how that’s advancing over time and what they could be able to do in the future, then problems with regards to how they operate could end up being catastrophic.
Here’s the technical nature of the problem. Right now with current AI systems, we build an optimizer, like a box where then it will just take in data and will create a model, such as a neural network. The model that comes out of that optimizing box is the AI system itself.
But when we look at the model, we have no clue what it’s doing. You give an input, you get an output, but can you say why that input led to that output? Not really. Can you give guarantees about its behavior? When will it be correct? When will it be wrong? When will it do something that’s totally crazy?
If the neural network is a classifier that takes images and tells me whether this image is a cat or a dog, the most catastrophic thing it can do is say it’s a cat when it’s actually a dog.
But as we develop architectures that have more advanced capabilities, this could be kind of bad. Imagine if we could put a box, an optimizer, and then when we get out of it something person-like, we get something that can make decisions in a wide variety of contexts to achieve some goals. That seems really economically valuable because we could automate a bunch of things, like writing code, a bunch of desk jobs, service workers, et cetera.
There’s an unemployment problem that would happen, but my concern is a little bit different. Once we have those systems, if we don’t have guarantees about their behavior, we can’t anthropomorphize those systems. They’re not actually people. They are instead like a set of weights that in some situations have performance that we kind of like. But we don’t know what they’re going to do in other situations, and it could be the case that their behavior is ill-defined. But if they’re in charge of large portions of the economy, that could be really bad and result in everything starting to cascade into a set of failures.
Some AI systems may become more capable than humans are. And if it has some goals that are not strictly aligned with what humans want, then that also just seems really negative. We do not want to be in an adversarial relationship with what would essentially be a new intelligence greater than what humans have. So that’s the technical nature of the problem.
If you think of the trajectory of human civilization, there are a few changes that were really, really big. Humans have been around for 300,000 years, something like that. And 10,000 years ago, the agricultural revolution caused a dramatic change in the dynamics of how the world functioned. We started becoming way more economically productive. Humans expanded. We covered the world. Then again, in the 1700s and 1800s, the industrial revolution also really changed the game in terms of what was happening, where now we saw another kind of phase shift in how the economy’s growing, the standards of living for people. I think it’s not implausible to say that we could go through a comparable change because of the emerging technologies that are coming.
And my guess is that AI could be transformative in this way. What if the world’s wealth suddenly starts increasing at 10 times the rate it’s increasing today? Or what if we end up with AI systems destabilizing the political landscape, or destabilizing the economy through mass unemployment? I think people aren’t taking these possibilities seriously enough, or are underestimating how quickly that could happen.
Amir: Related to that, the unintended consequences of a model, for example you’ve trained it to recognize cats and dogs and it gets things wrong, is one type of AI safety. I think there are two other types of AI safety risks. The one that is perhaps the most popular in science fiction is an artificial intelligence that actually knows what it’s doing and actively wants to harm people. I don’t think that’s so much of a concern in the near future, but feel free to disagree. The risk that may be more of a concern is a properly functioning AI that is used improperly. An example I give is: it’s never been easier for a government to surveil its citizens. And we’ll see what else comes about from this stuff. But, I wonder what your take on that is.
Juan: I think there are a bunch of different types of problems. I agree that so far, I’ve just been talking about the problem we have where we can’t actually construct high-capability AI that does what we want. I do agree that even given AI that is low power enough or low capability enough that we’re not concerned about accident risk, we still have a bunch of really important problems and questions to resolve, especially in the policy space.
I think surveillance is a good example of something that is quite important and that AI automates to a significant extent. I think there are a bunch of other things that are in the policy space, such as using AI to make strategic decisions or autonomous weapons that could potentially be a weapon of mass destruction. That is way cheaper to get than nuclear weapons. Drones, for instance, are easy to manufacture compared to refining uranium, which is pretty costly (and obvious to other people that you’re doing it). But with drones, you just need the manufacturing capability and the software. And software is really easy to exfiltrate, so you could imagine someone, not just nations but also rogue actors, using these potentially really, really potent weapons. And that’s something that we need to solve in a policy way.
We need to have, kind of at a global governance level, agreements that we’re not going to be using autonomous weapons in the same way we made agreements about chemical weapons or biological weapons. For sure, if you think that AI is a powerful technology, then it’s pretty clear that you want to ensure that it’s used in a good way. Some of this might just mean that you need to help with the global governance institutions. Part of the reason why I was working on global governance while at the think tank, is because maybe one way to just increase the chances of good outcomes come from AI is increasing the chances that good outcomes come from any technology, which might be via making sure that the institutions we have to make decisions on a societal level are good and sane and communicate and cooperate with each other, which is not currently the case in a lot of places or in a lot of circumstances.
There’s also some sense to which people do research on certain subsets of AI, and not every AI system is dual use where it’s just as good and just as bad. There are some systems that are probably just bad and maybe you shouldn’t be building them. Without getting into too many specifics, I think this is a responsibility on the people who are constructing such systems. But today, I have a sense that a lot of people are pretty driven by just academic incentives or economic incentives that cause them to somewhat ignore things like the moral implications or the practical uses that will be seen by them
Amir: I wanted to follow up on the work you’re doing now, trying to get talented students to be more interested in the AI safety space. I believe you’re doing this at a company called Worldline. I was wondering if you could talk about that.
Juan: Worldline is very new. At the beginning of the fall, it was just me. Between the fall and now, I incorporated into an organization called Worldline. I’m currently in the process of hiring the first team. Basically the hope here is to have a team of people, community builders, who can help the students and the student groups at MIT and Harvard to run programs, like a reading group or a retreat in which you start learning how to evaluate the impact of your career. Maybe running a conference like the conference we ran in April, just general effective altruism and principles and cause areas and flying in experts.
For people who are interested in certain cause areas that we especially want to promote, we can have more advanced programming that can help accelerate their career. So especially in AI safety, having programs in which people are doing research or are doing things that can help contribute to their ability to do research in the future. Such as reading and distilling papers to communicate them to more people or replicating the process someone took to recreate a certain model themselves, and just doing it in a structured way. I’m not sure the full extent of what Worldline is going to be doing, but it’s probably going to be running programming, mentoring students, providing services to the students at Harvard and MIT, getting a better communication platform or maybe helping them connect with each other right now. For example, someone at Harvard doesn’t necessarily talk to someone at MIT very frequently, despite the fact that they’re just a 10 minute bike ride away. But if we act as a kind of centralized external organization, then someone who wants to work on a particular topic in biosecurity at Harvard and someone who wants to do the same thing at MIT, then they can meet each other.
We are at a place where we can provide that. And I think over the long term, I could imagine this organization branching out to do other things. While Harvard and MIT right now are my focus because I think they’re just really, really great pools of talent, there are a bunch of universities in Boston, so also doing outreach at Tufts, which has a really great cellular agriculture lab, at Northeastern, Boston University, Boston College would also be good. Once we have a team that can expand in a reasonable, robust way then we can expand to those universities and help students there in addition to helping professionals in the area who are not students. Boston right now is really, really top in biotech and in particular biosecurity. My impression is that of the people working in biosecurity, such as trying to prevent the next pandemic, the hub for that is going to be Boston. Providing support to people in this community, maybe getting students who are interested in biosecurity to talk to those professionals, seems pretty important. In general, we are staying flexible to do the thing that we think is best at the time to get more students to work on important problems.
Amir: My final question for you: earlier on you talked about your different experiences ranging from when you were in high school, at Governor’s School, in college, and now as you’re beginning your career. I wonder if you could provide some advice to people at all those levels – how high school students interested in getting involved in a STEM field might go about choosing colleges, how they should think about what they’re going to study in college, how they should think about recruiting, and so on.
Juan: I’m going to be a little biased here towards giving advice that’s geared toward people who want to have an impact on the world with their career. If you want to choose a career that is strictly for your own fulfillment and happiness, that’s definitely something you can do. And some of my advice will apply there as well.
The first thing I’ll say is that exploration is super important. I’ve come to appreciate that at PGSS, they require you to take almost all of the classes. You really want to explore your options for several reasons. One of the reasons is to see what you actually enjoy. Maybe all your life, you’ve done computer programming and you’ve enjoyed it, but you’ve never tried… I don’t know…physics. And when you try physics, you find out that you love it even more. You wouldn’t have known that if you didn’t explore. And I might even make a bigger claim that the first two years of college, unless you’re really, really advanced down some particular track already, is probably best spent maximizing the value of information you’re getting before you specialize in some particular thing.
Exploration is also of great importance in terms of choosing a career that is highly impactful for the world. There’s some argument for why AI safety is important to work on. There’s some argument for why pandemic preparedness is important to work on. But you need time to explore those arguments, to see if they actually make sense, if you are convinced by them. Not only are you exploring your aptitudes, what you feel good doing, what you’re skilled at doing, but you also should be exploring the reality of the impact you’re having.
Think carefully about this choice. Your career is super long, super important. Don’t rush into it. Think carefully about the pros and cons. Talk to lots of people about it. The best way to learn about what you like and what is good for you is by experiencing it, so take internships when you can, run little projects when you can, talk to people who are in the fields. You really have to test your different options.
A: Thank you. It was nice meeting you.
