Podcast: How to train for a job developing AI at OpenAI or DeepMind

OpenAI’s Universe, a software platform for training AIs to play computer games.

Just two years ago OpenAI didn’t exist. It’s now among the most elite groups of machine learning researchers. They’re trying to make an AI that’s smarter than humans and have $1b at their disposal.

Even stranger for a Silicon Valley start-up, it’s not a business, but rather a non-profit founded by Elon Musk and Sam Altman among others, to ensure the benefits of AI are distributed broadly to all of society.

I did a long interview with one of its first machine learning researchers, Dr Dario Amodei, to learn about:

  • OpenAI’s latest plans and research progress.
  • His paper Concrete Problems in AI Safety, which outlines five specific ways machine learning algorithms can act in dangerous ways their designers don’t intend – something OpenAI has to work to avoid.
  • How listeners can best go about pursuing a career in machine learning and AI development themselves.

To listen on your phone, just subscribe to the ‘80,000 Hours Podcast’ (RSS) wherever you listen to podcasts.

The audio, summary, extra resources and full transcript are below.

Table of contents

1m33s – What OpenAI is doing, Dario’s research and why AI is so important
15m50s – How AI could be dangerous
24m20s – Would smarter than human AI solve most of the world’s problems?

Continue reading →

Which professions are paid too much given their value to society?

Many jobs have spillover effects on the rest of society. For instance, the value of new treatments discovered by biomedical researchers is far greater than what they or their employers get paid, so they have positive spillovers. Other jobs have negative spillovers, such as those that generate pollution.

A forthcoming paper, by economists at UPenn and Yale, reports a survey of the economic literature on these spillover benefits for the 11 highest-earning professions.

There’s very little literature, so all these estimates are very, very uncertain, and should be not be taken literally. But it’s interesting reading – it represents a survey of what economists think they know about this topic, and it’s surprisingly little.

Here are the bottom lines – see more detail on the estimates below. (Note that we already discussed an older version of this paper, but the estimates have been updated since then.)

We calculated mean income for 2005 in an earlier article. We increased income by 30% to account for nominal wage growth since then.

The paper uses the expressions spillover and ‘externality’. An ‘externality’ is a technical term for a ‘cost or benefit that affects a party who did not choose to incur that cost or benefit.’ The authors of the paper call it an ‘externality’ when someone who buys a service does (or does not) benefit after taking account of the cost of purchasing it. This is a nonstandard usage,

Continue reading →

Podcast: Prof David Spiegelhalter on risk, statistics and improving the public understanding of science

My colleague Jess Whittlestone and I spoke with Prof David Spiegelhalter, the Winton Professor of the Public Understanding of Risk at the University of Cambridge.

Prof Spiegelhalter tries to help people prioritise and respond to the many hazards we face, like getting cancer or dying in a car crash. To make the vagaries of life more intuitive he has had to invent concepts like the microlife, or a 30-minute change in life expectancy. He’s regularly in the UK media explaining the numbers that appear in the news, trying to assist both ordinary people and politicians to make sensible decisions based in the best evidence available.

We wanted to learn whether he thought a lifetime of work communicating science had actually had much impact on the world, and what advice he might have for people planning their careers today.

Subscribe to the podcast on iTunes or by searching 80,000 Hours wherever you get your podcasts. Links and a transcript are below.

“…What do we hear in the news? We hear about Ebola, we hear about terrorism, we hear about the latest threat that might be in what we eat and the way we travel, and we get very concerned about this, whether it’s a plane crash or whatever. Because that’s what’s in the news, that’s what is available to us. That’s what’s so prominent, but of course, so many of these risks are actually very small indeed…

What I am proud of is being part of a general community that’s very strong in Britain, to do with public engagement in science, which I’m just a small part of that because it covers material on the radio, stuff on television, stuff in some newspapers, and in various agencies. For example, in Statistics Authority, which is just trying to take a much more critical attitude to the way that numbers and evidence are used in society. I think it works. In Britain, we’re rather good compared with most people about, I don’t know, we don’t have these massive fears of vaccinations and nuclear power, of even GMOs. I think this is a sign that we in this country have developed quite a good public engagement with science community.”

Continue reading →

Podcast: The world desperately needs AI strategists. Here’s how to become one.

If a smarter-than-human AI system were developed, who would decide when it was safe to deploy? How can we discourage organisations from deploying such a technology prematurely to avoid being beaten to the post by a competitor? Should we expect the world’s top militaries to try to use AI systems for strategic advantage – and if so, do we need an international treaty to prevent an arms race?

Questions like this are the domain of AI policy experts.

We recently launched a detailed guide to pursuing careers in AI policy and strategy, put together by Miles Brundage at the University of Oxford’s Future of Humanity Institute.

It complements our article outlining the importance of positively shaping artificial intelligence. If you are considering a career in artificial intelligence safety, both are essential reading.

I interviewed Miles to dig deeper into his advice. We discuss the main career paths; what to study; where to apply; how to get started; what topics are most in need of research; and what progress has been made in the field so far.

The audio, summary and full transcript are below.

To listen on your phone, just subscribe to the ‘80,000 Hours Podcast’ (RSS) wherever you listen to podcasts. That way you can listen to it sped up and get alerts about future episodes.

Full transcript

Robert Wiblin: Hi,

Continue reading →

The Schwarzman Scholarship: An exciting opportunity to learn more about China and get a Masters in Global Affairs

Co-authored with Helen Toner.

In general, living in a foreign country – for example, by studying there – is a great way to learn about the country, its language, people, and culture.

There’s one country in particular we think it would be very valuable for some of our readers to become knowledgable about: China. It’s the world’s largest country by population, and gets closer every year to being the largest global economy too. On the international stage, China is the probably world’s second most influential country. Its influence in economic, geopolitical, and cultural terms looks set to continue to grow throughout the course of the 21st century.

All of this means that China is likely to play a larger and larger role in all kinds of areas, including topics we care a great deal about, like factory farming, pandemic preparedness, preserving international peace and cooperation, and AI research. Being familiar with China is likely to provide unique and high-value opportunities to people working in those areas. What’s more, there aren’t yet many westerners with deep (or even passing) familiarity with China, which suggests that now is an especially valuable time to be learning more.

If you’d like to become knowledgable about China, there is a promising new way to do that: the Schwarzman Scholarship program.

Modelled on the Rhodes Scholarship, the Schwarzman Scholarship is targeted at high achieving students from around the world with an interest in leadership,

Continue reading →

How much do hedge fund traders earn?

Hedge fund trading may be the highest paying job in the world, so to learn more, we spoke with a former manager at one of the world’s leading hedge funds. They gave us the following information, which allowed us to make a rough estimate of the typical earnings of hedge fund traders.

We also ran this document past several other people in the industry and asked them to point out mistakes.

We found that junior traders typically earn $300k – $3m per year, and it’s possible to reach these roles in 4 – 8 years. Senior portfolio managers can easily earn over $10m per year, though average earnings are probably lower. Read on for the details.

How do hedge funds make money and how is it shared among the employees?

Hedge funds trade in financial markets on behalf of clients in exchange for annual fees, and a cut of the profits. They’re similar to mutual funds but face fewer restrictions on what they can invest in, and can only be used by accredited investors.

The revenue of a hedge fund comes from the fees on the assets it manages. The typical fund charges a fee of 2% of assets under management per year, plus a performance fee. The performance fee is typically 20% of any returns it makes for the clients over and above the 2% base fee. So, if a fund makes 10% returns in a year,

Continue reading →

Most people report believing it’s incredibly cheap to save lives in the developing world

One way that people can have a social impact with their career is to donate money to effective charities. We mention this path in our career guide, suggesting that people donate to evidence-backed charities such as the Against Malaria Foundation, which is estimated by GiveWell to save the lives of children in the developing world for around $7,500.

Alyssa Vance told me that many people may see this as highly ineffective relatively to their optimistic expectations about how much it costs to improve the lives of people in people. I thought the reverse would be true – folks would be skeptical that charities in the developing world were effective at all. Fortunately Amazon Mechanical Turk makes it straightforward to survey public opinion at a low cost, so there was no need for us to sit around speculating. I suggested a survey on this question to someone in the effective altruism community with a lot of experience using Mechanical Turk – Spencer Greenberg of Clearer Thinking – and he went ahead and conducted one in just a few hours.

You can work through the survey people took yourself here and we’ve put the data and some details about the method in a footnote. The results clearly vindicated Alyssa:

It turns out that most Americans believe a child can be prevented from dying of preventable diseases for very little –

Continue reading →

How accurately does anyone know the global distribution of income?

World income distributionHow much should you believe the numbers in figures like this?

People in the effective altruism community often refer to the global income distribution to make various points:

  • The richest people in the world are many times richer than the poor.
  • People earning professional salaries in countries like the US are usually in the top 5% of global earnings and fairly often in the top 1%. This gives them a disproportionate ability to improve the world.
  • Many people in the world live in serious absolute poverty, surviving on as little as one hundredth the income of the upper-middle class in the US.

Measuring the global income distribution is very difficult and experts who attempt to do so end up with different results. However, these core points are supported by every attempt to measure the global income distribution that we’ve seen so far.

The rest of this post will discuss the global income distribution data we’ve referred to, the uncertainty inherent in that data, and why we believe our bottom lines hold up anyway.

Will MacAskill had a striking illustration of global individual income distribution in his book Doing Good Better, that has ended up in many other articles online, including our own career guide:
 
 

 
The data in this graph was put together back in 2012 using an approach suggested by Branko Milanovic,

Continue reading →

What skills are effective altruist organisations short of? Results from our survey.

In August 2016, we surveyed 16 organisations in the effective altruism community about their hiring needs, and to what extent they are constrained by talent compared to funding.

What follows is a summary of the results, grouped by question asked. You can see the list of organisations surveyed in the footnotes.

Note that since the survey was carried out over six months ago, some of the information may no longer be up to date. We intend to repeat the survey in August 2017, and will report back on how the situation has changed.

What types of talent does your organisation need?

Here are the options provided on the survey, along with the number of organisations which stated that they were looking to hire people for these roles:

(Note that this table is not weighted by budget or team size, although we do not expect that this would materially affect the results.)

In open feedback, several respondents also mentioned that the community is most in need of specialist researchers rather than generalist researchers. Some other skill sets which were not included as options, but which were mentioned more than twice in the open feedback include:

  1. Economists, in 3 cases.
  2. Math and AI researchers, in 2 cases.
  3. Policy experts, in 2 cases.
  4. Scientists, especially biologists, in 3 cases.

Also see the longer list of skill sets needed by the community at the end of this summary.

Continue reading →

    5 reasons not to go into education

    First published June 2015. Updated February 2017.

    When we first speak to people interested in doing good with their careers, they often say they want to get involved in education in the US or the UK. This could mean donating to a school, doing education policy work, or becoming a teacher.

    However, we haven’t prioritised careers in education at 80,000 Hours. We don’t dispute that education is a highly important problem – a more educated population could enable us to solve many other global challenges, as well as yield major economic benefits. The problem is that it doesn’t seem to be very easy to solve or neglected (important elements of our problem framework). So, it looks harder to have a large impact in education compared to many other areas. In the rest of this post, we’ll give five reasons why.

    The following isn’t the result of in-depth research; it’s just meant to explain why we’ve deprioritised education so far. Our views could easily change. Note that in this post we’re not discussing education in the developing world.

    1. It’s harder to help people in the US or UK

    Everyone in the US or UK is rich by global standards: the poorest 5% of Americans are richer than the richest 5% of Indians (and that’s adjusted for the difference in purchasing power, see an explanation and the full data).

    Continue reading →

    The effective altruism guide to donating this giving season

    People in the effective altruism community aim to use evidence and careful reasoning to work out how to best promote the wellbeing of all. To find the highest-impact charities this giving season, they’ve done tens of thousands of hours of research and published over 50,000 words of analysis this month. We read it all, and summed up the main recommendations by area.

    But which of the 9 problem areas listed should you personally give to? We’ve got you covered here too. This tool asks you six questions and adjusts the ranking based on your beliefs:2

    Quiz: Which problem should you give to? →

    In the full post, you can find (i) how we came up with the list, (ii) more advice on how to narrow down the list, (iii) more information on each charity.

    Continue reading →

    80,000 Hours annual review Dec 2016

    Summary

    2016 was an excellent year for 80,000 Hours. Here are some highlights – full details follow.

    Metrics

    • In our last review in May 2015, we set the goal of 50 significant plan changes per month by October 2016. That month, we actually recorded over 200.
    • To make it harder to grow by adding lots of small plan changes, in October 2015 we started “impact rating” the plan changes, and tracking the impact-weighted total. 31 Dec 2015, we set the target of tripling the monthly rate of impact-adjusted plan changes over the year, which we achieved in November 2016. We now track about 150 impact-adjusted significant plan changes (IASPC) per month.

    Impact and cost-effectiveness

    • Our costs in 2016 were £250,000, up 13% on 2015. Considering that our staff could have earned to give instead, the total opportunity cost is perhaps £350,000 – £500,000.
    • Since our last review, the ratio of costs to IASPC fell almost 3-fold.
    • In 2016, we caused 115 people to take the Giving What We Can (GWWC) 10% pledge. GWWC estimates this is worth about £5 million in donations to their recommended charities (counterfactually-adjusted, time-discounted, dropout adjusted). So this alone plausibly justifies our costs, although our aim is to solve talent gaps rather than funding gaps.
    • In addition, the plan changes since our last review now include three people who each intend to donate over $100m over their lifetimes,

    Continue reading →

    Has 80,000 Hours justified its costs?

    We set up 80,000 Hours because we thought it could become one of the most effective charities in the world. The idea was to achieve a multiplier – with a small amount of our time, we could enable thousands of others to spend their careers on whatever is most effective, and achieve thousands of times as much as we could individually.

    In this post, we examine whether 80,000 Hours has generated enough impact to justify its costs over our history, and make some rough estimates of our multiplier.

    Because it’s hard to estimate what would have happened if 80,000 Hours had never existed, all of these estimates are very uncertain, and can be debated. However, there are multiple ways we’ve plausibly justified our costs to date. In this document, we sketch out some of these pathways. We’re not aiming to be fully rigorous. Rather, consider the examples as a group. If only a few turn out to be genuine cases of impact, we’ll have justified our costs many times over.

    Is this the wrong question?

    80,000 Hours is a startup. Asking whether we’ve justified our costs to date is like asking whether Google was profitable in 2000. The aim of 80,000 Hours is to grow, and have a far larger impact years in the future.

    Most of the value of donations to 80,000 Hours comes from the chance that these donations enable us to grow 10-times or 100-times.

    That said,

    Continue reading →

    End of year update on plan changes

    This is an update on the number of significant plan changes we’ve caused as of the end of Nov 2016.

    We define a significant plan change as:

    Someone tells us that 80,000 Hours caused them to change the career path they intend to pursue, in a way that they think increases their lifetime impact.

    More on what counts as a significant plan change here.

    Our total number of plan changes as of the end of Nov 2016 is 1,854, and after impact-adjusting these it’s 1,504.8.

    Continue reading →

    80,000 Hours has a funding gap

    Over the past three years, we’ve grown almost 36-fold, more than tripling each year. This is measured in terms of our key metric – the number of impact-adjusted significant plan changes each month. At the same time, our budget has only increased 27% per year.

    Given this success, we think it’s time to take 80,000 Hours to the next level of funding.

    Over the next few weeks, we’ll be preparing our full annual review and fundraising documents, but here’s a preview.

    chart

    Overall, the 2017 target is to triple, measured in terms of impact-adjusted significant plan changes per month (which will mean over 3,000 over the year). We’ll do this by continuing to improve the advice, and starting to scale up marketing, with the aim of becoming the default source of career advice for talented, socially-motivated graduates.

    Concretely, here’s some priorities we could pursue:

    • Dramatically improve the career reviews and problem profiles, so we have in-depth profiles of all the best options. This will help our existing users make better changes, and bring in more traffic.
    • Upgrading – develop mentors and specialist content for the most high-potential users, such as those who want to work on AI risk, policy, EA orgs and so on. We now have a large base of engaged users (1300+ through the workshop, 80,000+ on newsletter), so there’s a lot of follow-up we could do to get more valuable plan changes from them.

    Continue reading →

    How much is one vote worth?

    Just 537 votes in Florida would have been enough to change the outcome of the 2000 election from George Bush to Al Gore – a margin of 0.009% (recount pictured above). And that wasn’t even the closest-won state that year: in New Mexico the margin was a mere 366 votes.

    People say it’s your civic duty to vote, but it also seems like it’s very unlikely your vote will make a difference.

    Who is right? Is voting really valuable, or a waste of time?

    We looked into the research on this, especially regarding the US Presidential election. The answer, surprisingly, is that the single hour you spend voting for the President and Congress can be the most important thing you do with an hour each four years – and we expect similar numbers for other kinds of elections outside the USA. It also looks like there are effective techniques you can use to ‘get out the vote’, if you want to do more than just vote yourself.

    The impact of your vote largely depends on 2 things, which we’ll investigate in turn:

    • The chances of your vote changing the election outcome.
    • How much better for the world as a whole one candidate is, compared to another.

    At first blush it might seem that the chances of your single vote changing the election outcome are zero. But while the chances are low, they could be around 1 in 10 million if you live in a swing state.

    Continue reading →

    Update on 80,000 Hours May 2015 – June 2016

    This is a quick update on our progress over the last year. Our next in-depth annual review, in which we’ll vet everything in more depth, will be in January 2017.

    Our impact across the year

    Here’s our key metrics for the top of our funnel. Ultimately we care about significant plan changes, which we report right below.

    Unique visitors to site
    New newsletter subscribers

    Our newsletter now has a total of over 50,000 subscribers, which we think makes it the largest in the effective altruism community (most others have about 10,000). Our total traffic also just overtook GiveWell, which we think is the next largest by traffic (we had 880,000 users over the 12 months ending June, compared to 860,000 when calculated the same way.)

    Continue reading →

    Should you work at GiveWell? Reflections from a recent employee.

    The following are some reflections on what it’s like to work at GiveWell written by one of our readers. We’re posting their thoughts because we’ve written about GiveWell as a high-impact career in the past, and are keen to share more information about it. The opinions below, however, may not reflect our views.

    I worked at GiveWell from August 2014 to May 2016. This piece is a reflection on my time there, on things I think GiveWell does well as an employer, on things I think it could do better, and why I decided to leave.

    I envision two functions for this piece: (1) as an exercise to help me process my time at GiveWell, and (2) as a resource for people considering working at GiveWell. When I was considering taking a job at GiveWell, I found Nick Beckstead’s reflection on his internship at GiveWell to be very helpful. Outside of Nick’s piece, there isn’t very much substantive information available about working at GiveWell. Many people consider employment at GiveWell; I hope some of those people find this reflection to be useful.

    Some background

    I learned about GiveWell in Spring 2014, after reading Peter Singer’s Famine, Affluence, and Morality in a college ethics class and encountering related topics on the internet. By the time I took the ethics class, I knew that I did not want to go to graduate school immediately after my undergraduate, but I was very taken by academic ethics and wanted to continue serious thinking about the topic.

    Continue reading →

    Trevor decided to move from a non-profit to a for-profit to do more good in the long run. Was it the right call?

    This is part of our series of profiles of people who changed their career in a major way in order to have more impact because of their exposure to 80,000 Hours.

    I recently spoke to Trevor Shorb about how his career plans changed as a result of 80,000 Hours. After finishing university, Trevor worked in the Peace Corps in El Salvador and planned to work for an NGO in the developing world. But after reading our advice, he decided to gain skills in the private sector first, in order to have a bigger impact in the long run. Today Trevor does business development for an international education company in emerging markets in Latin America. He plans to start a non-profit or for-profit in the developing world in the future.

    How and why did he make this transition? Read our interview with him to find out.

    How did you find out about 80,000 Hours and effective altruism, and what were you planning on doing with your career before that?

    I first became interested in effective altruism when I read “The Life You Can Save” around the time I graduated college and had committed to serve in the Peace Corps.

    Before that I had undergone a fundamental change in perspective. Recruited to college to play lacrosse, I was fully dedicated to the pursuit of being the best and leading the team. A case of chronic Lyme disease led to multiple operations and much time spent in doctor’s offices.

    Continue reading →

    Is deep work the most underappreciated skill for career success? An interview with Cal Newport.

    Deep-WorkIn the land of the blind, the one-eyed man is king – or so the saying goes. In his new book, Deep Work, Cal Newport argues that when it comes to deep concentration, we have become the land of the blind.

    He believes that the ability to do focused work is essential for career success, but becoming increasingly rare. And the fewer people are capable of deep work, the more of an edge you should be able to get by being the exception.

    We think Cal is one of the most interesting thinkers working on the issue of career strategy, so recommend you check it out. We interviewed Cal to learn more.

    Quick summary of the book

    Cal defines “deep work” as “the ability to focus without distraction on a cognitively demanding task”.

    In his previous book, So Good They Can’t Ignore You, Cal argues it’s better to focus on gaining career capital rather than “following your passion” (we agree). Part one of Deep Work takes off where his previous book ends – he argues that deep work is essential to gaining valuable career capital. In short:

    • Deep work is increasingly valuable in the modern economy because it’s what allows you to master new intellectual skills and produce creative breakthroughs. People able to do both of these can take work that’s unlikely to be automated and reach the top of their fields,

    Continue reading →