Podcast: How to train for a job developing AI at OpenAI or DeepMind

OpenAI’s Universe, a software platform for training AIs to play computer games.

Just two years ago OpenAI didn’t exist. It’s now among the most elite groups of machine learning researchers. They’re trying to make an AI that’s smarter than humans and have $1b at their disposal.

Even stranger for a Silicon Valley start-up, it’s not a business, but rather a non-profit founded by Elon Musk and Sam Altman among others, to ensure the benefits of AI are distributed broadly to all of society.

I did a long interview with one of its first machine learning researchers, Dr Dario Amodei, to learn about:

  • OpenAI’s latest plans and research progress.
  • His paper Concrete Problems in AI Safety, which outlines five specific ways machine learning algorithms can act in dangerous ways their designers don’t intend – something OpenAI has to work to avoid.
  • How listeners can best go about pursuing a career in machine learning and AI development themselves.

To listen on your phone, just subscribe to the ‘80,000 Hours Podcast’ (RSS) wherever you listen to podcasts.

The audio, summary, extra resources and full transcript are below.

Table of contents

1m33s – What OpenAI is doing, Dario’s research and why AI is so important
15m50s – How AI could be dangerous
24m20s – Would smarter than human AI solve most of the world’s problems?

Continue reading →

Which professions are paid too much given their value to society?

Many jobs have spillover effects on the rest of society. For instance, the value of new treatments discovered by biomedical researchers is far greater than what they or their employers get paid, so they have positive spillovers. Other jobs have negative spillovers, such as those that generate pollution.

A forthcoming paper, by economists at UPenn and Yale, reports a survey of the economic literature on these spillover benefits for the 11 highest-earning professions.

There’s very little literature, so all these estimates are very, very uncertain, and should be not be taken literally. But it’s interesting reading – it represents a survey of what economists think they know about this topic, and it’s surprisingly little.

Here are the bottom lines – see more detail on the estimates below. (Note that we already discussed an older version of this paper, but the estimates have been updated since then.)

We calculated mean income for 2005 in an earlier article. We increased income by 30% to account for nominal wage growth since then.

The paper uses the expressions spillover and ‘externality’. An ‘externality’ is a technical term for a ‘cost or benefit that affects a party who did not choose to incur that cost or benefit.’ The authors of the paper call it an ‘externality’ when someone who buys a service does (or does not) benefit after taking account of the cost of purchasing it. This is a nonstandard usage,

Continue reading →

Podcast: The world desperately needs AI strategists. Here’s how to become one.

If a smarter-than-human AI system were developed, who would decide when it was safe to deploy? How can we discourage organisations from deploying such a technology prematurely to avoid being beaten to the post by a competitor? Should we expect the world’s top militaries to try to use AI systems for strategic advantage – and if so, do we need an international treaty to prevent an arms race?

Questions like this are the domain of AI policy experts.

We recently launched a detailed guide to pursuing careers in AI policy and strategy, put together by Miles Brundage at the University of Oxford’s Future of Humanity Institute.

It complements our article outlining the importance of positively shaping artificial intelligence. If you are considering a career in artificial intelligence safety, both are essential reading.

I interviewed Miles to dig deeper into his advice. We discuss the main career paths; what to study; where to apply; how to get started; what topics are most in need of research; and what progress has been made in the field so far.

The audio, summary and full transcript are below.

To listen on your phone, just subscribe to the ‘80,000 Hours Podcast’ (RSS) wherever you listen to podcasts. That way you can listen to it sped up and get alerts about future episodes.

Full transcript

Robert Wiblin: Hi,

Continue reading →

How much do hedge fund traders earn?

Hedge fund trading may be the highest paying job in the world, so to learn more, we spoke with a former manager at one of the world’s leading hedge funds. They gave us the following information, which allowed us to make a rough estimate of the typical earnings of hedge fund traders.

We also ran this document past several other people in the industry and asked them to point out mistakes.

We found that junior traders typically earn $300k – $3m per year, and it’s possible to reach these roles in 4 – 8 years. Senior portfolio managers can easily earn over $10m per year, though average earnings are probably lower. Read on for the details.

How do hedge funds make money and how is it shared among the employees?

Hedge funds trade in financial markets on behalf of clients in exchange for annual fees, and a cut of the profits. They’re similar to mutual funds but face fewer restrictions on what they can invest in, and can only be used by accredited investors.

The revenue of a hedge fund comes from the fees on the assets it manages. The typical fund charges a fee of 2% of assets under management per year, plus a performance fee. The performance fee is typically 20% of any returns it makes for the clients over and above the 2% base fee. So, if a fund makes 10% returns in a year,

Continue reading →

The rent is too damn high – should you work on reforming land use regulations?

We’ve released a new ‘problem profile’ on reform of how land is used in cities.

Local laws often prohibit the construction of dense new housing, which drives up prices, especially in a few large high-wage urban areas. The increased prices transfer wealth from renters to landowners and push people away from centres of economic activity, which reduces their ability to get a job or earn higher wages, likely by a very large amount.

An opportunity to tackle the problem which nobody has yet taken is to start a non-profit or lobbying body to advocate for more housing construction in key urban areas and states. Another option would be to try to shift zoning decisions from local to state governments, where they are less likely to be determined by narrow local interests, especially existing land-owners who benefit from higher property prices.

In the profile we cover:

  • The main reasons for and against thinking that working on land use reform is among the best uses of your time.
  • How to use your career to make housing in prospering cities more accessible to ordinary people.

Read our full profile on land use reform.

Continue reading →

New career review: web designer

maxresdefault

What is the best career for someone whose main strengths are in visual design?

To start figuring that out we’ve released a new career review on web design.

Here’s a quick summary:

Pros

  • Web designers can work on a broad range of high impact projects because they are in-demand across many types of organisations, including charities, governments and startups.
  • As a backup, web designers can enter paths with good pay, like UX design ($80,000 median salary), and earn to give.

Cons

  • Good design is hard to measure, which makes it hard to prove your abilities to potential employers, meaning entry and progression can be difficult.

Who should do it?

  • You should consider web design if you studied graphic design or a related field; you’ve already spent several years developing web-design skills; and you are persuasive enable you to get a foot in the door when you’re starting out.
  • However if you have the technical skills to do web development, we recommend you do that instead, since it wins over web design on most dimensions (salary, number of jobs, job growth rate, quality of work is easier to measure).

Read the full review.

Continue reading →

What the literature says about the earnings of entrepreneurs

heres-how-much-the-giant-pile-of-money-on-breaking-bad-is-worth
It depends what kind of business you’re running.

This piece is part of our series on high impact entrepreneurship. Sign up to our newsletter and we’ll email you with the rest of the series.

Summary

  • Until recently, academics lumped ‘entrepreneurs’ together with all the ‘self-employed’. A new paper, however, split the self-employed into those who owned incorporated businesses and those who don’t. (Though note that the incorporated self-employed are still very different from startup founders.)
  • Self-employed people who own incorporated businesses earn about 50% more than people with regular jobs.
  • Most of this is due to them being more educated and working harder. However, even if you correct for these factors, it seems like shifting into owning an incorporated business boosts income by about 18%.
  • The unincorporated self-employed (mostly running things like hairdressers, restaurants, corner shops etc.) earn less than salaried workers on average.
  • Once you try to compare like-for-like workers, you find that when people switch into unincorporated self-employment, 50% earn less than they would as a salaried worker (but gain more freedom), and 30% earn more. The overall average is about the same.

Introduction

It’s widely believed that entrepreneurs earn more than salaried workers. However, until recently the research did not seem to back this up. In fact, the findings of several studies in 1989 presented a puzzle: entrepreneurs appeared to earn less than their salaried counterparts.

Continue reading →

Use our tool to decide whether you’re on the right career path

Screen Shot 2016-02-19 at 3

You know how you should review your career at least once a year to make sure that you’re on the right path and set goals for the coming year?

You did that already, right?

Oh, no?

Well, in that case we’ve created a tool to make it quick and easy. Just answer the questions, and we’ll email you your answers when you’re done. There are only six key questions:

Do your annual career review

 
Once you’re done and have decided what steps to take, you can relax about your career trajectory for another 12 months!

Continue reading →

Is nursing or headhunting the best career for you?

RedCrossNursen

Read our full review of nursing.
Read our full review of executive search.

 
One of the most frequent criticisms of our career recommender is that it usually recommends highly competitive options that are beyond the reach of most people. Furthermore, it disproportionately recommends careers for people with strong mathematical skills.

To begin to address this we have written two shallow career reviews of options that are both less competitive and less quantitative – nursing and executive search (also known as headhunting). Both are primarily ‘earning to give‘ careers.

Try our career recommender to get personalised career ideas.

Join our newsletter for regular updates about all our new career reviews.

What were the bottom lines?

Nursing:

  • is quite well paid in some countries, with a low risk of unemployment
  • provides a launching pad for a career in medical management
  • is satisfying work for most nurses, with flexibility around hours, though nurse ‘burn out’ at unusually high rates
  • offers the opportunity to study advanced nursing degrees which are even better paid.

On the other hand,

  • we expect more nurses in the developed world will improve health outcomes only a small amount
  • we are cautious about recommending a nursing degree to high-school leavers because it won’t be much use to them if they decide not to become nurses –

Continue reading →

How important is finding a career that matches your strengths?

One of the most common ideas in career advice is that finding a good career is a matter of finding the role that uniquely matches who you are. You’ll be fantastic at the career that best matches you, and terrible at other careers, so the mission should be to find the career that’s the best match.

We haven’t found much support for this idea so far. The most in-depth attempt to study “match” is Holland-types, but several meta-analyses have found no or only a very weak relationship between Holland-type match and performance (or job satisfaction). On the other hand, we’ve encountered some important general predictors of success. For instance, hundreds of studies have found that the smarter you are, the more likely you are to succeed in almost every career. With a general predictor like intelligence, more is always better – it’s not that it means you’ll do well in some jobs but worse in others depending on your “match”.

However, a new line of research into “strengths” might shift the picture. There have been two attempts – the Virtues in Action (VIA) Signature Strengths test and Strengths Finder – to determine people’s character strengths, and study the importance of leading a career in line with them.

We did a review of the literature to see whether we should incorporate them into our advice, which we summarise below.

Continue reading →

The value of coordination

Screen Shot 2016-02-09 at 12.26.20 AM

This post is intended for people who are already familiar with our key content. If you’re new, read the basics first.

In assessing your positive impact on the world, you need to look at the additional good you do after taking into account what would have happened if you hadn’t acted. But how can you evaluate this?

One way is the “single player approach” – consider what would happen if you act and what would happen if you don’t act, holding everyone else constant, and then look at the difference between the two scenarios.

This approach worked pretty well in the early days of effective altruism, but it starts to break down once you’re part of a community of thousands of people who will change their behaviour depending on what you do.

When you’re part of a community, the counterfactuals become more complex, and doing the most good becomes much more of a coordination problem – it’s a multiplayer rather than a single player game.

In this post, I’ll list five situations where this insight can help us to become even more effective, and I’ll suggest new rules of thumb that I think might be the best guide in a multiplayer world. This is a complex topic, so the answers I give are still tentative. I’m keen to see many more people in the community start thinking about these issues,

Continue reading →

Where should you donate to have the most impact during giving season 2015?

maxresdefault

Many of our readers choose to give away substantial sums over the ‘giving season’ around Christmas and New Year. Where should they give so that their money has the biggest social impact?

This post is based on a combination of my existing knowledge, some judgement calls based on three years working in effective altruism, and brief consultation with the people involved in the groups below. It’s not based on in-depth research, and the recommendations could easily change. Take this post as a starting point for your own analysis.

Note that we’re looking for the charities that help others the most, treating everyone’s welfare as equal. If you have a particular attachment to a specific cause, you’ll need to factor that in separately.

This flowchart is a summary of the advice below. Read on for more details.

Continue reading →

What’s the easiest way anyone can have a big social impact?

Many people want a career that contributes to the world, that helps others live happier lives. To do this, some become teachers, some work in the nonprofit sector, and others work in many other sectors. Sometimes this involves significant personal sacrifice – at the very least, “socially good” jobs usually have lower earnings.

Let’s suppose you want to minimise sacrifice and maximise social impact. What should you choose then? What would be good is a career option that’s:

  1. Open to most of our audience (college grads in developed countries).
  2. Involves little or no sacrifice.
  3. Has as large a social impact as possible, with high confidence.

I think a path like this exists, as I’ll argue in the rest of this post. I call it the easy baseline:

  1. Take whichever job you’d find most personally fulfilling.
  2. Give 10% of your income to the world’s poorest people.

As of 2008, you can give your income to the world’s poorest people through GiveDirectly, a charity that provides one-off cash transfers to the poorest people in Kenya via mobile app. Every $1 you give results in $0.90 in the hands of one of the world’s poorest people. This intervention could soak up billions of dollars in the coming years, so could be pursued by many people.

If you want to have a social impact with your career, giving 10% is the easiest thing you can do, and I think almost everyone reading this should do it. You can take a public pledge to do so in just a few minutes.

Below I’ll explain in more detail why doing so is such an attractive option.

Continue reading →

How to pursue a career in research to lower the risks from superintelligent machines: a new career review.

google-deepmind-artificial-intelligence

This is a summary of our full career review on artificial intelligence risk research.

Have you read the profile and think you want to contribute to artificial intelligence risk research? Fill out this form and we’ll see if we can help.

Many people we coach are interested in doing research into artificial intelligence (AI), in particular how to lower the risk that superintelligent machines do harmful things not intended by their creators – a field usually referred to as ‘AI risk research’. The reasons people believe this is a particularly pressing area of research are outlined in sources such as:

Our goal with this career review was not to assess the cause area of AI risk research – on that we defer to the authors above. Rather we wanted to present some concrete guidance for the growing number of people who want to work on the problem.

We spoke to the leaders in the field, including top academics, the head of MIRI and managers in AI companies, and the key findings are:

  • Some organisations working on this problem,

Continue reading →

Why you should focus more on talent gaps, not funding gaps

80K_posts_talentgap_balance_V2

Many members of the effective altruism community see making a difference primarily in terms of moving money to fill funding gaps rather than moving talent to fill talent gaps. This seems to me to be one of the community’s more serious mistakes, which causes us to:

  • Put too much weight on earning to give and fundraising.
  • Put too little weight on gaining expertise and developing the skills needed for direct work.
  • Overlook pressing causes that aren’t funding constrained.

In the rest of the post, I’ll:

  1. Outline what I mean by talent gaps.
  2. Suggest why the community might be biased towards focusing on funding gaps.
  3. Argue there are whole cause areas we’ve completely overlooked due to this focus.
  4. Argue that many of the causes the community does support are also more talent constrained than funding constrained.
  5. Argue that the importance of talent constraints compared to funding constraints is likely to increase over the next 2-5 years.
  6. Argue further that this imbalance is likely to persist in the long-term.
  7. Consider some of the arguments against focusing on talent gaps.
  8. Give ideas for what the community should do differently in order to focus more on talent gaps. In particular, I’ll outline who should earn to give and who shouldn’t, and list the greatest talent needs within the community.

Continue reading →

Our new tool can help you make the right career decision

map

We’ve released a new tool to help you think through career decisions, such as which major to study, which jobs to apply for, and which of various offers to accept. We’ve tested it in one-on-one coaching over the last six months, and are now making it freely available.

These decisions can both be very important and very difficult. This tool will make these decisions easier by walking you through a step-by-step process, asking you the questions we use during coaching, and checking that you’ve applied the most important results of our research. We designed the process using the scientific literature on decision making to reduce bias and it has received positive feedback from many users:

  1. Try out the tool.
  2. Share it on Facebook or Twitter.

It won’t tell you what to do, but it will make sure you haven’t missed anything obvious, are asking the right questions, and have a clear next step. It takes about 30 min to run through.

We will continue to improve the tool in coming months based on your feedback and our experience in coaching.

If you know a friend or family member trying to make a career decision you might also like to pass it on to them!

80K_howtochoose_flowchart_V5

Continue reading →

How to write a career plan

We see lots of career planning mistakes.

Some people simply don’t have a plan, and hope the future will figure itself out. This leads them to take steps that seem attractive in the short-run but don’t help in the long-run e.g. we’ve met quite a few people who ended up regretting doing a philosophy PhD.

Other people try to figure out “what to do with their lives”, or make a detailed “10 year plan”. That doesn’t work either.

Instead, we recommend:

  1. Have a plan, but make it flexible – we call this flexible plan a “vision”.
  2. Review your plan at least once a year. Think like a scientist testing a hypothesis.
  3. Make sure you gain flexible career capital, that way you’ll be in a better position no matter what the future holds.

We recently updated our key article on career planning to explain.

For long-term readers, what’s new?
1. New content on how to make your “vision”

We added a new step-by-step process for making your plan based on “ABZ planning”, an idea we found in Reid Hoffman’s excellent book “The Start-Up of You”.

Here’s the process we recommend:

First start by asking:

  1. What does the world most need? List the 1-3 causes that you think are most pressing. If you’re trying to make an impact, then you need to start by understanding what the world most needs.

Continue reading →

Startup employees don’t earn more

Image by Sebastiaan ter Burg. License: CC-BY-SA 2.0

Since the average startup founder who makes it to Series A earns more than a large company employee, many believe that early-stage startup employees also earn more (albeit less than founders). Dustin Moskovitz has even claimed that startup early employees have better earnings prospects than founders.

We’ve looked at the data, and this does not seem to be true on average. There are strong reasons why people might want to work at a startup (e.g. career capital), and it’s true the employees of the most successful startups will earn more; but someone deciding between working at a startup vs. a bigger company should rarely be making the decision based on income. On average, startup early employees earn at most only a little more than developers at larger companies.

Three estimates of how much startup early employees earn, including both equity and salary

According to AngelList, early-stage backend developers, for example, generally get about $110,000 in salary and .7% equity (salary data from Riviera is similar).

While the startup salary data is fairly clear, it’s hard to know how to value the equity portion of their compensation. Below are three different methods for doing so, which all show that developers at early-stage startups at most earn only a little more than they would at a large tech company.

1) Using average exit values

Let’s assume the 0.7% equity stake will eventually get diluted down to .35% at time of exit (a typical amount of dilution from Series A to exit).

Continue reading →

How to explore in your career

Tony Blair tried to make it in rock n roll before going into politics.

We often talk about the benefits of trying out several different paths, especially early in your career. But changing job too often has costs. How can you explore effectively?

Here’s a couple of ideas from our recently updated page on why and how to explore.

Tips for exploring
1. Use your natural exploration opportunities

There are times when, money permitting, society gives you a free pass to do something random:

  • In a “gap year” between school and university, or just after you graduate.
  • In your high school and university summer holidays.
  • In university courses that aren’t your major.

Take advantage of these opportunities as much as you can.

2. Use the “graduate school reset”

After you graduate from university, you can do something unusual and risky for a few years, then go to graduate school, then begin a regular professional career as if not much happened. More specifically:

  • You can take one to two years out before doing a PhD, and by the time you graduate, it probably won’t matter.
  • Sometimes you can use a Masters degree to transition into a new field.
  • You can do a law degree and become a lawyer or go into policy.
  • Consulting, finance and many corporate jobs accept people directly out of MBAs,

Continue reading →