Our computers are fundamentally insecure. Here’s why that could lead to global catastrophe.

Take a trip to Silicon Valley in the 70s and 80s, when going to space sounded like a good way to get around environmental limits, people started cryogenically freezing themselves, and nanotechnology looked like it might revolutionise industry – or turn us all into grey goo.

In this episode of the 80,000 Hours Podcast Christine Peterson takes us back to her youth in the Bay Area, the ideas she encountered there, and what the dreamers she met did as they grew up. We also discuss how she came up with the term ‘open source software’ (and how she had to get someone else to propose it).

Today Christine helps runs the Foresight Institute, which fills a gap left by for-profit technology companies – predicting how new revolutionary technologies could go wrong, and ensuring we steer clear of the downsides.

We dive into:

  • Can technology ‘move fast and break things’ without eventually breaking the world? Would it be better for technology to advance more quickly, or more slowly?
  • Whether the poor security of computer systems poses a catastrophic risk for the world.
  • Could all our essential services be taken down at once? And if so, what can be done about it? Christine makes a radical proposal for solving the problem.
  • Will AIs designed for wide-scale automated hacking make computers more or less secure?
  • Would it be good to radically extend human lifespan?

Continue reading →

Is it time for a new scientific revolution? Julia Galef on how to make humans smarter, why Twitter isn’t all bad, and where effective altruism is going wrong

The scientific revolution in the 16th century was one of the biggest societal shifts in human history, driven by the discovery of new and better methods of figuring out who was right and who was wrong.

Julia Galef – a well-known writer and researcher focused on improving human judgment, especially about high stakes questions – believes that if we could develop new techniques to resolve disagreements, predict the future and make sound decisions together, we could again dramatically improve the world. We brought her in to talk about her ideas.

Julia has hosted the Rationally Speaking podcast since 2010, co-founded the Center for Applied Rationality in 2012, and is currently working for the Open Philanthropy Project on an investigation of expert disagreements.

This interview complements a new detailed review of whether and how to follow Julia’s career path

We ended up speaking about a wide range of topics, including:

  • Her research on how people can have productive intellectual disagreements.
  • Why she once planned on becoming an urban designer.
  • Why she doubts people are more rational than 200 years ago.
  • What the effective altruism community is doing wrong.
  • What makes her a fan of Twitter (while I think it’s dystopian).
  • Whether more people should write books.
  • Whether it’s a good idea to run a podcast, and how she grew her audience.
  • Why saying you don’t believe X often won’t convince people you don’t.

Continue reading →

How Alex GB earned millions for charity within years by working in quant trading

Quantitative financial trading is one of the highest paying parts of the world’s highest paying industry. 25 to 30 year olds with outstanding maths skills can earn millions a year in an obscure set of ‘quant trading’ firms, where they program computers with predefined algorithms to trade very quickly and effectively.

This makes it an attractive workplace for people who want to ‘earn to give’, and we know several people who are able to donate over a million dollars a year to effective charities by working in quant trading. Who are these people? What is the job like? And is there a risk that their trading work directly harms the world?

To learn about all this I spoke at length with Alexander Gordon-Brown, who has worked as a quant trader in London for the last three and a half years and donated hundreds of thousands of pounds. We covered:

  • What quant traders do and how much they earn;
  • Whether their work is beneficial or harmful for the world;
  • How to figure out if you’re a good fit for quant trading, and if so how to break into the industry;
  • Whether Alex enjoys the work and finds it motivating, as well as what alternatives careers he considered;
  • What variety of positions are on offer in quant trading, and what the culture is like in the various firms;
  • How he decides where to donate, and whether he has persuaded his colleagues to join him in becoming major philanthropists.

Continue reading →

Which professions are paid too much given their value to society?

Many jobs have spillover effects on the rest of society. For instance, the value of new treatments discovered by biomedical researchers is far greater than what they or their employers get paid, so they have positive spillovers. Other jobs have negative spillovers, such as those that generate pollution.

A forthcoming paper, by economists at UPenn and Yale, reports a survey of the economic literature on these spillover benefits for the 11 highest-earning professions.

There’s very little literature, so all these estimates are very, very uncertain, and should be not be taken literally. But it’s interesting reading – it represents a survey of what economists think they know about this topic, and it’s surprisingly little.

Here are the bottom lines – see more detail on the estimates below. (Note that we already discussed an older version of this paper, but the estimates have been updated since then.)

We calculated mean income for 2005 in an earlier article. We increased income by 30% to account for nominal wage growth since then.

The paper uses the expressions spillover and ‘externality’. An ‘externality’ is a technical term for a ‘cost or benefit that affects a party who did not choose to incur that cost or benefit.’ The authors of the paper call it an ‘externality’ when someone who buys a service does (or does not) benefit after taking account of the cost of purchasing it. This is a nonstandard usage,

Continue reading →

Podcast: The world desperately needs AI strategists. Here’s how to become one.

If a smarter-than-human AI system were developed, who would decide when it was safe to deploy? How can we discourage organisations from deploying such a technology prematurely to avoid being beaten to the post by a competitor? Should we expect the world’s top militaries to try to use AI systems for strategic advantage – and if so, do we need an international treaty to prevent an arms race?

Questions like this are the domain of AI policy experts.

We recently launched a detailed guide to pursuing careers in AI policy and strategy, put together by Miles Brundage at the University of Oxford’s Future of Humanity Institute.

It complements our article outlining the importance of positively shaping artificial intelligence and a podcast with Dr Dario Amodei of OpenAI on more technical artificial intelligence safety work which builds on this one. If you are considering a career in artificial intelligence safety, they’re all essential reading.

I interviewed Miles to ask remaining questions I had after he finished his career guide. We discuss the main career paths; what to study; where to apply; how to get started; what topics are most in need of research; and what progress has been made in the field so far.

The audio, summary and full transcript are below.

We suggest subscribing, so you can listen at leisure on your phone,

Continue reading →

How much do hedge fund traders earn?

Hedge fund trading may be the highest paying job in the world, so to learn more, we spoke with a former manager at one of the world’s leading hedge funds. They gave us the following information, which allowed us to make a rough estimate of the typical earnings of hedge fund traders.

We also ran this document past several other people in the industry and asked them to point out mistakes.

We found that junior traders typically earn $300k – $3m per year, and it’s possible to reach these roles in 4 – 8 years. Senior portfolio managers can easily earn over $10m per year, though average earnings are probably lower. Read on for the details.

How do hedge funds make money and how is it shared among the employees?

Hedge funds trade in financial markets on behalf of clients in exchange for annual fees, and a cut of the profits. They’re similar to mutual funds but face fewer restrictions on what they can invest in, and can only be used by accredited investors.

The revenue of a hedge fund comes from the fees on the assets it manages. The typical fund charges a fee of 2% of assets under management per year, plus a performance fee. The performance fee is typically 20% of any returns it makes for the clients over and above the 2% base fee. So, if a fund makes 10% returns in a year,

Continue reading →

New career review: web designer

maxresdefault

What is the best career for someone whose main strengths are in visual design?

To start figuring that out we’ve released a new career review on web design.

Here’s a quick summary:

Pros

  • Web designers can work on a broad range of high impact projects because they are in-demand across many types of organisations, including charities, governments and startups.
  • As a backup, web designers can enter paths with good pay, like UX design ($80,000 median salary), and earn to give.

Cons

  • Good design is hard to measure, which makes it hard to prove your abilities to potential employers, meaning entry and progression can be difficult.

Who should do it?

  • You should consider web design if you studied graphic design or a related field; you’ve already spent several years developing web-design skills; and you are persuasive enable you to get a foot in the door when you’re starting out.
  • However if you have the technical skills to do web development, we recommend you do that instead, since it wins over web design on most dimensions (salary, number of jobs, job growth rate, quality of work is easier to measure).

Read the full review.

Continue reading →

What the literature says about the earnings of entrepreneurs

heres-how-much-the-giant-pile-of-money-on-breaking-bad-is-worth
It depends what kind of business you’re running.

This piece is part of our series on high impact entrepreneurship. Sign up to our newsletter and we’ll email you with the rest of the series.

Summary

  • Until recently, academics lumped ‘entrepreneurs’ together with all the ‘self-employed’. A new paper, however, split the self-employed into those who owned incorporated businesses and those who don’t. (Though note that the incorporated self-employed are still very different from startup founders.)
  • Self-employed people who own incorporated businesses earn about 50% more than people with regular jobs.
  • Most of this is due to them being more educated and working harder. However, even if you correct for these factors, it seems like shifting into owning an incorporated business boosts income by about 18%.
  • The unincorporated self-employed (mostly running things like hairdressers, restaurants, corner shops etc.) earn less than salaried workers on average.
  • Once you try to compare like-for-like workers, you find that when people switch into unincorporated self-employment, 50% earn less than they would as a salaried worker (but gain more freedom), and 30% earn more. The overall average is about the same.

Introduction

It’s widely believed that entrepreneurs earn more than salaried workers. However, until recently the research did not seem to back this up. In fact, the findings of several studies in 1989 presented a puzzle: entrepreneurs appeared to earn less than their salaried counterparts.

Continue reading →

Is nursing or headhunting the best career for you?

RedCrossNursen

Read our full review of nursing.
Read our full review of executive search.

 
One of the most frequent criticisms of our career recommender is that it usually recommends highly competitive options that are beyond the reach of most people. Furthermore, it disproportionately recommends careers for people with strong mathematical skills.

To begin to address this we have written two shallow career reviews of options that are both less competitive and less quantitative – nursing and executive search (also known as headhunting). Both are primarily ‘earning to give‘ careers.

Try our career recommender to get personalised career ideas.

Join our newsletter for regular updates about all our new career reviews.

What were the bottom lines?

Nursing:

  • is quite well paid in some countries, with a low risk of unemployment
  • provides a launching pad for a career in medical management
  • is satisfying work for most nurses, with flexibility around hours, though nurse ‘burn out’ at unusually high rates
  • offers the opportunity to study advanced nursing degrees which are even better paid.

On the other hand,

  • we expect more nurses in the developed world will improve health outcomes only a small amount
  • we are cautious about recommending a nursing degree to high-school leavers because it won’t be much use to them if they decide not to become nurses –

Continue reading →

Startup employees don’t earn more

Image by Sebastiaan ter Burg. License: CC-BY-SA 2.0

Since the average startup founder who makes it to Series A earns more than a large company employee, many believe that early-stage startup employees also earn more (albeit less than founders). Dustin Moskovitz has even claimed that startup early employees have better earnings prospects than founders.

We’ve looked at the data, and this does not seem to be true on average. There are strong reasons why people might want to work at a startup (e.g. career capital), and it’s true the employees of the most successful startups will earn more; but someone deciding between working at a startup vs. a bigger company should rarely be making the decision based on income. On average, startup early employees earn at most only a little more than developers at larger companies.

Three estimates of how much startup early employees earn, including both equity and salary

According to AngelList, early-stage backend developers, for example, generally get about $110,000 in salary and .7% equity (salary data from Riviera is similar).

While the startup salary data is fairly clear, it’s hard to know how to value the equity portion of their compensation. Below are three different methods for doing so, which all show that developers at early-stage startups at most earn only a little more than they would at a large tech company.

1) Using average exit values

Let’s assume the 0.7% equity stake will eventually get diluted down to .35% at time of exit (a typical amount of dilution from Series A to exit).

Continue reading →

Can you have more impact working in a foundation than earning to give?

Photo credit: Flickr – Refracted Moments

Key points

  • Working to improve grants at a foundation could well be more effective in terms of the impact of the money moved than earning to give. Which is better will usually come down to how good your personal opportunities are to make money, or get a job at a large foundation working on an important cause.
  • If you know of a cause area or organisation that is many times more effective than what any foundations you could work at would make grants to, then earning to give is likely to be better.
  • There are other issues, like the impact on your long-term career trajectory, that you have to consider as well as the direct impact of the money you move.

As soon as we thought of the idea of earning to give, we started thinking of ways to beat it. One idea that was floated in the very early days of 80,000 Hours was working in a foundation to allocate grants to more effective causes and organisations. Since a foundations grantmaker might allocate tens of millions of funding, far more than they could earn, maybe they could have a greater impact this way?

In this post, we provide a model for comparing the impact of foundations grantmaking and earning to give, which some people may find useful for specific scenarios where they have more info on the inputs. We also provide some very tentative estimates using the model to demonstrate how it works.

Continue reading →

If you want to save lives, should you study medicine? Probably not.

About 1 in 200 people become doctors, many of them because they want to cure the sick and generally make the world a better place. Are they making the right decision?

To help answer that question, we’ve produced an exploratory career profile on medical careers.

The conclusion of our research is that most people skilled enough to make it in a field as challenging as medicine could have a bigger social impact through an alternative career.

The best research suggests that doctors do much less to improve the health of their patients than you might naturally expect. Health is more determined by lifestyle factors, and most of the treatments that work particularly well could be delivered with a smaller number of doctors than already work in the UK or USA.

However, medicine is high earning and highly fulfilling, and we expect there are more promising opportunities to help others through biomedical research, public health, health policy and (e.g. hospital) management.

Overall, we think going to medical school would be the best way to have a social impact only if someone felt they were a significantly better fit for medicine than the other options we recommend.

Dr Greg Lewis, a practicing physician in the UK, wrote most of the career profile.

Key findings

  • Having more physicians in the developed world has a surprisingly small impact on the health of recipients.

Continue reading →

I want to make a difference. Should I work in marketing?

If you want to make a difference, should you work in marketing? The short answer: probably not. Although marketing may have positive effects through informing consumers, there’s also arguments that marketing is harmful, so it’s overall effect is unclear.

However, marketing is a valuable, transferable skill. So spending several years in marketing keeps your options open and could open up positions in high-impact organisations.

It’s also well paid, so worth considering for earning to give.

Overall, it’s worth considering as an early career option, especially if you’re stronger on verbal rather than quantitative skills, and don’t want to work in consulting (which is also highly paid and keeps your options open).

Read our full new career profile.

Continue reading →

Should you go into journalism to make a difference?

We just completed an exploratory profile on journalism. To write the profile, we interviewed an NPR correspondent and a writer for the New Yorker, and spent a day reading the best advice we could find on the career.

When it comes to having a social impact, journalism might not be the first career you think of, but we think it’s actually a pretty good option, because you can use it as a platform to promote neglected causes to a big audience. The main downside is its competitiveness, which is exasperated by reductions in the number of positions over the last decade. Spending a couple of years in journalism is also better for career capital than it first looks, because you can use it the build a good network.

Read the rest of the profile.

Continue reading →

New in-depth profile on management consulting

We’ve released a major update to our career profile on management consulting.

See the updated profile here.

See the new in-depth report upon which it’s based here.

Overall, our recommendation is similar to before:

Consider a job in consulting if you have strong academic credentials and you aren’t sure about your long-term plans and want to experience work in a variety of business environments, or you want to pursue earning-to-give but not a good fit for quantitative trading or technology entrepreneurship.

But we’ve gone much more in-depth into:

  • The chances of becoming a partner, showing that it’s about 10% but requires a great deal of dedication.
  • Common exit options, showing that consultants enter a very wide range of fields when they leave.
  • What proportion of people who want to become consultants actually make it.
  • The potential for direct impact, arguing it’s worse than other common alternatives.

This is our first ‘medium-depth’ career profile, and we hope it will act as a template for further work.

Thank you to Nick Beckstead for carrying out the research.

Continue reading →

The camel doesn’t have two humps – update to software engineering profile

In our current software engineering profile, we say:

Programming ability seems to roughly divide into two groups: those who find it relatively easy and those who don’t. If in the past you’ve done well at mathematics and science and can think abstractly, then it’s likely you can learn to program well enough to get an entry-level job within about six months.

In evidence of the first claim, one piece of evidence we cited was a paper called “The Camel Has Two Humps” by Dehnadi and Bornat.

However, we’ve just discovered that Bornat has publicly redacted this paper. He says:

It’s not enough to summarise the scientific result, because I wrote and web-circulated “The camel has two humps” in 2006. That document was very misleading and, in the way of web documents, it continues to mislead to this day. I need to make an explicit retraction of what it claimed. Dehnadi didn’t discover a programming aptitude test. He didn’t find a way of dividing programming sheep from non-programming goats. We hadn’t shown that nature trumps nurture. Just a phenomenon and a prediction.

Though it’s embarrassing, I feel it’s necessary to explain how and why I came to write “The camel has two humps” and its part-retraction in (Bornat et al., 2008). It’s in part a mental health story. In autumn 2005 I became clinically depressed. My physician put me on the then-standard treatment for depression, an SSRI. But she wasn’t aware that for some people an SSRI doesn’t gently treat depression, it puts them on the ceiling. I took the SSRI for three months, by which time I was grandiose, extremely self-righteous and very combative – myself turned up to one hundred and eleven. I did a number of very silly things whilst on the SSRI and some more in the immediate aftermath, amongst them writing “The camel has two humps”.

Based on this, we’ve removed the paper from the profile, and removed the claim about the distribution dividing into two clumps.

We intend to do a more thorough review of the predictors of success in this field when we release our full profile of software engineering in the new year.

Did we make a mistake in this case? The profile was only at the “considered” stage, so not the result of in-depth research. Even so, when most skills and abilities are normally or log-normally distributed, we should have been cautious about the existence of a bimodal distribution without relatively strong evidence.

Continue reading →

The payoff and probability of obtaining venture capital

Venture capital has facilitated the growth of many companies including Apple, Google and Facebook. But is venture capital a key to success for most startups? In this post, we answer three component questions:

  1. What are the likely outcomes for companies backed by venture capital?

  2. What fraction of companies attract venture capital?

    a) How many startups and venture capital deals are there?

    b) What proportion of applicants to venture capitalists say they accept?

  3. How much work is it to apply for venture capital?

We found that:

  • According to the data of Professors Hall and Woodward, the average venture capital-backed founder exits with $5.8 million of equity.

  • Roughly 1% of companies that aspire to obtain venture capital obtain it.

  • Finding out whether you will receive venture capital can take months to years of work.

Continue reading →

Which university has better entrepreneurs?

Some of the most successful companies in recent years have been founded by students of America’s most prestigious universities. The founders of Google and Facebook, from Stanford and Harvard respectively, are prime examples. So which universities have the most successful entrepreneurs? To answer this question, we’ve assessed how many students from each top US university have obtained investment in their startup, how many are worth over $30 million, and how many are worth over $1 billion. This builds upon Jonah Sinick’s work on the wealth of Harvard alumni.

Continue reading →