Coaching Applications Analysis

Overview

Who has been applying to coaching from 80,000 Hours? In this post, we analyze our coaching applications to understand what kinds of people are requesting coaching.

Table of Contents

Summary

Data sources and quantity

We analyzed responses from our Social Impact Coaching applications. These consisted of multiple choice questions, text-response paragraphs, and CVs or resumes. This analysis looked at all 91 Social Impact Coaching responses from Oct 15th 2013 to Jan 22 2014.

Key questions

What were the key demographic characteristics of the audience?

• Only 30% were from the UK. 38% were from the US, with others spread
  around the globe, especially Australia and Canada.
• 73% were in their twenties, and 20% were over 30.
• We estimated that approximately 40% of applicants were not students.

Where do most coaching applicants come from?

• The most important source was personal referrals at 28% of applicants.
• Next, came the student groups in Oxford and Cambridge, which yielded 24% of applicants.
• Google search was a surprisingly common source at 16% of applicants.
• Two other important sources were the CFAR/LW community and Peter Singer’s TED talk.

How high achieving is the audience?
Our impression of the audience was that they were extremely ‘high achieving’ from the standpoint of intelligence and general prestige. It seems like we have a surprisingly high number of top young academics, entrepreneurs and charity workers applying.

This could have been biased because it has become known that 80,000 Hours coaching applications are highly competitive. Therefore it may be that many applicants who didn’t feel impressive did not apply.

How knowledgeable about effective altruism is our audience?
Approximately 1/4th of the applicants seemed to be very familiar with effective altruism, 1/4th somewhat familiar, and the other 1/2 seemed unfamiliar (see the ‘Knowledge of effective altruism’ rating below for more details). About 45% said that they support one of the causes common in the effective altruism community.

How altruistic is the audience?
They appear to be highly altruistic on average, with 30% pledging at least 10% of their income to charity and over half saying that positive impact is the main or only relevant factor in choosing their career. We might expect this to be biased upwards because it was obvious from the application which answers we’d prefer. From examining CVs qualitatively, we classified about ? of the audience as ‘highly altruistic’.

Is there a subsection of the audience who might be willing to pay for
coaching?
We’re interesting in the possibility of making part of the coaching self-funding. Our best guess was that the people who will be most willing to pay for coaching are people from tech and finance backgrounds aged 25-35. We found that about 20% of the requests fell in this category, which was higher than our expectations.

How has our audience changed over time?
There was a 0.17 correlation between audience ID (the order in which they joined) and the achievement score. Therefore it seems like the applications are becoming slightly higher in average achievement, which is a positive sign. However, it’s hard to draw firm conclusions because the period of time was very short (October 15th 2013 to Jan 15th 2014).

What were the most common types of question?
The vast majority of questions were about choosing careers. Approximately 30% of questions seemed directly focussed on optimizing social impact as opposed to improving the career from a personal perspective. Taking other parts of the applications into account, it seemed that most applicants primarily care about social impact. After reviewing these questions we came up with an alternative method of categorizing questions.

Additional findings

Doing this analysis required us to personally read each application and skim each resume. These were highly revealing.

1. Entrepreneurs, global ‘shapers’, international lawyers, genius geeks, and lots of otherwise different groups all share uncertainty but desire to do good in the world. Our audience base seemed diverse indeed.

2. Some applicants discussed frustrating experiences at the forefront of careers in several ‘ethical’ industries. For instance, several applicants experienced frustration at the difficulties of getting positions at international nonprofits (and some of these people spoke several languages and did diverse work on many different continents). These could represent very useful opportunities to learn from this community, perhaps in direct interviews.

3. Many of these applicants could probably benefit greatly from meeting each other. They are, on the whole, extremely intelligent and talented, but often confused. Many are looking for future startup or nonprofit co-founders. We’re not sure what organization or who should facilitate connections in the community, but this seems like a really valuable service.

4. 80,000 Hours’ coaching service attracts a large community that is not familiar with effective altruism. Much of this seems to be what is call the ‘Globalists’ below. This group seems to have very different goals and needs from the ‘Rationalists’. It may make sense for 80,000 Hours to either focus on one of these groups, or at least experiment more with the ‘Globalist’ group.

5. We noticed that a significant number of the applicants wanted to do one of the following:
   a. Consult international NGOs on effectiveness
   b. Create new social ventures to help the world, (often not particularly effective altruism inspired)
   c. Technology related to decision making, policy making, or global poverty
   This leads us to believe that impressive new organizations doing (a) or (c) may be able to find many excellent employees. It may be useful to create an incubator or social groups to encourage (b).

Statistics

Demographics and application questions

What is your age?

18-19: 7, 7.6%
20-24: 43, 47%
25-29: 24, 26%
30-34: 11, 12%
35+: 3, 3.2%

How did you hear about us?

Google Search: 13, 16.2%
Friend: 11, 13.8%
Effective altruist community: 11, 13.8%
80,000 Hours: Cambridge: 10, 12.5%
80,000 Hours: Oxford: 9, 11.2%
Peter Singer: 8, 10.0%
Facebook: 3, 3.8%
CFAR or Less Wrong: 5, 6.2%
The High Impact Network: 1, 1.2%
Bjorn Lomborg: 1, 1.2%
80,000 Hours: Harvard: 1, 1.2%
Ted: Ryan Coogler: 1, 1.2%
Other: 6, 8.4%

For me, making the world a better place is:

One of many considerations for choosing a career: 2, 2.2%
One of the most important factors in choosing my career: 42, 45.7%
The most important factor in choosing my career: 39, 42.4%
The only relevant factor in choosing my career: 9, 9.8%

How much time do you intend to spend on this career decision in total?

A couple of weeks of full-time work: 48, 52.2%
More: 26, 28.3%
A couple of days of full-time work: 13, 14.1%
A couple of hours: 5, 5.4%

Do you give any of your income to charity? If so, roughly how much?

Current Donation
0%: 20, 21.7%
1-9%: 45, 48.9%
10-30%: 14, 15.2%
31-50%: 8, 8.7%
Over 50%: 5, 5.4%

*Of the 5 who selected ‘over 50%’, 3 of them had backgrounds of philosophy.

How likely do you think it is that you’ll be supporting a different cause in three years time?
Very unlikely: 16, 17.4%
Unlikely: 39, 42.4%
Likely: 30, 32.6%
Very Likely: 7, 7.6%

What cause do you primarily expect to use your career to work towards over the next 5 years and beyond? Your cause is the problem you’ll use your career to help solve.

Developing world health: 13, 14.3%
Targeted interventions to help the long-run future: 12, 13.2%
Improving rationality and decision making: 11, 12.1%
Sustainability: 10, 11.0%
Developed world social interventions: 6, 6.6%
Increasing economic growth: 6, 6.6%
Other development interventions: 6, 6.6%
Other developing world causes: 5, 5.5%
Promoting effective altruism: 3, 3.3%
Science: 3, 3.3%
Other future causes: 2, 2.2%
Animal welfare: 2, 2.2%
Extreme poverty: 1, 1.1%
Other meta causes: 1, 1.1%
Global prioritisation: 1, 1.1%
Undecided: 5, 5.5%
Other: 4, 4.4%

What are your three main questions for our career coach?

We randomly sampled 20 responses, resulting in 57 questions. We categorized these into the question categories below.

The categorization process was surprisingly difficult. After completion we went back and analyzed the categories. We designed an improved category system, which is discussed in the question methodology section.

The order of the most popular question categories roughly matches that of the previous study, where each item was approximately within 3-4 places in the order of frequency as the previous study. However, this could have been biased by knowledge of the previous results.

In general, the questions seemed to focus on choosing types of careers, specifically for impact. Several different types of professions and causes were referred to, but there were no apparent trends.

Locations

Countries
US: 34, 37.4%
Britain: 28, 30.8%
Canada: 5, 5.5%
Australia: 5, 5.5%
Other, Europe: 9, 9.9%
Other, East Asia: 4, 4.4%
Other, Africa: 2, 2.2%
Other, Middle East: 2, 2.2%

Cities in England
Cambridge: 10, 35.7%
Oxford: 9, 32.1%
London: 5, 17.9%
Croydon: 1, 3.6%
Warlingham: 1, 3.6%

Regions of the United States
East: 19, 57.6%
Midwest: 7, 21.2%
West: 4, 12.1%
Florida: 2, 6.1%
Other: 1, 3.0%

Skill sets

From examining the responses and CVs of the applicants, we made a rough intuitive judgement of the skill sets and careers of the audience. ‘2’ indicates a strong fit, ‘1’ indicates a moderate fit, and ‘0’ indicates no fit. We ranked the categories in order of which had the highest number of ‘strong fits’. See the skill set methodology below for a full explanation of the skill set categories.

Science and Economics
0: 55, 61.8%
1: 14, 15.7%
2: 20, 22.5%

Technology
0: 58, 65.2%
1: 14, 15.7%
2: 17, 19.1%

Business
0: 47, 52.8%
1: 28, 31.5%
2: 14, 15.7%

Global (Experience working internationally)
0: 58, 65.2%
2: 18, 20.2%
1: 13, 14.6%

Philosophy
0: 74, 83.1%
1: 9, 10.1%
2: 6, 6.7%

Entrepreneurship
0: 78, 87.6%
1: 6, 6.7%
2: 5, 5.6%

Law
0: 79, 89.8%
1: 4, 4.5%
2: 5, 5.7%

Nonprofit Organizations
0: 52, 58.4%
1: 25, 28.1%
2: 12, 13.5%

Effective Altruist Organizations
0: 76, 92.7%
1: 5, 6.1%
2: 1, 1.2%

Medicine
0: 88, 96.7%
1: 1, 1.1%
2: 2, 2.2%

Humanities
0: 74, 85.1%
1: 6, 6.9%
2: 7, 8.0%

Journalism / Media
0: 87, 95.6%
1: 2, 2.2%
2: 2, 2.2%

Impression Data

In addition to the skill set data, we also tracked several general qualities, and rated the applicants accordingly. See the impression methodology section below for the reasoning and descriptions of each quality and score.

Achievement – Approximately how much has this person achieved given
their age?
0: 3, 3.4%
1: 30, 34.1%
2: 41, 46.6%
3: 14, 15.9%

Altruistic – Did this person demonstrate a substantial desire to
help others?
0: 5, 5.6%
1: 22, 24.7%
2: 32, 36.0%
3: 30, 33.7%

Knowledge of Effective Altruism – Did this person seem
knowledgeable about organisations and research in the effective
altruism community?
0: 44, 49.4%
1: 21, 23.6%
2: 24, 27.0%

Test Coaching Market – How well did this person fit the profile of
our test paid-for coaching market?
0: 30, 36.6%
1: 36, 43.9%
2: 16, 19.5%

Analysis

Correlation Matrix

We made a matrix of the correlations between each scalar metric. This
is shown below.

Interesting Correlations

Science Workers were noted as having high achievement (+0.19) and often have extra skills with software or engineering (+0.11 for Tech). On average they aren’t interested in Entrepreneurship.

Philosophers are knowledgeable of effective altruism (+0.23), are interested in working at EA organizations (+0.19) and donate a lot of their money (+0.37), but aren’t very global (-0.2) nor do they seem to have many other skills or interests.

Tech Workers mainly referred to people learning or working on software or engineering. They were rated as having high achievement (+0.38), and knowledgeable in effective altruism (+0.2).

Business work was one of the most popular options, which can be understood as it was a large field. It was correlated with accomplishment (+0.26), but didn’t have many other correlations.

Entrepreneurs were people who had started companies or were interested in making companies. On average they were high-achievement (+0.26), which is to be expected because it is difficult to start a company. They did not have many strong correlations with the other questions however.

Global workers highly correlated with Nonprofits (+0.12), Humanities (+0.22), Medicine (+0.144). They were low, on average, in knowledge of effective altruism (-0.31), Tech (-0.24), Business (-0.2), and Philosophy (-0.31).

Audience clusters

We noticed two basic clusters, though these were inferred from correlational data and were not incredibly strong. It would be useful to do an actual statistical cluster analysis in the future.

Cluster 1: Globalists
This group seemed to identify as ‘Globally Altruistic Individuals’.
Often they had specific causes they cared about. Seemed similar to the
TED community in terms of skillsets and intentions.
Skillsets: Law, Nonprofit, Humanities, Medicine (slightly)
Correlations: Average Achievement, Highly Altruistic, Very Low effective altruism knowledge, highly Global, high in ‘importance’ on the question ‘For me, making the world a better place is…’.

Cluster 2: Rationalists
This group seemed similar to the LessWrong community, many of whom
claimed to be influenced by that community. Often they were not very
set on a particular career, especially on the tech side (but not for
science students in advanced programs). Often choosing between tech,
entrepreneurship, and finance.
Skillsets: Tech, Science, Business, Entrepreneurship
Correlations: High Achievement and Average Altruism. High effective altruism for Tech and average effective altruism for Science, Business, Entrepreneurs.

Methodology

Data source and quantity

In October 2013 we made substantial changes to our Social Impact Coaching application form. Previously the form only contained 3 all-text questions. We decided to add several multiple choice questions and other shorter text questions for more specificity and to help make more objective decisions for acceptances. The form is viewable here.

Because this form was so different than the previous one, we decided to only take the responses from this new form to compare in this analysis.

It should also be noted that there were some modifications made to the application over the relevant time period. In particular, the question ‘How did you hear about us?’ was added in November (after about one third of the analyzed responses), and the specific categories for ‘Do you give any of your income to charity?’ had minor changes.

This left us with 91 responses from Oct 15th 2013 to Jan 22 2014.

Data cleansing

There were several stages of cleansing the data for analysis. For several of the application questions, a significant number of users selected ‘other’, often filling out a description of what that meant. In these cases, we interpreted the results into categories.

A few applicants submitted their applications twice and others had resumes that were either illegible or did not attach resumes (this was later made mandatory in the application). In these cases we would either exclude them from the entire analysis or just the skill set and
impression data.

Location of users were found based on the IP address, which was provided by Mailchimp. These results were put into categories based on geographical location and popularity.

General skill sets/career paths were interpreted from application text and resumes. Impression data was collected by a complete reading of applications.

Statistical analysis

Once the data was cleaned, we counted the frequencies of each response
option using a very simple python script. A Python script was written to accept a CSV file and output each option, it’s frequency, and the percentage of the total. The format is similar to that of the recent LessWrong survey analysis.

We tried correlating the results from each scalar metric (mainly, the skillsets and impression data). Some of the multiple choice survey questions were made into scalars by associating each answer with corresponding integers. The pearson statistical function on Excel was used to find the correlation between each scalar metric with each other scalar metric. This was done in one large grid. The pearson function gave values between -1 and +1 for each set of two different scalar metrics; however, the highest correlations found were approximately 0.5. In cases when we refer to a correlation above, it was a result of this analysis.

How Did You Hear About Us? methodology

This was originally a multiple choice answer with a separate space for a clarifying response. Two of the options were: ‘Word of mouth (please say who below)’ and ‘Other (please specify below).’ The responses from these were arranged into categories for analysis. ‘Effective Altruist Community’ refers to individuals who cited specific people who we knew and were familiar with effective altruism.

Main questions methodology

The question categories for this study were created for a previous report. Part-way through this analysis it was realized that the previous categories were poor fits for the existing data. The previous study put each question into one category, but this in practice proved difficult and arbitrary. For instance, the question, ‘Should I get a PhD in physics and pursue academia, or take a business position?’ could hypothetically fit into the ‘Evaluation of specific careers or plans’, ‘Best next steps, potentially with a long-term focus’, ‘Deciding about additional education’, ‘Direct advice/”tell me what to do”’, ‘Suitable careers, given individual background, mention impact’, ‘How to weigh options’, and ‘How to identify skills or careers” categories. Because of this, we’ve come up with a different system for future classifications.

This alternative question category system could evaluate each question on the following criterion, as applicable:

Topic: Career, Causes, Skill set, Lifestyle
Service: Validation, Survey, Comparison, Evaluation, Pursuance
Scope: Narrow, Broad
Timescale: Short-term, Long-term
Purpose: Effective Impact, Impact, Success
Specific Topics: Education

This would result in questions being classified in groups such as “Narrow Long-Term Career Survey for Success”, “Cause Evaluation for Effective Impact”, and “Broad Skill set Pursuance”. We would expect that most questions would deal with careers, but the other options would also be useful to have. One could imagine an applicant or user selecting each option from a form, and then being sent to a corresponding webpage or service.

Validation: Validate a specific choice
(Actual, simplified questions):
“Does this make sense?”
“Am I missing anything obvious?”
“Am I following the right career path?”
“Are there any other options I should consider?”
“Are there other options that could be optimal that I might be missing?”
“Is X the best way for me to make an impact? (or should I consider something else)”
“Is there another career path where you think I would have more impact?”

Survey: Give choices
“What are my options having studied X?”
“How can I have an impact whilst at City X?”
“What high impact careers are available to X PhDs?”

Comparison: Choose one choice out of 2-4 choices
“What is my most realistic option of the ones I listed?”
“Should I be focussing on developing skills to do X or Y?”
“Is the most effective way to do Z through A or B?”

Evaluation: Learn about a specific career/choice
“Are development economics majors in demand?”
“How difficult is it to get a highly-paid position in Law?”

Pursuance: How to get into a career, or do it well
“What ways exist to parachute into public policy without tedious and value-compromising struggle inside a political party?”
“What steps should I take now to pursue a high-impact career?”

Skill sets methodology

The skill sets data was collected by reading the applications along with the resumes or CVs of each applicant. Originally there were only 5 categories (Technology, Entrepreneurship, Global, Philosophy, Science), but others were added reading through the first applications. These categories were not mutually exclusive, and often one person would score points in 3-5 different categories.

To be clear, applicants did not directly provide this data via multiple choice or tick boxes, but instead we inferred it from the written information we were provided with.

The basic scale was as follows:
0: No mention
1: Mention for either experience or intention
2: Substantial experience or primary intention

There were definitely a large number of judgement calls in order to establish this data. The choice between giving a candidate a 1 or 2 in a category at times felt a bit arbitrary.

The categories themselves were highly specific to this group and what the investigation
was interested in. While a few of them seemed specific (Medicine, Journalism), many others were established to suit the data. These are explained below.

Science/Economics: This group was primarily for people studying or working in the fields of science or economics outside of computer science, electrical engineering, and medicine. Many of these individuals were either Undergraduates or Postgraduates in Physics, Math, Economics, Psychology, Decision Making, or Chemistry.

Technology: This primarily referred to software development or engineering. This included data analysis skills if they related to programming languages like R and Python.

Global: We noticed that many of the applications seemed particularly well versed in multiple languages or interested in doing work in a variety of countries. This group was so large and noticeable that we created a special category to help distinguish them. Scoring on this category was done based on how much an individual’s resume emphasized or highlighted experiences in several countries, in particular ones on different continents. To give some examples, this group included one person who took a gap year visiting over 30 countries, and several who did global aid work in Africa, Asia, and South America.

Business: This was for people who had work experience or interest regarding to what we deemed to be generally understood as ‘business’. This included the corporate positions, finance positions, sales positions (including local retail), consulting, and small business (though not owned directly). In part because it was such a broad term almost half of applicants had some experience in business, often as part of a summer internship or short time position. Perhaps in future studies this could be broken up into different categories, though we didn’t notice any obvious sub-categories from the study.

Philosophy: We separated this group from the Humanities because there was a particularly high number of applicants with experience in philosophy. This mainly pertained to people studying Philosophy as an Undergraduate or Postgraduate.

Social: This category is for people who listed doing volunteer or professional work at a nonprofit or related enterprise.

EA Org: This category was for individuals who had experience interning or working for an effective altruist organization. This included college organizations that were set up to promote some kind of effective altruism.

Humanities: This field was mostly for individuals with experience in literature, anthropology, or sociology. It did not include economics or philosophy.

Impression Methodology

Impression data categories were extremely subjective and were chosen after all data collection. We wanted to have a general sense of how accomplished our audience was, how altruistic they were, and how familiar they were with effective altruism. In addition, the team is considering a very specific market for future expansion and we wanted to see how much of the existing applicant pool fit this.

These categories were only chosen for this particular individual study. The methodology to choose applicants for case studies was very different.

Achievement
0: A bit less achievement than a typical college student per age. This rating was given very rarely.
1: College student or similar. Approximately average for a college student.
2: Very accomplished for the age. This often meant good grades at a very good or top school, followed with a list of outside activities and internships.
3: Extremely accomplished for the age. Substantially more so than the
standard for a 2. This could mean near-perfect grades from an incredibly reputable school (4.0 from Harvard for instance, personally funding notable organisations, or/and doing other incredibly prestigious work.

Test paid-for coaching market
80,000 Hours is considering monetizing a portion of our coaching service in the future. To do this, we decided to focus on a specific market. After consideration it was decided that the best initial test market may be people in the tech, finance, entrepreneurship, and business careers who are roughly between the ages of 25-35, and portray a willingness to change careers and/or focus on strategic cause selection. This is because this audience seemed to be fairly flexible and often seems to have enough money to pay for coaching services.

In order to better understand the size and characteristics of this group, we rated all applicants on how well they fit this specific market.
0: Definitely not in the target market.
1: Close to the target market.
2. Fits the target market.

Altruism
The original plan for this category was to try to understand how much each individual identified as an ‘altruistic’ person. This came out to be incredibly difficult to judge from the resumes and applications. For example, many graduate students specialized in areas that seemed concerned with the public good. However, most graduate specialties can be related to the public good, so it can be very difficult to judge how altruistic the
student was who chose them.

Partially because of this, much of the rating here depended on other
categorical information. For example, the questions of how
much an individual donated and how important altruism was for their
career. Therefore this category may not have been very useful and
shouldn’t be taken with much precision.
0: Little or no information that indicated intended altruism.
1: Some hints of altruism, for example some volunteering.
2: Altruism seems like a primary factor, perhaps mentioned several
times.
3: Most information, including life career, seemed guided by altruism.

Knowledge of Effective Altruism
0: No mention of effective altruism or related concepts.
1: 1-3 mentions.
2: Discussed in detail or seemed to be particularly knowledgeable about the ideas and organisations in the effective altruism community.