The camel doesn’t have two humps – update to software engineering profile

In our current software engineering profile, we say:

Programming ability seems to roughly divide into two groups: those who find it relatively easy and those who don’t. If in the past you’ve done well at mathematics and science and can think abstractly, then it’s likely you can learn to program well enough to get an entry-level job within about six months.

In evidence of the first claim, one piece of evidence we cited was a paper called “The Camel Has Two Humps” by Dehnadi and Bornat.

However, we’ve just discovered that Bornat has publicly retracted this paper. He says:

It’s not enough to summarise the scientific result, because I wrote and web-circulated “The camel has two humps” in 2006. That document was very misleading and, in the way of web documents, it continues to mislead to this day. I need to make an explicit retraction of what it claimed. Dehnadi didn’t discover a programming aptitude test. He didn’t find a way of dividing programming sheep from non-programming goats. We hadn’t shown that nature trumps nurture. Just a phenomenon and a prediction.

Though it’s embarrassing, I feel it’s necessary to explain how and why I came to write “The camel has two humps” and its part-retraction in (Bornat et al., 2008). It’s in part a mental health story. In autumn 2005 I became clinically depressed. My physician put me on the then-standard treatment for depression, an SSRI. But she wasn’t aware that for some people an SSRI doesn’t gently treat depression, it puts them on the ceiling. I took the SSRI for three months, by which time I was grandiose, extremely self-righteous and very combative – myself turned up to one hundred and eleven. I did a number of very silly things whilst on the SSRI and some more in the immediate aftermath, amongst them writing “The camel has two humps”.

Based on this, we’ve removed the paper from the profile, and removed the claim about the distribution dividing into two clumps.

We intend to do a more thorough review of the predictors of success in this field when we release our full profile of software engineering in the new year.

Did we make a mistake in this case? The profile was only at the “considered” stage, so not the result of in-depth research. Even so, when most skills and abilities are normally or log-normally distributed, we should have been cautious about the existence of a bimodal distribution without relatively strong evidence.

  • Richard Batty

    Interesting comment from Alan Kay on the paper: http://secretgeek.net/camel_kay

    “I saw this a few years ago. They could be right, but there is nothing in the paper that substantiates it.

    (How to do a short reply here?)

    Notion 1: Good science can rarely be pulled off in an environment with lots of degrees of freedom unless the cause and effect relationships are really simple. Trying to assess curricula, pedagogy, teaching, and the learners all at once has lots of degrees of freedom and is *not* simple.

    So for example we’ve found it necessary to test any curriculum idea over three years of trials to try to normalize as much as possible to get a good (usually negative) result.

    Notion 2: Most assessments of students wind up assessing almost everything but. This is the confusions of “normal” with “reality”.

    For example, in our excursions into how to help children learn powerful ideas, we observed many classrooms and got some idea of “what children could do”. Then I accidentally visited a first grade classroom (we were concerned with grades 3-6) in a busing school whose demographic by law was representative of the city as a whole. However, every 6 year old in this classroom could really do math, and not just arithmetic but real mathematical thinking quite beyond what one generally sees anywhere in K-8 [kindergarten and grades 1 through 8].

    This was a huge shock, and it turned out that an unusual teacher was the culprit. She was a natural kindergarten and first grade teacher who was also a natural mathematician. She figured out just what to do with 6 year olds and was able to adapt other material as well for them. The results were amazing, and defied all the other generalizations we and others had made about this age group.

    This got me to realize that it would be much better to find unusual situations with “normal” populations of learners but with the 1 in a million teacher or curriculum.

    I found Tim Gallwey, who could teach anyone (literally) how to play a workable game of tennis in 20 minutes, and observed him do this with many dozens of learners over several years.

    I found Betty Edwards who could teach (again literally) anyone to draw like a 2nd year art student in one intense week.

    And so forth, because what the exceptional teaching is doing is actually allowing assessment of what general human beings from a typical bell curve can learn from crafted instruction.

    And, I think some of the keys here are in the metaphor of bell curve. Students will exhibit distributions of talent, motivation, learning skills, style, etc., and one will see these show up right away in any simple-minded form of instruction and curriculum.

    But if the battle cry is “Learner’s First”, then what we really want to know is what can be done to help the different types of learners. Some don’t need any help. Some need to learn some things before they tackle the main subject. Some need to be shown different POVs so they can see a route for them to learn.

    Really good teachers want to get all the students to be fluent, and they often find ways to do this. “Regular” teachers often just want to get through the material. Some school systems want to use education to sort the population rather than to educate the whole population. Etc.

    I don’t know the general answers here, but our research groups in the mid-70s [presumably the Learning Research Group at Xerox Parc] set a goal of 90% fluency for (say) 10-12 year olds, and then we proceeded to fail to achieve this until about 1998, when enough things had been done in the computer environment to provide hooks to many different kinds of children without losing the essential high quality of powerful ideas that was our goal.

    I think as a teacher, one has to embrace the bell curve idea and be prepared to deal with at least three tiers of preparedness in the students. One could hope that a lot more general prep about thinking and symbolizing would have happened in K-12, but it doesn’t in the US for sure.

    There has been some very interesting work with respect to science teaching that seems parallel here (for example, by Tinker and others at Tufts). They not only found a pretest (could they interpret various kinds of graphs?) that would predict the grades of the 1st year physic students, but found that teaching the kids skills in doing well on the pretest (using some very creative ideas that Jerome Bruner would find familiar) would also vastly improve their performance in the physics class itself.

    So the pretest was not just testing, but also finding some forms of relational and figurative thinking that some of the students needed skills in, before tackling physics.

    I think every musician who is reading this will know what I’m driving at here. Music is a lot of skills and types of thinking and few musicians are naturally good at all of them. The desire to be a musician plus decent music instructors will find the things each learner will need to work on to get fluent. The result is that most skilled musicians can play advanced stuff, but they are all rather different on their outlook, how they practice, what they practice, etc.

    (Sports and art also … and almost certainly the more holy subjects sanctified by society, and those pretenders to the throne such as computing ….)

    Cheers,

    Alan”

  • Ryan Carey

    I agree, although the test remains a reasonable predictor of programming ability, probably:
    “Neither bombastic elation nor depressive rejection was a correct response. Dehnadi, to his credit, stuck to his guns and did the meta- analysis that showed that he’d discovered a phenomenon and that his test was a worthwhile predictor. He successfully defended his thesis (Dehnadi, 2009) and we published a summary of his results (Dehnadi et al., 2009).”