Home

A Working Definition of Expertise



The common wisdom: people who have studied something long enough, and understand its inner workings, are experts on it.

Expertise, by my definition, is probably not what you think — and you don’t know any experts.

What is an Expert?

A Short story:
I was shadowing an airplane mechanic, walking around the hanger/runway, asking task analysis questions: “how do you communicate to the machine floor manager? How do you know you have completed a job?” etc. Then something interesting happened. The guy i’m walking around with looks up in the sky and points out a plane. The plane was impossibly high in the sky, it looked like a spec, motionless. “That’s a Cessna Skyknight,” he casually tossed out, “you don’t see them very much here.” I asked him how he could possibly identify the plane based on the cookie crumb-like object we spotted several miles away. He gave a bunch of nothing answers like, “you know it when you see it.” I returned to the hanger, and asked who i could talk to about planes in the airspace. Of course they tracked all of this info. And, of course they would NOT give it to me. I did however get the flight log owner to confirm for the time, location, and travel direction — there was in fact a Cessna Skyknight on the log — and that was, in fact out of the ordinary. I also confirmed that my interviewee did not have access to those logs. So, how in the hell did he know?

That experience (and others like it), have lead to my working definition of expertise:

Having expertise is when you can discern more information in the situation than the evidence alone provides.

Consider a spectrum of problems requiring expertise:

  • Some problems don’t require any information beyond what is available, like tying your shoes. Expertise is not required in these cases. (you are thinking, “what about being able to tie them really well so they stay, or double knot them, etc.?” I’ll get to that in the next section.)
  • There are problems that require some information beyond what is available, like navigating a foreign city with signage in another language. Navigation expertise might have you following street A instead of B, as the expert mysteriously knows that A will lead to less options, or the wrong way entirely.
  • There are problems that absolutely require expertise for success, like entrepreneurship — where you find yourself in that murky spot, having to make something out of nothing for the business to succeed.
  • Still there are problems that rely heavily on split second expertise — yet only at certain times, like driving home from work. In that moment just before an accident, seeing more information in the circumstance than is immediately present could save your life.

What about expert shoe tying or typing?

Cleverly tying your shoes with interesting knots falls somewhere between art and style — it doesn’t require expertise. In fact being, say, a good typist also doesn’t require expertise, it requires skill. Most activities result in success with an application of skill. I’d argue that skill is probably more useful than expertise. Skill also has two very nice qualities: you can (reasonably) measure how much a person has, and it’s hard to fake. Expertise is the opposite though, it’s hard to measure — isn’t it? (<- you’ll get that joke later, it’s #5)

How much expertise does one have?

An attractive part of thinking about expertise like i do: it could be measured…

Persons A and B, having 2 years and 20 years of study in a particular problem respectively, could observe a circumstance containing a problem to be solved. Persons A and B list out — or otherwise note — all information they can glean from the scenario. Then, removing all information that was present in the available evidence, you could tally remaining factual observations, yielding a score of sorts. More tallies is more expertise.

The measurement setup described above would also allow you to test within the spectrum of problems requiring expertise, normalizing for those times when expertise is needed versus not. However, measuring expertise in that way is fraught with errors. How do you objectively correct for observation overlap? How do you determine what information (as observed) was outside the scope of the available evidence? How do you rank the importance of information — perhaps factual — but meaningless? Not small questions.

Expertise or just good guesswork?

Putting measurement of expertise aside, let’s focus on characterization. Given most problem oriented circumstances, the solver is either going to see information beyond present evidence or not. The question pattern then follows:

  1. Does the circumstance require expertise to successfully solve the problem
  2. Did the subject report information beyond present evidence
  3. Was reported information used to solve the problem
  4. Was the subject successful in solving the problem

Four “Yes” answers would indicate this person is an expert — but they also could just be good at guessing (perhaps even an expert guesser …). Here is what i propose:

The test of expertise is not one test — you have to test for expertise all along the spectrum, testing circumstances not requiring expertise on to absolutely requiring expertise.

My hypothesis is that good guesswork is only useful in the middle of the ‘requires expertise spectrum’ — where finding successful solutions could benefit from good guessing, yet don’t actually require expertise.

Fakers

A 2009 Op-ed in the NYTimes described a set of research studies where an actor, introduced as a well known authority, gives a fake lecture about complex sounding (nonsense) topics to a group of scholars — when asked to review his performance, they give him good marks. Appeal to authority is the punch line. But “well known authority Dr. so and so” is only half the equation. The actor delivered the lecture “full of jokes and interesting neologisms.” He was likable in the context they saw him. The short article also goes on to talk about Philip Tetlock’s (UC Berkeley) research on accuracy of expert predictions, opinions, etc. Spoiler alert: he finds most are no better than guessers, regardless of education, experience, etc. The only significant predictor: the more famous, the worse the expert. There are many research studies debunking experts (Lehrer 2012 for example).

Fakers, con artists and ego obsessed know-it-alls alike have spent years honing their craft of looking like an expert. They know all the tricks:

  1. Comfortably delivering slightly unknown content: conversations/talks are “full of jokes and interesting neologisms.”
  2. Telling you how long they’ve been ‘doing this:’ “I’ve been doing this for 15 years.”
  3. Often employing meaningless reverse comparisons: “if we were in the opposite situation, we would feel the same way about X but Y would be completely different” — where Y relates only to what they want you to do/think/believe.
  4. Using vague language to describe a circumstance: “that option does not make good on our mission of innovation.”
  5. Asking for agreement or assuming you agree with them: “obviously X is not correct. right?”
  6. Claiming to know ‘data’ yet won’t give it to: “I know exactly how much money we’ve spent on travel this year!”
  7. Supporting their argument with data you couldn’t possibly confirm: “we sell a daily average of 1,500 tickets!”

Under my definition of expertise, you either see/report information (not present to a skilled ‘non-expert’) in the form of observations that lead to correct decisions — OR YOU DON’T.

Are you a faker?

From childhood we have internalized the hallmarks of so-called experts we aspire to be. The list of 7 top con artist tricks (above) just rolled off my tongue — because i catch myself doing all of them. This is a side point, but i attribute wisdom to people that don’t commit the obvious infractions listed above AND actively condemn the acts when they see them. I don’t do this myself, making me unwise — or perhaps just too nice.

Experts are overrated

Focus, effort, and genuine interest in a problem are predictors of success. Adding skill to that equation improves odds. Expertise alone, in practice, probably yields mixed results.

How do you gain expertise?

I have no idea how you ‘learn’ to be an expert. It’s tempting to conclude that a combination of experience and practice naturally results in expertise. A book (Malcolm Gladwell’s “Outliers”) and a commonly quoted study (Ericsson et al. 1993) to that effect — the so-called ‘10,000 hours’ idea— promotes the theory that focused practice results in expertise. The ‘10,000 hours’ idea was not completely aimed at expertise, let alone the strict way in which i am defining it. At any rate, many have debunked the ‘practice makes expert’ theory (Shaunacy Ferro wrote a fine review of recent debunkers), leading us to wonder about the role of experience.

Kind of similar, but not exact experiences seem good for expertise building.

In “Make Yourself an Expert” (2013) the authors write about management problem solving:

He simply recognized patterns from experience and applied solutions that had worked well in the past. It was second nature to him, like managerial muscle memory.

Having the smarts to apply successful past tactics and avoid past pitfalls no doubt contributes to solving problems — but past experience doesn’t naturally lead to seeing more information in the circumstance than is available in the immediate evidence. However, if the pattern of the problem or situation looks familiar yet doesn’t exactly fit, the ill fitting parts (from your past experience) are not present in the current situation … and might lead to information beyond present evidence, like puzzle pieces that haven’t shown themselves yet.

This discussion of experience is really incomplete. I don’t have any good thoughts on how the process of building your holistic cognitive pattern from an experience translates to how one might match that pattern to a present circumstance. So, I’m closing up.

Here’s the punchline…

I think experts, fitting my definition, are rare. I don’t know any personally. I sure as hell am not one of them.

In the way i’m defining it, it would be strange if somebody claimed to be an expert. It’d be like a Wizard, never having heard wizards, claimed to be a wizard.

Finally, i’d also bet good money that you don’t know any experts.