What is an Expert?

I’ll tread carefully here, because we live in a strange time of questioning the motives and knowledge of experts to bolster every bizarre conspiracy theory under the sun. No one trusts any information anymore. It’s not even clear if trusting/doubting expert opinion is anti/hyper-intellectual. But that isn’t the subject of today’s topic.

I listen to quite a few podcasts, and several of them have made me think about expertise recently.

For example, Gary Taubes was on the Sam Harris podcast and both of them often get tarred with the “you don’t have a Ph.D. in whatever, so you’re an unknowledgeable/dangerous quack” brush. Also, Dan Carlin’s Hardcore History podcast is insanely detailed, but every ten minutes he reminds the audience “I’m not a historian …”

Many people who value the importance of expertise think that the degree (the Ph.D. in particular but maybe an MFA for arts stuff) is the be-all-end-all of the discussion. You have the Ph.D., then you’re an expert. If you don’t, then you’re not.

The argument I want to present is that if you believe this, you really should be willing to extend your definition of expertise to a wider group of people who have essentially done the equivalent work of one of these degrees.

Think of it this way. Person A goes to Subpar University, scrapes by with the minimal work, kind of hates it, and then teaches remedial classes at a Community College for a few years. Person B has a burning passion for the subject, studies all of the relevant literature, and continues to write about and develop novel ideas in the subject for decades. I’d be way more willing to trust Person B as an expert than Person A despite the degree differences.

Maybe I’ve already convinced you, and I need not go any further. Many of you are probably thinking, yeah, but there are parts to doing a degree that can’t be mimicked without the schooling. And others might be thinking, yeah, but Person B is merely theoretical. No one in the real world exists like Person B. We’ll address each of these points separately.

I think of a Ph.D. as having three parts. Phase 1 is demonstration of competence of the basics. This is often called the Qualifying or Preliminary Exam. Many students don’t fully understand the purpose of this phase while going through it. They think they must memorize and compute. They think of it as a test of basic knowledge.

At least in math and the hard sciences, this is not the case. It is almost a test of attitude. Do you know when you’re guessing? Do you know what you don’t know? Are you able to admit this or will you BS your way through something? Is the basic terminology internalized? You can pass Phase 1 with gaps in knowledge. You cannot pass Phase 1 if you don’t know where those gaps are.

Phase 2 is the accumulation of knowledge of the research done in your sub-sub-(sub-sub-sub)-field. This basically amounts to reading thousands of pages, sometimes from textbooks to get a historical view, but mostly from research papers. It also involves talking to lots of people engaged in similar, related, or practically the same problems as your thesis. You hear their opinions and intuitions about what is true and start to develop your own intuitions.

Phase 3 is the original contribution to the literature. In other words, you write the thesis. To get a feel for the difficulty and time commitment of each step, if you do a five year Ph.D., ideally Phase 1 would be done in around a year, Phase 2 is 2-4 years, and Phase 3 is around a year (there is overlap between phases).

I know a lot of people aren’t going to like what I’m about to say, but the expertise gained from a Ph.D. is almost entirely the familiarization with the current literature. It’s taking the time to read and understand everything being done in the field.

Phase 1 is basically about not wasting people’s time and money. If you’re going to not understand what you’re reading in Phase 2 and make careless mistakes in Phase 3, it’s best to weed those people out with Phase 1. But you aren’t gaining any expertise in Phase 1, because it’s all just the basics still.

One of the main reasons people don’t gain Ph.D.-level expertise without actually doing the degree is because being in such a program forces you to compress all that reading into a small time-frame (yes, reading for three years is short). It’s going to take someone doing it as a hobby two or three times longer, and even then, they’ll be tempted to just give up without the external motivation of the degree looming over them.

Also, without motivating thesis problem, you won’t have the narrow focus to make the reading and learning manageable. I know everyone tackles this in different ways, but here’s how it worked for me. I’d take a paper on a related topic, and I’d try to adapt the techniques and ideas to my problem. This forced me to really understand what made these techniques work, which often involved learning a bunch of stuff I wouldn’t have if I just read through it to see the results.

Before moving on, I’d like to add that upon completion of a Ph.D. you know pretty much nothing outside of your sub-sub-(sub-sub-sub)-field. It will take many years of continued teaching and researching and reading and publishing and talking to people to get any sense of your actual sub-field.

Are there people who complete the equivalent of the three listed phases without an actual degree?

I’ll start with the more controversial example of Gary Taubes. He got a physics undergrad degree and a masters in aerospace engineering. He then went into science journalism. He stumbled upon how complicated and shoddy the science of nutrition was, and started to research a book.

Five years later, he had read and analyzed pretty much every single nutrition study done. He interviewed six hundred doctors and researchers in the field. If this isn’t Phase 2 of a Ph.D., I don’t know what is. Most students won’t have gone this in-depth to learn the state of the field in an actual Ph.D. program.

Based on all of this, he then wrote a meticulously cited book Good Calories, Bad Calories. The bibliography is over 60 pages long. If this isn’t Phase 3 of a Ph.D., I don’t know what is. He’s continued to stay abreast of studies and has done at least one of his own in the past ten years. He certainly has more knowledge of the field than any fresh Ph.D.

Now you can disagree with his conclusions all you want. They are quite controversial (but lots of Ph.D. theses have controversial conclusions; this is partially how knowledge advances). Go find any place on the internet with a comments section that has run something about him and you’ll find people who write him off because “he got a physics degree so he’s not an expert on nutrition.” Are we really supposed to ignore 20 years of work done by a person just because it wasn’t done at a University and the previous 4 years of their life they got an unrelated degree? It’s a very bizarre sentiment.

A less controversial example is Dan Carlin. Listen to any one of his Hardcore History podcasts. He loves history, so he obsessively reads about it. Those podcasts are each an example of completing Phase 3 of the Ph.D. And he also clearly knows the literature as he constantly references hundreds of pieces of research an episode off the top of his head. What is a historian? Supposedly it’s someone who has a Ph.D. in history. But Dan has completed all the same Phases, it just wasn’t at a university.

(I say this is less controversial, because I think pretty much everyone considers Dan an expert on the topics he discusses except for himself. It’s a stunning display of humility. Those podcasts are the definition of having expertise on a subject.)

As a concluding remark/warning. There are a lot of cranks out there who try to pass themselves off as experts who really aren’t. It’s not easy to tell for most people, and so it’s definitely best to err on the side of the degree that went through the gatekeeper of a university when you’re not sure.

But also remember that Ph.D.’s are human too. There’s plenty of people like Person A in the example above. You can’t just believe a book someone wrote because that degree is listed after their name. They might have made honest mistakes. They might be conning you. Or, more likely, they might not have a good grasp on the current state of knowledge of the field they’re writing about.

What is an expert? To me, it is someone who has dedicated themselves with enough seriousness and professionalism to get through the phases listed above. This mostly happens with degree programs, but it also happens a lot in the real world, often because someone moves into a new career.


The Myth of a Great Seminar

Sometimes I peruse the debates at Intelligence Squared to see if any catch my eye. There was one this time that seemed really interesting to me. It was a debate on whether or not MOOCs are reasonable replacements for actual in-class and campus college experiences. You can see the full thing here.

This was interesting to me, because I’ve actually gone through a few MOOCs from start to finish and found them to be extremely good experiences. I was curious if there was research that would be mentioned about the effectiveness of one or the other. The debate was pretty disappointing in this regard. The main anti-MOOC argument was based around how wonderful small seminars are and that you can’t get this in a MOOC. That’s why I want to write a response to this mythical seminar.

Before talking about why I think such seminars don’t really exist in this Platonic, pristine state at any university, I want to first address the fact that the existence of seminars at all is pretty mythical. I decided to check the University of Washington’s Spring 2014 schedule. The senior level literature classes had a student range of 25-40, but most were about 30. Should I consider a 30 person class a “small seminar?” I get it. We’re a gigantic school, so I fully admit that small liberal arts colleges probably do have a lot of small seminars. But most students at most schools will graduate with few to no small seminars as their classes.

Even middle level courses like Introduction to the Theory of Literature at Ivy League schools are gigantic. That class probably has 100 students or more in it, and those are the types of courses that are offered as MOOCs. I think the comparison is a bit disingenuous when you take some capstone seminar and compare it to an “intro” MOOC. The MOOC side of the debate also responded to this criticism and pointed out that some MOOCs offer small group breakout sessions which actually do simulate small seminars. So the point doesn’t even stand.

Now that that rant is over, let’s pretend like the comparison is fair. Here are some of the myths I heard and why I think they are mostly myth (I’ll grant that maybe a few seminars run according to plan):

Let’s suppose for the sake of argument that the teacher is practically invisible in this mythical seminar and the students are all enraptured in high level critical conversation about Dostoevsky or some such nonsense. This seems to be the ideal the seminar aspires to. This is going to sound extremely cynical, but just how interesting can this conversation actually be? The seminar is going to be made up of an incredibly homogeneous group. Everyone is going to be about 20, never having had to make a living. They are all educated at the same school, which means they have roughly the same cultural experience, read the same books, and developed the same theories about how to analyze books.

What’s so great about this perfect conversation in comparison with a MOOC? When you take the exact same course as a MOOC, you will probably have a math professor in India, a farmer in the American midwest, a retired middle school teacher in Scotland, etc. The conversation about the same books is going to be infinitely more interesting and enlightening, because the perspectives will be so varied.

Now let’s back up a little from the perfect situation and get a little more realistic. We’ve all been to these seminar classes before. The free-flowing and enlightening conversation essentially never happens. You have some people who didn’t read the stuff. You have people who aren’t very good at articulating their thoughts on the spot. The whole thing usually turns into the professor calling on someone, a brief sentence or two is mumbled, and then the professor carries on along that point. The “conversation” is forced, and the student input is more like a prompt for the professor to riff on.

Depending on the day and material, the degree to which this is the case will vary, but I think the overall sentiment is what happens most days in most seminars. This is actually why I think a written discussion board in a MOOC is actually a far better method for discussion than a conversation in a seminar.

First off, there are hundreds of more topics and conversations going on at a discussion board than in class. This means that you can search around for conversations that you really want to participate in. Second, you have to write your thoughts down. This gives you time to figure out what you are going to say rather than awkwardly spewing out some muddled nonsense while everyone stares at you. It also gives you time to figure out what other people mean before responding to them.

It is amazing the number of times you start typing a response, and then when you go back to what was actually said you realize you misunderstood at first. Which brings me to my next point. A discussion board records all of it. You can continually return to conversations as your understanding of a topic develops. The conversation doesn’t end at the end of the hour. Once you leave the physical setting of a seminar, it probably only takes a few hours to forget most of what most people said. The discussion board allows you to go back whenever you want to recall certain parts of certain conversations.

To summarize, I think most courses most people take are not seminars, so it is pointless to use them as a main argument against MOOCs. I also think that the MOOC setup is actually a better platform for enlightening discussion in almost every respect than an actual seminar. That being said, I think the anti-MOOC side has a point when they say that communication skills are developed in class discussion. Unfortunately, even small seminars tend not to have real “discussions,” so I don’t find that compelling (along with the fact that some MOOCs are incorporating small group live chat sessions now).

Don’t get me wrong. I don’t think all university education should be relegated to the online setting. I’m just saying that using some idealized small seminar as the main argument is a highly flawed way to go about it.

Room 237 and Some Post-Modern Problems

I think everyone involved in academia should see Room 237 but for some strange reasons. The movie is a fascinating look at some people who, to put it mildly, are obsessed with Stanley Kubrick’s movie The Shining. They’ve developed all sorts of theories about hidden messages in the film. Is it secretly a metaphor for the genocide of Native Americans? Is it secretly about the Holocaust? Is it Kubrick trying to tell us he faked the moon landing for the government? Is it a subliminal message that Danny was abused by his father and then he kills him for it? Is it meant to tell us the entirety of human history and how to surpass it? Is it a retelling of Theseus and the Minotaur?

I recommend you watch the movie with suspended disbelief to really try to get inside the heads of these people. It will make the movie much more fun. Once it is over you should then pop on over to David Segal’s New York Times article on it for a healthy dose of skepticism. But here’s the point. The movie should be used as discussion starter in academia on some issues that get swept under the rug, but used to keep me up at night (and now they do again after seeing this movie and it all came rushing back).

I’ll say up front that I’m going to open a big can of worms and not offer any sort of solution. If this is going to frustrate you, then you can stop reading now. To explain the issues, I’ll start in academic fields that are easiest to pick on like the fine arts and more specifically “critical theory.” Let it be known that since these issues are actually discussed there, I actually think they are in better shape for facing them. We’ll then discuss how they arise in “objective” subjects like math. Here it is much more dangerous because people will outright deny these same issues exist. I personally think these are issues that cut across every discipline in the university (except maybe the experimental hard sciences).

For the purposes of this post I’ll define post-modernism as the philosophical position that an interpretation of something is valid as long as it can be supported by a sound argument involving some type of evidence from the work being interpreted. Two things immediately spring to my mind with this definition. First, this idea is the bread-and-butter (at least at the undergraduate level) of what is taught in universities. We actually reward papers that take risks with original and maybe even controversial interpretations as long as the paper that gets turned in uses sound logic, is well-written, and supports its arguments with evidence. It is like we are training our students to make connections where none exist and become future conspiracy theorists.

This brings us to the second point. Even though on the surface post-modernism seems like a totally reasonable idea (again, all of academics seems based on it), Room 237 really brings to light why we might want to be a bit more cautious. Post-modernism tells us that every single one of those interpretations in the movie are valid. Just think about some student writing down the fake moon landing interpretation for an intro to film studies class. That student will get an A+ on that paper. As the New York Times article points out, basically all of the symbols and details that support that theory were mere accidents or conveniences.

Since this is a theoretical discussion, let’s do a thought experiment where we know beyond any reasonable doubt that Kubrick did not intend in the slightest to allow this interpretation. In what sense then is that interpretation “valid?” To put the problem much more bluntly, let’s take any work of art that is reasonably robust. If you have enough time, are well-versed in symbolism, and are fairly clever, then you can probably take any bizarre theory you want and connect the dots of the work to argue convincingly for that interpretation.

More specifically, if a work can mean anything, then the work means nothing. Someone might try to get out of this problem by saying that an interpretation is valid if in addition to the evidence from within the work some evidence from outside the work is provided to show some sort of plausibility that the interpretation could have been intended by the artist.

I think even pre-modern theorists probably rejected this “fix” as too narrow, because a work of art can’t have no valid meaning outside of the intent of the artist. Anyway, I think the problem in the fine arts departments has been addressed and I promised to point out how this cuts across all academic disciplines, so we’ll move on.

If we phrase the problem slightly differently it becomes clear how the problem translates. We’ll rephrase post-modernism to mean that a connection between two things is meaningful if a sound argument can be made showing how they are connected. We recover the art version of the definition by saying the two things are the work and the interpretation. When talking about math, the phrase “sound argument” should just be read as a proof that the two mathematical objects/theorems/ideas/theories/whatever are related.

I know at this point some mathematicians are scoffing. If you prove they are related, then of course they are related. Why care about such value judgments as whether or not it is “meaningful.” I don’t want to say whether or not we ought to care about such things, but the fact is that in current mathematical culture we do care about such things. Also, we could change the word meaningful back to valid to try to avoid value judgments, and I think the problem still exists. Here’s an example.

Mathematicians often use the term “deep.” This means roughly that the connection is both meaningful and difficult to establish. The term cannot merely mean difficult to establish, because with very little thought one can come up with an extreme example of a difficult to establish connection that would be written off as ridiculous and frivolous. For example, the proof might be exceedingly long and include steps that are totally arbitrary like adding 1 to every coefficient of some Fourier series to get a new function and taking the value of the function at 12 to get 145926144000 and noting that there is only one simple group of that order whose double cover is related to the Gaussian integers and so on.

Of course this is an extreme example, but now let’s just pare back the arbitrariness of this example or extend the length and number of somewhat unrelated steps of the “deep” theorem to get to a middle ground. It becomes much less obvious where the line should be drawn between something that is hard to establish and deep versus something that is hard to establish because it involves some arbitrary steps that cause it to lose being a meaningful connection. Arguably a lot of math (and other disciplines as well) do publish these papers establishing these tenuous connections. The publish or perish stress and threat I think exacerbate the problem.

Overall, here’s why I want people to watch Room 237. I hope that it opens up some much needed discussions in academia about these issues. The summary question is as follows. Suppose you notice some pattern or think there might be some connection in what you’re studying. You need another paper, so you over-analyze the situation until you see a way to force an argument for the relationship. You publish a paper on it. In what sense is this legitimate academic work and the moon landing theory is not? How can we tell the difference? I’m not saying there isn’t a good answer, but I think the lack of admitting that this could be a problem allows moon landing style theories to exist without any criticism about legitimacy from the university.