On Meta-Interpretation

In 2005 while I was technically employed by the Psychology department of McGill University, I went to see a talk by Stevan Harnad, a professor of Cognitive Science at Universite de Quebec a Montreal. This was a rather philosophical discussion of how Cognitive Science ought to proceed and contained, to my understanding, one of the most misguided notions that I have come across. It was based around the proposition that

“Whatever can do whatever we do indistinguishably from us is a cogniser. And the explanation of how it does it is the explanation of how we do it.”

There are, I think, two fundamental misconceptions here (besides the jargon). The first is simply utility: I was inclined to say to this proposition “Give me a willing female and fifteen years and I will produce a machine that does everything humans do indistinguishably from us, and yet I defy you to explain how a teenager can fail to notice the washing left by the stairs, or why, for that matter.” **  This is a logical and correct application of Harnad’s statement — I’m fairly sure that how my niece does things is pretty much the same as how I do things — and yet the cognizer in question is of no greater help to cognitive science than any other human. ** So the fact that you can create a cogniser does not mean you can explain it.

But this notion is wrong at a more fundamental level because it simply fails to acknowledge that the same thing can be done in more than one way. My recollection was that Harnad wasn’t suggesting we need to create some form of cyborg that would mimic humans in all aspects of life, but that his statement applied if we isolated some particular cognitive task and could accurately reproduce human performance at that task. He was particularly enthusiastic about the use of neural networks to carry this out.

This runs straight up against the notion of universal approximators. We saw in Post 2 that there are currently several methods to produce “machines” that accept inputs and produce outputs (using which we can “do” many things) and which are capable of mimicking any reasonable model up to arbitrary accuracy, given enough data. Neural networks are certainly one of these, and they have the advantage that they appear — in a very abstract way — to mimic the biology of the brain. But we might say the same thing about nearest neighbour methods basing outputs on a library of previously seen examples; another plausible explanation, following some introspection. We might also mimic the process as well as you like with a big decision tree. This last idea feels less like the way I observe myself thinking, but certainly would have fit in with early notions of how to build artificial intelligence.

The point is that each of these universal approximators has a very different explanation of “how” they do what they do and yet they can all be made to look like they do the same thing as each other, as closely as you like. More generally, this gets at the notion of meta-interpretation. We can readily understand the mechanics of how a decision tree works, or nearest neighbours, or a neural network in general principles. This does not tell us what it is doing because that depends on the specifics of the of the parameters, structures and data involved. A general statement of “it does this sort of thing” is not sufficient to “understand” any particular instance.

To be fair to Harnad, there are a number of alternative ways you could think about his statement. You might declare it unfair to be allowed as much data as you want — “what we do” may mean “from scratch” rather than just at the point where we measure performance (although how you define the starting point which, given that certainly some of what humans do is simply innate, is somewhat problematic). In this case, you also have to invoke how you get from data to model, which machine learning probably does not do as efficiently as the human brain, at least for doing human sorts of things. But surely I can learn a complicated meta model of how one does that, and then I would still have many different paths to the same output.

Harnad might also have had a much more mechanistic approach in mind (I will get to explaining what that adjective means, someday): “If we can produce an understandable machine that does what humans do, then the explanation for how the machine does it is the explanation for how we do it.” This neatly steps around the teenager in the room and is something of an appeal to Ockham’s razor, relying on interpretability to exclude alternative, more complex, explanations. But that does actually need to be explicit: I might add something extraneous (in Biology, the human appendix is a great example) to the process and still have an understandable, if redundant, model. Simplicity is great for fitting the world into our heads, but to make a claim that the world really does try to minimize some notion of complexity takes more of a stretch.

Most of this blog is fairly ambivalent about the issues it presents, but for this particular post I am prepared to be definitive. The structure of a function, without its specifics, is not an explanation of the phenomenon that it mimics. This does not mean that universal approximators cannot (at least sometimes) produce interpretable prediction models, but the models must be interpreted in their specifics.

Next: on simplicity, approximate models and another commentary left over from McGill.

**  And yes, the initial impetus for this blog was to provided just this retort, some 8 years too late.

**  It is now pre-ordained that she will go into psychology just so the world can prove me wrong.

Leave a Reply

Your email address will not be published. Required fields are marked *