by

On Asking Questions and Academic Love

I remember my first forays into question-asking at the CS colloquium in my first semester.  John Riedl told me to have a question ready for every single talk and, if no one else has one, ask it, no matter how good you think it is. This was pretty scary for someone with a music ed undergrad from Ohio State and a fine, but professionally oriented, CS masters from James Madison. So I felt ill-prepared for deep thoughts on talks about phase transitions in the difficulty of 3-SAT problems or statistics questions for Bob Kraut.

But I made the questions, and I asked the questions, and as those who have seen me at conferences can attest, I continue to do so. I’m glad John made me do it; here’s why*.

First, paying enough attention to a talk to make a good question means that I’m actively engaging with the ideas and with the speaker. This, in turn, means I’m not checking e-mail, working on my own slides, making mildly amusing comments on Facebook, or other things that we regularly criticize our students for in the classroom that take us away from the moment and the speaker.

Second, thinking hard about topics, especially ones that aren’t in my comfort zone, can lead to new ideas. I try to go into talks (and papers) with a “what’s in this for me” attitude–what can I learn or use from this talk?  When I do that, almost any talk or topic is interesting, even if I have to do translation work to connect topic X to my own interests**.

Finally, questions are love in academia. I’ve had talks when I’ve gotten two questions, and although I try to tell myself it’s because I’m full of blinding insight, that statement is full of something else. Asking a question says “I cared enough about your work to think hard about it.” We can all use a little more love.***

So, spread the love. Pay some attention and ask some questions. It’s part of being in the community.

A couple of quick notes for new question askers. First, don’t make the question about you. It’s okay to ask hard questions^, and asking good questions can help you be more visible^^, but don’t ask a question just to demonstrate that you’re smart. We’re all smart. Second, don’t make the question about you. Some questions are thinly disguised opinion pieces and/or self-promotion, and we don’t need any more of those.  Third, don’t make the question about you. If you’re taking the ideas into your domain, help bring them back out for the speaker and the audience so they make sense.

On question style, be more like an interviewer than a lawyer. Lawyers often ask yes-no questions, leading questions, and questions where they already know the answer. Usually, those lead to boring answers. So do questions where the answer is likely to be about details that are in the paper but that (probably correctly) got left out of the talk for space^^^.

Instead, shoot for more open-ended questions that give the speaker room and context to breathe and be creative. I don’t have a lot of canned strategies for this, but folks with some humanities/critical/philosophical background often ask nice questions by bringing the topic up an abstraction level. For example, at a talk on how recommender interfaces change opinions, someone asked whether this was a good or bad thing, which was an awesome question that got at the heart of what recommender systems do. It can be fun, done with care, to call attention to an aspect of the topic that’s not focused on in the talk. In the HCI/CSCW world, asking about social implications or compelling applications of technology/systems papers is a common model, as is asking about technical or design implications of experimental or observational studies. Asking about how the ideas fit nearby, related domains or ideas can lead to nice chats. Finally, asking about unexpected things that happened during the study can lead to interesting insights.

Here’s hoping I ask fewer questions at CSCW next week because you pass this on to students and other folks who carry the torch. Thanks for reading and let everyone know if you have strategies (or alternate opinions) around good question-asking.

* A few other folks who have written about this, with extra detail and somewhat different takes, for folks who want to think more about question-asking goals and strategies:

** I really like dodging outside of my topic areas. Although there’s value in and pressure to be deep and a little narrow in academia, there’s also value in being broad and establishing intellectual trade routes.

*** Questions can lead to longer-term conversations later. My first encounter with Jofish Kaye and Janet Vertisi was asking a question at their CHI 2006 talk on personal archiving. Little did I know I’d be hanging out with them at Cornell shortly thereafter, and having gone, thought about their topic, and asked the question paved the way for natural interaction later.

^ Best done gently. I remember someone basically calling someone else a fraud at a machine learning conference once, which put a damper on things.

^^ Especially if you announce your name and institution, which should be a conference norm. At most talks most people won’t know most people, and it’s good for community to announce.

^^^ Methodology questions often have this flavor. If you have a deeper question that depends on method info, go for it, but conversations about big ideas are usually more fun than those about p-values, Krippendorph’s alphas, or algorithm parameters.

 

Write a Comment

Comment

  1. Practicing what Dan preaches…
    Do you think that the same should hold for academic blog posts, or is this mainly part of the in-person contract at a talk?

    And why don’t we have discussants at conferences, anyway?

  2. My general inclination is that more communication is better than less, until it gets to the point where it overwhelms attention.

    And, good question on the discussants in HCI-land. Victoria had a paper at CSCW last year in a session that didn’t look too coherent at the outset, but had some really cool themes at the end — and it would have been nice for someone to have a chance to talk about those themes. Good speakers and questions sometimes refer back to earlier talks in the session or the conference as a whole but I wish there were a better job of perspective-taking, and discussants would be a nice way to do this. (Benkler’s keynote, also at CSCW 2012, was really a nice example of this). But, having discussants implies both work for them and less time for either talks or questions; both of those are real costs.

  3. Really great post, Dan. I think Michael’s question wasn’t just academic, though. :-) We have a community research blog over at crowdresearch.org/blog where we’ve been trying to generate discussion, and it’s a struggle. You talk about all the intrinsic motivations for asking questions… any hope for extrinsic incentives? Can and should question askers be rewarded?

  4. I like this advice (and hope to follow it myself), but it presents an interesting scaling problem. Most talks only leave time for 1-3 questions, so most audience members wouldn’t be able to ask their questions. Online backchannels are an alternative, but it’s unlikely the presenter will respond to all the issues people raise. One point of comparison is the typical pyramid of online participation, where 90% lurk, 9% comment, and 1% create content. If those ratios get substantially altered, we probably need to develop new ways to organize and structure the interactions.

  5. The flip side of this is being able to answer dumb questions (“there are no stupid questions”) without being condescending or dismissive.

    Nothing worse than being a PHD student at a big conference, getting up the nerve to ask the question, and then getting the icy stare of “that’s so obvious”.

    I speak from painful experience.

  6. Neil: Ah, sorry to hear that on the painful experience front; some academic folks are not very nice to students. One of my first conference socializing experiences was asking [name deleted] at [conference deleted] what they did. It turned out this was a pretty senior person, who looked at me, said something along the lines of “you should already know that”, and walked away. As someone who was (still in some ways is) painfully shy and self-conscious, this was a pretty horrible outcome. This person had some major organizing duties so maybe they were just busy and stressed, but still no excuse.

  7. Kurt: Depends on the conference. At the big ones, often there’ll be enough people to step up, but there are plenty of talks and lots of conferences where the average is “not enough” rather than “too much”. As we get to saturation, alternatives such as going up afterwards to ask one-on-one are fun (and useful for folks who are more nervous), as is emailing. At even higher scales, there’s the whole “write your questions down and a moderator will pick and choose” model that you see at some keynotes. Those are often unsatisfying because I think many people never get addressed, in somewhat the same way that you talk about backchannels, although I wonder if there’s some cool way to use something like reddit to let people pose questions and vote up each others’ to get the things that are top of the collective mind.

    This is a problem I’d like us to have, though. See: http://xkcd.com/810/

  8. Unfortunately, no deep thoughts on this one. I took a look at the blog(*); it looks like a main activity is boaster-posts/extended abstracts for papers in the general space of crowdsourcing. I think this is a cool kind of awareness stream idea if you can get people to post about their papers, to know that this is a place to see summaries that are more useful than an abstract but less papery than, well, a paper.

    It’s also clear that you’d like to build a community/crowd around this.

    But it’s less clear that there’s an incentive or norm for readers to comment on others’ posts. I see that there are some thought leaders trying to ask questions/post comments on a lot of the papers coming in who appear to be mostly the “Who should I blame” team on the About page.

    I guess I would try hard to get a norm going of posters being expected to check back for others’ stuff, and at least sometimes to pay it forward to other posters. I’d be super-explicit about this in my comments, both on blog posts (“Don’t forget to pay it forward and comment on someone else’s interesting stuff”) and through email exchanges with authors who have submitted.

    I’d also make clear the personal rewards of doing this: being visible in this community and raising awareness (nicely) of your own work; having a chance to talk to folks who share your interests, exchange ideas, and find collaborations; and having an entry point to chat someone up at a conference in person later. I’d probably try to have birds-of-feather, open space, informal lunches, or other kinds of times f2f at conferences to complement and build the online presence.

    External rewards are harder to picture in this case: do I really want to be a level 8 commenter? A points system that allows people to “earn” the right to post about their own stuff works in some contexts where there are plenty of contributors. It takes me back to anti-leeching efforts on BBSes back in the 80s, which is a pleasant memory, but I wouldn’t likely do it here. The community is small-ish and I’d rather have more posts and some comments than fewer posts and, really, probably not that many more comments. Featuring the best comments on the blog might be cool and rewarding for some folks. But it’s not an obvious win to me.

    One other idea I’ve heard is that the core folks should really try hard to make some space for other comments before posting themselves (which maybe you already do), in order to give new contributors a chance to jump in. I know you want to reward posters too, but they get some value already from the eyeballs — and it would be sad if the core folks are draining away question energy that might otherwise come from other folks.

    It might also be interesting to survey authors to see if they’ve gotten any email contacts about the papers that they think might be related to the blog — it’s possible that it’s having activity and community building effects, just not in a visible place.

    Anyways, way more than you wanted, most of which wasn’t even about your specific question and much of which you’ve already probably thought about. But I’d happily chat with folks at CSCW if that has some value.

    (*) I didn’t have a lot of awareness of the blog beforehand, despite being a co-author on a paper that got posted about.

  9. And a very pleasant natural interaction it has been too!

    This is a great post, and I’m glad to see it.

    I’d add that one thing to do is to listen to other questions people ask and think about what makes a good question. Seminar series are good places to watch this happening before big conferences. It’s important for graduate students to try and get away from the who and focus on the what – seniority doesn’t mean people ask good questions, and vice versa.

  10. I wonder if in order to get more questions on blogs one could appeal to status and reciprocity.

    I suspect that if a blog post receives a comment from a high status individual, others might follow and will want to hang out in that blog in the future. For example, I think I posted this comment here because I saw other people I respect commenting here :)

    As for reciprocity the idea is that those who are commented on, might comment back in the future. So assigning a discussant to a post could help foster that.

    All of this tends to require a lot work at first. I think there’s no cheap solution.

    I really doubt badges would help in a small community blog because those badges don’t have much value outside.