As an assistant professor entering the summer before my sixth year, I’m spending non-trivial effort on putting together my tenure package. For those who haven’t had this experience, it involves a number of things. One is a full-on assault on my CV: papers published, talks given, awards garnered, funding gotten and sought, and service rendered to the department, university, and broader academic community.

Another is a “research statement” that tells stories about what I’ve done, how it fits together, why it’s important, and how it’s affected the world [1]. Along with this, you send in a list of names of potential “letter writers”, who testify to how you’ve influenced the research community [2].

A third part is a “teaching statement” about my philosophy of teaching and the results I’ve gotten [3]. This pairs nicely with an “advising statement” that talks about how I work with PhD students and undergrads, both on research and in more general career (and occasionally, life [4]) advice.

There are probably other parts that I’ll find out about along the way, but these are the main ones you hear about. And it’s daunting, for a number of reasons. First is the uncertainty, about the process, the criteria, and the outcome [5]. I feel fortunate to be at Cornell, because even if I don’t get tenure I expect to have options [6]–but I’d still rather get it! Cornell has done a fair amount of work to make the process and criteria transparent, but it’s still a little nervous-making.

Second is that it forces you to confront the question of whether you’re doing good and valuable work. You’re reflecting on your career as a whole, which is much different than the more situated way I suspect most of us approach the work [7]. Who have you helped? Hurt? What does it mean? What’s next? It’s good to face these questions every so often–but they can still be scary.

Third is that you are working with incomplete information. Some of this is because much of the power rests in other people, and you only sort of know how others see you [8]. More practically, you probably didn’t record everything along the way, either about the activities or why you did them, so you’ll be spending some time grinding through email, filesystems, and neurons trying to dredge it back out [9].

There’s lots more to say about this, and I plan to come back to the blog as I do some of these activities to talk about them in ways that hopefully help other folks down the road. But for now let’s leave it here, while I go off and ponder my teaching statement some more.

[1] Someday we’ll have a nice blog post about reasons to do research even though most of it does not have such impact.

[2] Or, call bullshit on your research statement. Or just decline to write a letter, which is apparently not a good sign if too many people do so.

[3] Technically, I should be writing this right now; this  post is total procrastination.

[4] Though it’s not obvious that I’m qualified for this.

[5] Tenure is always uncertain, and everyone needs a backup plan. Cliff Lampe’s plan was goat farming, but he recently got tenure (which means, I guess, he could just do it anyways). Following in my parents’ footsteps, mine is to be a truck driver. I like driving, I like trucks, and I like stuff. It’s perfect.

[6] My original PhD plan was to teach at a liberal arts school; I still have my statement of purpose for PhD apps floating around. It’s a little cringeworthy, which means that I’m sure to post it someday.

[7] Frantic CSCW submitters from yesterday, I salute you.

[8]  A common piece of tenure advice is to clarify this through giving talks at other schools and asking either indirectly or explicitly about tenure and letter-writing. This is sometimes called the “tenure tour”.

[9] Though, this has been surprisingly rewarding as a way of reminiscing about people and events.

With the CSCW deadline coming up, it’s time to remind authors that folks who submit papers should review papers as well. There are lots of good reasons to review, including professional skill development, building a reputation as a thoughtful contributor, becoming aware of other work in your area, and being part of the academic conversation–reviewing helps shape the direction of fields [0].

My main point in this post, though, is that your submission is consuming the community’s resources. On average, papers submitted to conferences get around five reviews: three reviewers and one metareviewer almost always; at CHI, CSCW, and friends, there’s usually a second metareviewer and sometimes more than two [1].

Between the finding of reviewers, the reading of submissions, the writing of reviews, and the discussion of the paper between reviewers both before and at the PC meeting, I’m guessing the average paper burns through 15 hours of other people’s time.

So, two implications.  First is that if you’re consuming, you should be contributing as well. My rule of thumb is that the authors of every submission should conspire to do (at least) five reviews. This won’t always work–sometimes people submit across fields, or everyone on a paper is junior and unlikely to be chosen as a reviewer–but it’s a good rough estimate.

Second is that you should consume wisely. Every time I hear someone say “I don’t think this will get in but it would be useful to get some reviews”, I cringe, because you’re burning away people’s academic energy [2]. Go find some people who know the area, whose opinions you respect, and/or whose job it is is to help you improve your work and ask them for help. Trade reads with fellow travelers. Submit to works in progress, workshops, posters, and other venues where a part of the game is to get feedback on work in progress.

Sorry, a little ranty today, and I’m happy to hear counterarguments around the role of reviewing (in particular, iterative improvement feels more like part of the lifecycle of journal papers), obligations to review, or really anything else. 🙂 But take the big point: think about the costs and contributions you’re making to your community [3] and make choices that respect the community and the people in it.

[0] $1 to Phoebe Sengers, who pointed this out to me.

[1] This is almost always a bad thing for papers in my experience; usually, this means a paper that the ACs aren’t passionate enough to argue for–or one where they picked reviewers who liked the paper, but the ACs disagree–that is being sent off to die under the weight of reviews.

[2] Technically, you’re burning away their lives, period.

[3] You can apply this to other aspects of the game as well, including doing “practice interviews”, writing but not reviewing grants, taking but not giving help, contributing to your local (grad student/faculty/industry) social and professional infrastructure, et al.

I had a great CHI experience this year, and I want to share much of it with you as a trip report [0]. This will be a series of short posts because one long post would be daunting to write and, like CHI, a little long to really do all in one sitting. So today, I am going start in on the Monday sessions I went to, which were pretty awesome on average [1].

Immediately after the keynote was the “Managing Social Media” session, which went five for five for being interesting.

Xuan Zhao (disclaimer: I’m part of her “et al.”) gave the first talk about the “Many Faces of Facebook” [2], tying together Goffman’s stage metaphor, Hogan’s ideas around exhibition, and the strand of work on the personal value of digital archives (Whittaker/Petrelli/van den Hoven/Lindley work; our own stuff; Odom, Zimmerman angle; etc.) to show how both present and past, and public and personal goals, affect people’s decisions about creating and curating content in social media. Solid talk and a design idea around home metaphors that people found stimulating.

Yumi Jung then walked us through their paper on carving up social capital into tractable and meaningful sub-dimensions in social media. It had a nice, clever experimental design where folks were asked to post real favor requests (controlled by the experimenters) to their social networks and look at response rate. It was also nice to see work to more clearly define notions of social capital, which is often thrown around in a pretty loose way. I was debriefing with Xuan a little so I have to confess to less-than-full-focus on the talk but it seemed solid.

Michael Bernstein came next, talking about whether people are good at accurately estimating their audiences in social media for posting particular pieces of content [3]. Answer: no, it turns out that folks underestimate their audience by ~4x (and perhaps not surprisingly, wish their audiences were bigger). The particular measure of whether someone was actually an audience member for a given piece of content could be quibbled with (visible for ~1 second onscreen), but it was justified well, and the general question of audience and response in social media feels important.

Following through on the question of response, Moira Burke then presented the Wang et al. paper on which topics get responses in social media, and whether both topic use and response differ by gender. Answer: yes, especially for adults [4]. The paper is high on data mining and relatively low on theory for why particular topics might lead to differential responses, so caveat emptor based on your method preferences. But the paper and talk–Moira really knows how to give a clear presentation–are super-straight up about this and the work and topic are interesting.

Finally, Yunan Chen presented a note on behalf of co-authors Shi and Xu in which they applied Nissenbaum’s notion of contextual integrity to understanding privacy violation in Facebook. In particular, Friendship Pages, when they were first introduced, aggregated information in a way that people didn’t expect, anticipate, or appreciate [5]. The paper takes Nissenbaum as a lens to pick this apart more thoroughly, and, reassuringly, many of their observations parallel some of Xuan’s paper: the need for human curation and choice; the costs of bringing everything together in places and ways that could be misperceived; the idea of temporal transitions affecting the acceptability of tools like friendship pages.

So, CHI started off with a real bang for me, and I’m going to stop there lest I screw up my own audience’s attention and response. I’ll keep putting out bits that are hopefully digestible and useful as I have time over the next few weeks. Happy to hear others’ reactions to things I saw, and your own highlights, in the comments.

[0] Perhaps the most famous trip reports are by Ed Dijkstra (for example, this one), though they are a little dated. Ed Chi has an example from CHI 2009 that was fun to retrospect on a couple of years later.

[1] I’m skipping keynotes because that might make for an interesting post on its own, and social stuff I’m either going to collect all as one post or just leave out for later. I’ll also point out that I tend to go to more papers sessions than I think most folks would recommmend; a lot of the action happens outside of the formal sessions. Scott Berkun has a nice article about this, and though he’s a little too dismissive of papers sessions (especially at bigger conferences, you’re probably not going to actually grind through the proceedings, and I’ve had lots of good social interactions with people about their presentations), but it’s an interesting read. Gilly Leshed at Cornell also has some nice slides and gives a yearly talk about this at Cornell IS, though I can’t find them on the web to share.

[2] For some reason, it’s kind of cool to be page 1 of the conference proceedings.

[3] Delightfully cheesy prop use, with a curtain between him and the audience for the first part of the talk to emphasize the idea of an unseen audience. It made me reminisce about eating, and talking about eating, an ice cream bar to induce audience reminiscence during the Pensieve talk at CHI 2010. It also made me wish we could have organized a mass evacuation of the room so that when he pulled the curtain down no one would have been there.

[4] There are some amusing gender differences between adults and teens that weren’t discussed in the talk, but are super-clear in the paper; slang, swearing, complaining, relationships, and negativity about people are relatively infrequent for adults but top categories for teens.

[5] Arguments about the perils of information aggregation go all the way back to the Lotus marketing database fiasco in 1990; see Laura Gurak’s Persuasion and Privacy in Cyberspace: The Online Protests over Lotus MarketPlace and the Clipper Chip for more details.

Citations to the papers above are, well, below.

“Managing Social Media”

Xuan Zhao, Niloufar Salehi, Sasha Naranjit, Sara Alwaalan, Stephen
Voida, and Dan Cosley. 2013. The many faces of facebook: experiencing
social media as performance, exhibition, and personal archive. In
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems (CHI ’13). ACM, New York, NY, USA, 1-10.
DOI=10.1145/2470654.2470656 http://doi.acm.org/10.1145/2470654.2470656

Yumi Jung, Rebecca Gray, Cliff Lampe, and Nicole Ellison. 2013. Favors
from facebook friends: unpacking dimensions of social capital. In
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems (CHI ’13). ACM, New York, NY, USA, 11-20.
DOI=10.1145/2470654.2470657 http://doi.acm.org/10.1145/2470654.2470657

Michael S. Bernstein, Eytan Bakshy, Moira Burke, and Brian Karrer. 2013.
Quantifying the invisible audience in social networks. In Proceedings of
the SIGCHI Conference on Human Factors in Computing Systems (CHI ’13).
ACM, New York, NY, USA, 21-30. DOI=10.1145/2470654.2470658
http://doi.acm.org/10.1145/2470654.2470658

Yi-Chia Wang, Moira Burke, and Robert E. Kraut. 2013. Gender, topic, and
audience response: an analysis of user-generated content on facebook. In
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems (CHI ’13). ACM, New York, NY, USA, 31-34.
DOI=10.1145/2470654.2470659 http://doi.acm.org/10.1145/2470654.2470659

Pan Shi, Heng Xu, and Yunan Chen. 2013. Using contextual integrity to
examine interpersonal information boundary on social network sites. In
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems (CHI ’13). ACM, New York, NY, USA, 35-38.
DOI=10.1145/2470654.2470660 http://doi.acm.org/10.1145/2470654.2470660

tl;dr: Read Dima Epstein’s post summarizing advice and thoughts from a panel we did about social media and academia and Phil Agre’s Networking on the Network.

Dima Epstein, Natalie Bazarova, and I were on a panel for using social media as an academic. I’d say none of us is super-expert, but between us, the audience, and some of the folks I know on Facebook, we generated potentially useful thoughts that Dima compiled on his blog in this post: Making social media work for you – notes from a workshop.

My biggest thought on this is that it’s useful to think about social media as just another tool for helping you define, find, be aware of, and contribute to the communities you care about. For a more comprehensive treatment of being part of an academic community, check out Phil Agre’s Networking on the Network, which I wish I had read as a grad student. It’s from around 2002, so it talks about email rather than Twitter, but the principles are timeless. Sections 1-6 are particularly useful for folks before the job stage, and would be useful to skim before your next conference.

I also especially liked Andrea Forte’s high-level thought below, about using social media as another way to be your awesome self. Many more thoughts and specific strategies are at Dima’s post, and below from helpful Facebook friends. If you have more tips and comments, they’re probably most useful attached to Dima’s post since that’s where most of the meat is, though I’m happy to see them here as well.

Ines Mergel Here are a few things I usually talk about: (1) Set up a blog as their homepage to showcase their work: when search committees google them they will find valuable information. Plus academics tend to move and it make sense to have one digital homebase that doesn’t change with every move & they can control on their own without getting faculty assistants involved. (2) Use Twitter to distribute their own work, follow journals, editors, other academics in their area, ask questions. (3) I also include RSS feeds to the journals in their area, so that they automatically receive notifications. (4) Think about your FB privacy settings vis-a-vis students, search committees, etc. (5) I’m sure there is tons more…

Thomas Høgenhaven Try to submit guest posts to existing (somewhat successful) blogs + generously share others work on Twitter

Andrea Forte i tell people… you know all the awesome things you are in everyday life that make you someone people want to hire/work with – supportive of colleagues, insightful about your field, modest but excited about your work, fun – be that online too.

Joseph Jofish Kaye I like all these suggestions. I’d add one, which is to email people when you get a paper published that you think they might want to read. Focused, not everyone you’ve ever met, but particularly if you’ve got a book chapter which you think certain people might find useful, then send it to them when it comes out. I think not enough people do this. (I learned it from Kristina Höök.)

tl/dr: Concrete: think about spending a summer at another school rather than an industrial lab, doing an internship outside of the summer, and/or an internship that’s not publication-centric.  General: creatively structure your grad school activities to get experiences, skills, information, fun, and people that you want.

When I was a grad student at Minnesota, folks there, at Michigan, and at CMU got a joint grant that led to great collaboration around social science and design, including the Building Successful Online Communities book (preprint chapters are downloadable here).

Part of the challenge of being on that team, especially early on, was that we were both distributed geographically and from different disciplines. This made it harder to figure out how to work together, transfer knowledge, and plan activities where everyone got value from them. This is common in (interdisciplinary) teams, and we used standard strategies such as bringing the team together for regular retreats and having virtual meetings and seminars.

These were helpful, but one idea we kicked around was an emissary model in which folks from place X would spend substantial time at place Y. I saw this as a big potential win: a distance collaboration could go turbo [1] and its members could get cross-training in other disciplines, experience different work styles and cultures, expand their professional networks, and maybe get letter writers from other institutions. All of these seem like pretty good outcomes.

So the specific question is: how could something like this work? Professors do sabbaticals and undergrads do REUs, but AFAIK it’s super-rare for PhD students [2]. My analogy is “academic internship”, with variations compared to a traditional industry experience. Without NDAs and intellectual property issues it might be easier for the advisor to be an active collaborator–which might in turn simplify funding, allowing the advisor to pay a summer stipend or RAship as they might do anyways. I’m sure there are reasons why this is more complex than I think, and I’d be happy to hear them in the comments, but it feels like something worth exploring.

The more general question is: how can students think more creatively about how to structure grad school experiences? Victoria Sosik and Xuan Zhao are doing spring and summer industrial internships this year; other students I work with are also thinking about the possibility of semester vs. summer internships. Not all companies are able to do this, but some are, and some might welcome it–maybe worth thinking, and asking, about? [3]

It’s also not necessary to turn every internship into publications, especially if you’re considering non-academic career options [4]. Victoria’s done one pure product, one mostly research, and one balanced internship; Xuan is doing one of each. This will give them a lot of information about academia vs. industry, and if industry, which flavors taste good.

So, that’s my story today: think about creative ways to structure your internships and other grad school activities to help you get the experiences, skills, information, fun, and people that you want to be part of your (academic) life.

What’s your story about this idea?

— Dan

[1] Not to be confused with “Going Turbo” from Wreck-It Ralph, which I recently saw and loved.

[2] I would love to hear stories about this working, or not working, if you’ve done it or sponsored it.

[3] From a student funding point of view, it’s like supporting yourself for a semester (especially if you can get low tuition through an in absentia or on leave kind of status).

[4] Which you should do; alternative academic careers can be necessary, lucrative, rewarding, and/or fun.

My girlfriend, alas, will be disappointed by the sense of “engagement” I’m about to speak of.

I had an awkward conversation at CSCW where I completely failed to connect to someone who I was trying to honestly engage. This made me sad, and got me to thinking about other failures to engage I’ve been involved in.

• I was listening to a couple of women about the lack of women in computer science. I’d had some conversations with women students about their experiences and frustrations when I taught at James Madison. I started to share one, but was cut off by one who told me because I wasn’t a woman I could never understand and wasn’t qualified to have opinions on the topic.
• At GROUP 2007, I was chatting with an anthropologist at a dinner at GROUP 2007 who told me about their work, which was pretty fun, then asked what I do. I replied that I use theory to design systems, to which they replied “You don’t use theory, you make theory” and turned away. End of conversation.
• I’ve also been on the giving end of Fail To Engage. The fact that I use theory is funny, because I used to have trouble taking social sciences seriously because people are complicated; the theories and models felt so limited that I didn’t see their value until I started working with social science folks through the CommunityLab project between CMU, Michigan, and Minnesota. But I had a number of conversations early on where I probably sounded like a total engineering shit. [1]

Most likely, you can conjure up memories when you’ve been on both sides of this, and chances are they’re not great memories. So, my main point is that, just as asking questions is a kind of academic love, the will to engage is academic love as well.

This is not a Lieberman- or Dourish-style call for people from different disciplinary, methodological, or theoretical backgrounds to lay down their arms and embrace alternate perspectives. [2] I do think being open to this is generally good, but you have to call your shots when deciding on extended, serious engagement with other perspectives or disciplines. It can make you uncomfortable, it takes time to learn the lingo, your home tribe may not value your expeditions, and you can’t afford to engage with everything. [3]

In the context of a single conversation, though, refusing to engage is probably a net lose, especially with someone who is reaching toward the things you care about. Engaging in these contexts is a very practical kind of academic love that gives you a chance to spread your work, interact with people, and connect to ideas you otherwise might not. Those people and connections might in turn propagate your ideas into communities that might not otherwise see them ($1 to Ron Burt).

This willingness to engage is a hallmark among people I deeply respect in academia. Jon Kleinberg has a lot on his plate, but when you do talk to him, you know that he’s engaged with you. Phoebe Sengers does effective critical work around technology in part because she has a real empathy for and understanding of the things she critiques, and she’s happy to engage with people and ideas across the spectrum. Helen Nissenbaum has impact across intellectual communities because she’s willing to engage with folks who speak other languages.

We could do worse than to emulate them. [4]

A couple of years ago I wrote a little advice for systems papers writers for the CHI 2011 Facebook group; with the CSCW deadline coming up and CHI closer than you think in the rearview mirror, it seems useful to resurrect here.  It’s imperfect advice, and others will disagree; I’m hoping they’ll put some of their own bits here.

The tl;dr: Tell readers why your work matters, not what you did, and think about what they can take away. Bonus list of crash landings at the end, along with snarky but useful footnotes and pointers to very useful articles.

The long form: Cliff asked me (in 2010) to write a little bit about what makes for a good paper with a significant system component in CHI. Henry Lieberman’s The Tyranny of Evaluation, Saul Greenberg and Bill Buxton’s Usability evaluation considered harmful (some of the time), James Landay’s discussion of the pain of systems work, and CHI’s own guide to successful archive submissions have said a fair amount already, and you should go read those, too.

But, as someone who’s reviewed dozens of papers a year for the last 4 years, I’ll cheerfully weigh in on what I look for as a reviewer, tempered by things I’ve overheard in PC meetings. You’ll have to forgive all the footnotes; I just read a long law review article and they’re all the rage in that discipline. This is also more collected thoughts than careful treatise. So, caveat emptor.

First, your system, itself, is probably the least important, least interesting thing in your paper [0]. A lot of papers read like a story about “what I did on my summer vacation” [1] (literally, often, because of the rhythm of internships and HCI conference deadlines [2]). Here’s my idea. Here’s what I did. Isn’t it cool? And it’s natural to focus on yourself and what you did, because you know it best.

But people didn’t care about your summer vacation when you were talking about it in fifth grade, and they don’t care about it now, unless it matters to the field or to society at large, and unless you did it well [3]. To tell that story, “what I did” is like the skeleton — and like most skeletons, it’s a little creepy if you see it all by itself.

Instead, think of the raw work as the basis for talking about things that other people can learn from. Sometimes, people can learn from “how I did it”. In some scientific traditions, providing enough detail that someone could implement approximately the same system if they wanted to replicate the work is really important [4]. Especially if the mechanics of the implementation are novel, or informative because they illustrate problems or approaches that might have broad applicability, then they become interesting and worth spending precious writing time, paper space, and reader attention on. But if the guts are mostly things a competent senior could do, they’re not the important part.

The real action is on “why”: why what you did matters [5]. Many aspects of this are outlined in the CHI guide to successful papers  and the reviewing guide , both of which you should read every year [6]. These include demonstrating a contribution, originality, validity of the work, and offering benefits to the reader. With design/system papers, there’s such a nice paragraph there that I will just quote it:

“[Reviewers] often criticize authors for conducting studies without adequate theoretical basis, or for not providing enough evidence or sound reasoning for claims. A further concern is lack of justification for design choices and not explaining why certain design features have been included. In summary, you should explain not only what you did, but also why you did it, so that readers (including reviewers) can be convinced that you made appropriate choices. Explaining your choices can also stimulate more research by helping others see alternative approaches.”

One way to think about that little nugget is that people want to understand why this system: why is your system is arguably “right”, or “better”, or “interesting”, or “useful”, in the context of your problems and your contributions. A solid evaluation of the system/design/technique in use through some combination of usability testing, lab studies, field studies, and logitudinal deployment (the right technique(s) should be chosen based on your problems and contributions), showing its potential or actual value, is one way to do that [7].

But it’s only one way. You can use theories and empirical work about specific designs, about individuals’ capabilities and goals and values, about psychology or sociology or economics or S&TS or [insert your favorite discipline here] to show that your problem is important and your system is a reasonable response to that problem. You can also use these, as well as making parallels to other problems and fields, to argue that your system has a greater, general value, and that many researchers can benefit from it. You can argue that a system that successfully does X might improve lives or the world. But make the case [8].

Many of the same techniques can also be used to motivate specific design choices. You can reason about alternate systems, and their choices, and consider alternate choices [9] and why they might be better. Don’t limit yourself to the academic literature, either; there are a lot of non-research systems that work just fine, and comparing to/critiquing/borrowing from/expanding on those designs is working smarter, not harder. You can appeal to your own experience in the past, to iterations you did along the way, to user studies you or other people did to give you insight into the design. But again, as a consumer, I want to know why this system and not that one; the considerations you had and tradeoffs you made are money in my bank as a designer and as a researcher [10].

So, that’s my rant, for now; hopefully it’s useful. Remember this is my perspective, not a universal guide to CHI success (reread the guide to successful submissions!), and that other, really smart people will disagree with me on some points. But I think most reviewers would agree with the following list of crash landings:

• Fail to motivate why the problem is important.
• Don’t tell me why your system is a plausible approach for that problem.
• Do a cursory job of talking about related work that doesn’t help me understand how yours fits in and what you took from it.
• Spend your time on minutae of implementation that doesn’t matter.
• Present your system as though, like Athena, it sprang fully formed from the head of Zeus.
• Avoid showing me what I can learn from your choices, or talking about the general issues other designers might face.
• Perform a bad evaluation, and/or fail to provide other kinds of justification.
• Make me guess what your contributions are, and how one might apply your work in other contexts.
• Write badly, confusingly, disrespectfully of readers’ time.

Don’t crash land. You might not get in anyways; sometimes the work is not ready, sometimes it gets bad reviews, or bad reviewers, or tired or grumpy reviewers who make a mistake. And sometimes you’re just unlucky. But you’ll do better, more useful, and more publishable work by thinking about how to communicate “why” and not just “what”, and thinking harder about the reader and not the writer of the paper. And that’s a good thing.

— footnotes —

[0] Unless you have a truly bad evaluation, in which case that is the least interesting thing in your paper. The general sentiment at the UIST 2009 PC meeting was “I’d rather see a paper with no evaluation than a bad evaluation”.

[1] And let’s not get started on “what was done on the summer vacation provided to me”, passive voice, allegedly dispassionate and detached boring writing style that papers affect and that makes readers die a little inside. Read The Elements of Style, or On Writing Well, and take it to heart. Readers will thank you, and they will probably feel a little better about your paper as well.

[2] Johnathan Grudin, among others, has been advocating for a move toward journal publication over the last few years, to get at deeper, better research. There are practical career values to doing some journal-based work, too; I didn’t get a job out of grad school in part because I didn’t have any journal papers — direct quote from a respected mentor. I’ve also gotten advice from several people that even if your department, or the field, doesn’t care about journal publication, your school or college might. Do you really want your tenure application to go down in flames because of an angry group of civil engineers or economists or historians? So journals seem like a good idea, although please don’t let that stop you from submitting to CHI along the way!

[3] People who have written NSF proposals should have heard “intellectual merit” and “broader impact” when they read that sentence.

[4] Based on CHI reviews and PC meetings I’ve seen and done, as well as the explicit statement in the Guide to Successful Papers about originality, replication is much more likely to succeed if there’s a novelty component as well. Jeff Heer and Michael Bostock’s CHI 2010 replication of some classic information visualization perception studies using Mechanical Turk to solicit participants is one such.

[5] If you can’t clearly articulate this, you may be doing it wrong. Go read Richard Hamming’s You and Your Research immediately. Crotchety, but valuable. Then, as an encore, start working your way through Phil Agre’s Networking on the Network about how to manage the professional side of the career. It’s not perfect, but valuable.

[6] And if you’re not reviewing, shame on you, you freeloader. The average submission gets attention from 5 people, so the authors of every submission should conspire to do at least 5 reviews.

[7] There is an unfortunate perception that to get a systems/design paper through the CHI reviewing process you need an evaluation. Not true. You need a good evaluation. 🙂 Or a really good non-evaluative justification. Both is even better.

[8] Usually in the introduction. You’d really like your readers to want to read your paper, so convincing them it’s important and valuable work should start early.

[9] That is, don’t just list related work, but use it, respectfully, to talk about your system, your choices, and why what you did is novel, interesting, and valuable. And try to avoid the style you sometimes see in papers where the related work goes at the end and is mostly used to talk about limitations with that work that the current paper heroically overcomes. It’s awfully easy for that to sound like “look how dumb they are and how smart I am”, and that’s a big turn-off to reviewers, especially if (as is often the case) they did some of that work.

[10] You should be writing this stuff down as you go along, so you can talk about it later, and keeping every version of the system that matters, even a little bit, for helping people — including yourself — learn more from your work. Bonus points if you tell me about your failures, and what didn’t work. I realize it’s scary to talk about the making of the sausage, and it’s hard to get an epic fail published, but things that didn’t work along the way are useful.

Good luck, fellow travelers. And please, add your own thoughts.

The trip report is a bit of a lost art, but I did want to capture a few things I liked at CSCW and share them with folks around me. It’s roughly chronological and focuses on sessions, and is only one of many paths through the conference (I wish I’d had about 3 of me; there were a lot of things I’d like to have seen); hopefully other folks will talk about their own paths in other spaces.

Both plenaries were big fun: good topics, engaging speakers.  Ron Burt had a new wrinkle around the idea of bridging ties in networks, calling out the importance of how the network is built. In particular, bridging ties that come from being embedded in a specific community for a while appear to be most valuable: the most successful performers oscillate between tight, local clusters within communities (in which trust is built and shit gets done) and broad connections across communities. It was a nice layer of nuance on the strong ties/weak ties story. Moira Burke and Bob Kraut’s paper analyzing the effect of strong and weak ties in Facebook added another, finding that talking with strong ties adds social support but also social stress — and was a better predictor of finding work than talking to weak ties.

Ron also talked about the value of encountering information outside your normal circles, so I took that at face value and went to the Gesture and Touch session.  Richard Harper and Helena Mentis gave a fun talk about how the gross, exaggerated motions that sometimes are needed to connect with Kinect led people to a playful, “carnival” attitude. I also liked Svetlana Yarosh, et al.’s paper about  ShareTable, designed for parent-child communication across divorced households; they were sensitive both to the issues of divorced families and the insight that families need to do things together at least as much as talk together. And all four were nice, effective talks–in general, the talks I saw at CSCW were good, better than average for conference talks I’ve seen.

The filter bubble panel was fine, though it was so focused on political discourse, and particularly the U.S. conservative/liberal split, that I wondered how generalizable the stories would be to other contexts and information domains. There was some useful theoretical grounding that hopefully helps with that, but I did wish the discussion had been broader. I also really wish I’d seen the Making the World a Better Place session, but I wound up spending that helping set up the demos session instead.

Tuesday I was more in my own space, and, as Ron Burt would predict, I got less stunningly new information, though it was still fun. At the “Practices in Social Networks” session I had high curiosity for the Manya Sleeper, et al. talk about self-censorship in Facebook because of Xuan Zhao’s work around self-curation in Timeline. It sounded like they were trying to figure out how much good something like Google+ circles would do if they were zero-cost, and I do think the idea of thinking hard about audience in social media is going to be important. It was a tale of two halves: the question of self-censorship focused on the types of information while the question of making it more share-able felt more focused on audiences, and there wasn’t as much connection as I’d hoped between the two.  Maybe in the paper. Then Eric Gilbert presented the shortest paper in CSCW, on underprovision of attention to new submissions in Reddit.  It was great to see someone studying Reddit (Pinterest is also ripe for colonization), and his method of counting multiply-submitted items where the n>1’th submission made it to the front page (with the implication that the earlier submissions had been ignored) was clever. The Cliff Lampe, et al. paper (presented by Jessica Vitak) also did a nice job of pointing out that use is not binary, and of looking at folks who are not college students using Facebook, so kudos there as well.

We had a couple of papers in the “Not Lost in Translation” session so I spent some time there.  Mary and Hao-Chuan did a nice job overall; practice talks paid off handsomely in both cases. Hao-Chuan’s observed that bilingual speakers allow us to design asymmetrical systems that selectively apply machine translation (think Chinese native-English second language speakers generating turns in Chinese that are translated for an English-only partner, but getting the English statements from their partner un-translated), allowing us to leverage the bilingual abilities for better outcomes. The idea that cultural difference should be a resource rather than a barrier is often raised, and this is one example of how to do it. I also liked Naomi Yamashita, et al.’s discussion of transmission lag in second language contexts. Usually CSCW systems stamp out lag wherever it’s found, but here in small doses it improved group outcomes. In large doses, it led to interaction chaos among native speakers who wound up talking over each other unawares, but the idea of lag as a resource was also cool.

Stuart Geiger and Aaron Halfaker’s talk about how different ways to measure participation in Wikipedia leads to different results was fun as well: great talk and cool point. The high-level idea was to look at time spent on Wikipedia and measure in labor hours, rather than edit counts. The argument was that labor hours was a more natural way to think of work outputs; a little Marxist, but interesting. This has all the problems around estimating session times that weblog analysis has, and doesn’t account well for tool efficiencies (think using Huggle to revert vandals and add warnings versus doing it by hand), but it was stimulating and they were thoughtful.

In the Controversy, Arguments, Rule Breakers, and Politics session, the R. Kelly Garrett and Brian Weeks paper about real-time corrections to political misinformation online was relevant to some of our work around coaching commenters in a discussion forum to be better posters–you have to give information in ways that don’t trigger defensive reactions (“Ego Threat”, they called it). Likewise the, Ben Towne, et al. paper that studied how seeing controversy and deliberation around an artifact would affect people’s perceptions of artifact quality looked fun. In general, it lowers it, though I wondered if folks more  embedded in the community norms would be more comfortable seeing the disagreements. At this point I kind of ran out of gas so I took off. I probably should have done this earlier; you should spend at least a little bit of your time getting out of dodge to talk to people, experience the location, have fun, and stay sane, but there were so many cool papers that I really wanted to try to stick with them.

On Wednesday, the Future of Crowd Work paper by Niki Kittur, et al. was intriguing: what would it take for crowd work to be something you’d be happy for your kid to grow up doing? They were proposing a move away from a faceless, fluid assembly line of repetitive tasks toward a way of organizing crowd work that would support advancement, dignity, and fairness for workers while broadening the kinds of work that might be done. They had a near-infinite supply of questions, and the 17-page paper is probably worth a read. My main question was that it still felt like a two-tier system: researchers and organizations would create tasks for the workers. I don’t think that’s how they meant it, but it would be useful to avoid an “us and them” mentality.

As an alum, I also had to go see the Most Cited CSCW paper session about the original GroupLens work. It was fun, and slightly campy, and interesting to see what they thought they got right and got wrong. It’s always amazed me how that first trio of papers from MIT, Bell, and GroupLens anticipated so many of the issues that would arrive later (and a little sad that so much of the followup work addressed only algorithmic accuracy). I also saw last year’s CSCW talk where Leysia Palen and Beki Grinter talked about their 10-year old paper about instant messaging in teen life, and I think there’s real value in this kind of look back.

Finally, at the closing plenary, Jascha Franklin-Hodge’s talk about the relentless (but somewhat disorganized and decentralized) use of A/B testing and data mining in political campaign messaging, and the value of serious thinking about user interfaces that make participation easier, also resonated well. He was thoughtful about the tension between getting the job done and generalizing results, and you could imagine interesting collaborations between academic researchers and political campaigns that could lead to insights around motivating participation. Studying the digital side of the campaign would be a fantastic ethnographic opportunity, too. My main question is that it’s not clear that good UI design should determine who gets elected 🙂 — but it was a nice way to close the conference.

So, that’s it. Left out are all the hallway conversations, the reconnecting with old friends, the meeting of new (lots of chats with grad students this time around, which was fun), the deliciously kitschy Buckhorn banquet, and all the other things that make the conference both intellectually and interpersonally stimulating. Overall, great conference and kudos to all the folks who put it together.

I remember my first forays into question-asking at the CS colloquium in my first semester.  John Riedl told me to have a question ready for every single talk and, if no one else has one, ask it, no matter how good you think it is. This was pretty scary for someone with a music ed undergrad from Ohio State and a fine, but professionally oriented, CS masters from James Madison. So I felt ill-prepared for deep thoughts on talks about phase transitions in the difficulty of 3-SAT problems or statistics questions for Bob Kraut.

But I made the questions, and I asked the questions, and as those who have seen me at conferences can attest, I continue to do so. I’m glad John made me do it; here’s why*.

First, paying enough attention to a talk to make a good question means that I’m actively engaging with the ideas and with the speaker. This, in turn, means I’m not checking e-mail, working on my own slides, making mildly amusing comments on Facebook, or other things that we regularly criticize our students for in the classroom that take us away from the moment and the speaker.

Second, thinking hard about topics, especially ones that aren’t in my comfort zone, can lead to new ideas. I try to go into talks (and papers) with a “what’s in this for me” attitude–what can I learn or use from this talk?  When I do that, almost any talk or topic is interesting, even if I have to do translation work to connect topic X to my own interests**.

Finally, questions are love in academia. I’ve had talks when I’ve gotten two questions, and although I try to tell myself it’s because I’m full of blinding insight, that statement is full of something else. Asking a question says “I cared enough about your work to think hard about it.” We can all use a little more love.***

So, spread the love. Pay some attention and ask some questions. It’s part of being in the community.

A couple of quick notes for new question askers. First, don’t make the question about you. It’s okay to ask hard questions^, and asking good questions can help you be more visible^^, but don’t ask a question just to demonstrate that you’re smart. We’re all smart. Second, don’t make the question about you. Some questions are thinly disguised opinion pieces and/or self-promotion, and we don’t need any more of those.  Third, don’t make the question about you. If you’re taking the ideas into your domain, help bring them back out for the speaker and the audience so they make sense.

On question style, be more like an interviewer than a lawyer. Lawyers often ask yes-no questions, leading questions, and questions where they already know the answer. Usually, those lead to boring answers. So do questions where the answer is likely to be about details that are in the paper but that (probably correctly) got left out of the talk for space^^^.

Instead, shoot for more open-ended questions that give the speaker room and context to breathe and be creative. I don’t have a lot of canned strategies for this, but folks with some humanities/critical/philosophical background often ask nice questions by bringing the topic up an abstraction level. For example, at a talk on how recommender interfaces change opinions, someone asked whether this was a good or bad thing, which was an awesome question that got at the heart of what recommender systems do. It can be fun, done with care, to call attention to an aspect of the topic that’s not focused on in the talk. In the HCI/CSCW world, asking about social implications or compelling applications of technology/systems papers is a common model, as is asking about technical or design implications of experimental or observational studies. Asking about how the ideas fit nearby, related domains or ideas can lead to nice chats. Finally, asking about unexpected things that happened during the study can lead to interesting insights.

Here’s hoping I ask fewer questions at CSCW next week because you pass this on to students and other folks who carry the torch. Thanks for reading and let everyone know if you have strategies (or alternate opinions) around good question-asking.

—

* A few other folks who have written about this, with extra detail and somewhat different takes, for folks who want to think more about question-asking goals and strategies:

• http://www.ethomaz.com/2012/10/01/asking-good-questions-at-conferences/
• http://blog.1degreebio.org/?bid=60 
• http://scientopia.org/blogs/science-professor/tag/asking-questions-at-a-talk/
• http://sliscon.files.wordpress.com/2011/03/conference-asking-questions-resource.pdf

** I really like dodging outside of my topic areas. Although there’s value in and pressure to be deep and a little narrow in academia, there’s also value in being broad and establishing intellectual trade routes.

*** Questions can lead to longer-term conversations later. My first encounter with Jofish Kaye and Janet Vertisi was asking a question at their CHI 2006 talk on personal archiving. Little did I know I’d be hanging out with them at Cornell shortly thereafter, and having gone, thought about their topic, and asked the question paved the way for natural interaction later.

^ Best done gently. I remember someone basically calling someone else a fraud at a machine learning conference once, which put a damper on things.

^^ Especially if you announce your name and institution, which should be a conference norm. At most talks most people won’t know most people, and it’s good for community to announce.

^^^ Methodology questions often have this flavor. If you have a deeper question that depends on method info, go for it, but conversations about big ideas are usually more fun than those about p-values, Krippendorph’s alphas, or algorithm parameters.