Submit implies review

With the CSCW deadline coming up, it’s time to remind authors that folks who submit papers should review papers as well. There are lots of good reasons to review, including professional skill development, building a reputation as a thoughtful contributor, becoming aware of other work in your area, and being part of the academic conversation–reviewing helps shape the direction of fields [0].

My main point in this post, though, is that your submission is consuming the community’s resources. On average, papers submitted to conferences get around five reviews: three reviewers and one metareviewer almost always; at CHI, CSCW, and friends, there’s usually a second metareviewer and sometimes more than two [1].

Between the finding of reviewers, the reading of submissions, the writing of reviews, and the discussion of the paper between reviewers both before and at the PC meeting, I’m guessing the average paper burns through 15 hours of other people’s time.

So, two implications.  First is that if you’re consuming, you should be contributing as well. My rule of thumb is that the authors of every submission should conspire to do (at least) five reviews. This won’t always work–sometimes people submit across fields, or everyone on a paper is junior and unlikely to be chosen as a reviewer–but it’s a good rough estimate.

Second is that you should consume wisely. Every time I hear someone say “I don’t think this will get in but it would be useful to get some reviews”, I cringe, because you’re burning away people’s academic energy [2]. Go find some people who know the area, whose opinions you respect, and/or whose job it is is to help you improve your work and ask them for help. Trade reads with fellow travelers. Submit to works in progress, workshops, posters, and other venues where a part of the game is to get feedback on work in progress.

Sorry, a little ranty today, and I’m happy to hear counterarguments around the role of reviewing (in particular, iterative improvement feels more like part of the lifecycle of journal papers), obligations to review, or really anything else. 🙂 But take the big point: think about the costs and contributions you’re making to your community [3] and make choices that respect the community and the people in it.

[0] $1 to Phoebe Sengers, who pointed this out to me.

[1] This is almost always a bad thing for papers in my experience; usually, this means a paper that the ACs aren’t passionate enough to argue for–or one where they picked reviewers who liked the paper, but the ACs disagree–that is being sent off to die under the weight of reviews.

[2] Technically, you’re burning away their lives, period.

[3] You can apply this to other aspects of the game as well, including doing “practice interviews”, writing but not reviewing grants, taking but not giving help, contributing to your local (grad student/faculty/industry) social and professional infrastructure, et al.

CHI trip report part 1 of N

I had a great CHI experience this year, and I want to share much of it with you as a trip report [0]. This will be a series of short posts because one long post would be daunting to write and, like CHI, a little long to really do all in one sitting. So today, I am going start in on the Monday sessions I went to, which were pretty awesome on average [1].

Immediately after the keynote was the “Managing Social Media” session, which went five for five for being interesting.

Xuan Zhao (disclaimer: I’m part of her “et al.”) gave the first talk about the “Many Faces of Facebook” [2], tying together Goffman’s stage metaphor, Hogan’s ideas around exhibition, and the strand of work on the personal value of digital archives (Whittaker/Petrelli/van den Hoven/Lindley work; our own stuff; Odom, Zimmerman angle; etc.) to show how both present and past, and public and personal goals, affect people’s decisions about creating and curating content in social media. Solid talk and a design idea around home metaphors that people found stimulating.

Yumi Jung then walked us through their paper on carving up social capital into tractable and meaningful sub-dimensions in social media. It had a nice, clever experimental design where folks were asked to post real favor requests (controlled by the experimenters) to their social networks and look at response rate. It was also nice to see work to more clearly define notions of social capital, which is often thrown around in a pretty loose way. I was debriefing with Xuan a little so I have to confess to less-than-full-focus on the talk but it seemed solid.

Michael Bernstein came next, talking about whether people are good at accurately estimating their audiences in social media for posting particular pieces of content [3]. Answer: no, it turns out that folks underestimate their audience by ~4x (and perhaps not surprisingly, wish their audiences were bigger). The particular measure of whether someone was actually an audience member for a given piece of content could be quibbled with (visible for ~1 second onscreen), but it was justified well, and the general question of audience and response in social media feels important.

Following through on the question of response, Moira Burke then presented the Wang et al. paper on which topics get responses in social media, and whether both topic use and response differ by gender. Answer: yes, especially for adults [4]. The paper is high on data mining and relatively low on theory for why particular topics might lead to differential responses, so caveat emptor based on your method preferences. But the paper and talk–Moira really knows how to give a clear presentation–are super-straight up about this and the work and topic are interesting.

Finally, Yunan Chen presented a note on behalf of co-authors Shi and Xu in which they applied Nissenbaum’s notion of contextual integrity to understanding privacy violation in Facebook. In particular, Friendship Pages, when they were first introduced, aggregated information in a way that people didn’t expect, anticipate, or appreciate [5]. The paper takes Nissenbaum as a lens to pick this apart more thoroughly, and, reassuringly, many of their observations parallel some of Xuan’s paper: the need for human curation and choice; the costs of bringing everything together in places and ways that could be misperceived; the idea of temporal transitions affecting the acceptability of tools like friendship pages.

So, CHI started off with a real bang for me, and I’m going to stop there lest I screw up my own audience’s attention and response. I’ll keep putting out bits that are hopefully digestible and useful as I have time over the next few weeks. Happy to hear others’ reactions to things I saw, and your own highlights, in the comments.

[0] Perhaps the most famous trip reports are by Ed Dijkstra (for example, this one), though they are a little dated. Ed Chi has an example from CHI 2009 that was fun to retrospect on a couple of years later.

[1] I’m skipping keynotes because that might make for an interesting post on its own, and social stuff I’m either going to collect all as one post or just leave out for later. I’ll also point out that I tend to go to more papers sessions than I think most folks would recommmend; a lot of the action happens outside of the formal sessions. Scott Berkun has a nice article about this, and though he’s a little too dismissive of papers sessions (especially at bigger conferences, you’re probably not going to actually grind through the proceedings, and I’ve had lots of good social interactions with people about their presentations), but it’s an interesting read. Gilly Leshed at Cornell also has some nice slides and gives a yearly talk about this at Cornell IS, though I can’t find them on the web to share.

[2] For some reason, it’s kind of cool to be page 1 of the conference proceedings.

[3] Delightfully cheesy prop use, with a curtain between him and the audience for the first part of the talk to emphasize the idea of an unseen audience. It made me reminisce about eating, and talking about eating, an ice cream bar to induce audience reminiscence during the Pensieve talk at CHI 2010. It also made me wish we could have organized a mass evacuation of the room so that when he pulled the curtain down no one would have been there.

[4] There are some amusing gender differences between adults and teens that weren’t discussed in the talk, but are super-clear in the paper; slang, swearing, complaining, relationships, and negativity about people are relatively infrequent for adults but top categories for teens.

[5] Arguments about the perils of information aggregation go all the way back to the Lotus marketing database fiasco in 1990; see Laura Gurak’s Persuasion and Privacy in Cyberspace: The Online Protests over Lotus MarketPlace and the Clipper Chip for more details.

Citations to the papers above are, well, below.

“Managing Social Media”

Xuan Zhao, Niloufar Salehi, Sasha Naranjit, Sara Alwaalan, Stephen
Voida, and Dan Cosley. 2013. The many faces of facebook: experiencing
social media as performance, exhibition, and personal archive. In
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems (CHI ’13). ACM, New York, NY, USA, 1-10.

Yumi Jung, Rebecca Gray, Cliff Lampe, and Nicole Ellison. 2013. Favors
from facebook friends: unpacking dimensions of social capital. In
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems (CHI ’13). ACM, New York, NY, USA, 11-20.

Michael S. Bernstein, Eytan Bakshy, Moira Burke, and Brian Karrer. 2013.
Quantifying the invisible audience in social networks. In Proceedings of
the SIGCHI Conference on Human Factors in Computing Systems (CHI ’13).
ACM, New York, NY, USA, 21-30. DOI=10.1145/2470654.2470658

Yi-Chia Wang, Moira Burke, and Robert E. Kraut. 2013. Gender, topic, and
audience response: an analysis of user-generated content on facebook. In
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems (CHI ’13). ACM, New York, NY, USA, 31-34.

Pan Shi, Heng Xu, and Yunan Chen. 2013. Using contextual integrity to
examine interpersonal information boundary on social network sites. In
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems (CHI ’13). ACM, New York, NY, USA, 35-38.