The “post-publication feedback” game
After a paper is published, researchers usually proceed to the next project, and do not speak too much about what has been done. This is unfortunate, because much can be learned by discussing what has gone well or wrong in the paper, and what can be used to make the next publications better. The “post-publication feedback” game is a way of getting feedback from the authors of a paper, without taking them too much time and without boring them.
Participants: This game should be done after one or two months after a paper is published. All the authors of the paper should participate, although other members of the lab can come as well.
Game: In the game, the group leader or the facilitator splits a blackboard into a few sections. You should choose only three or for section titles from the following list (explanation of section titles is given within parenthesis):
- Message of the paper (Can you resume the most important message of this paper, in a sentence? Sometimes, authors of the same paper may disagree on this, and it is interesting to know if it is the case)
- What would you change? (What would you change if you had to start this work from the beginning?)
- What analysis will you keep? (Which analysis/method/type of visualization did you find so useful that you will use it in other projects?)
- What went well?
- What went wrong?
- What’s next? (What is the next thing to do to continue this work?)
After the blackboard has been splitted into section, the facilitator should give three post-its to each person. After giving 5 minutes to everybody to write, each person is asked in turn to post a post-it to the blackboard, in any section. Each person will be given 2 or 3 minutes to explain what he wrote, and then the turn will pass to another person.
After finishing the game, it would be a good habit to copy all the contents of the blackboard and put them into an archive.
Notes: This game must be very quick, and should not last more than half an hour. There are three reasons to keep this short. First, we want to avoid to create discussions between members of the group (your results are all wrong! I could not finish my part because you didn’t give me the materials! etc..). If someone talks for more than 5 minutes, the group leader or the faciliator should interrupt him and proceed to the next person. Second, if people are aware that there it will be no discussion on what they will say, and that they won’t be forced to explain much of what they wrote, they are more likely to give negative comments, which is what we want to get. The third advantage of making this game quick is that people won’t get bored after the meeting, so it will be easier to propose them to do the same after future publications. In theory, members of a group should get the habit of making one such meeting after each publication, and not be depressed by it.
very nice post on your blog.
thank you :-)
Hi Giovanni, I work at EMBL-EBI (along with the colleague who works in UniProt, actually!), and I use Gamestorming, Innovation Games, and similar, home-made structures a lot in workshops. As it happens, I am facilitating one tomorrow on ontologies.
I'm also involved in a Cambridge gamestorming group (currently on hiatus) where a range of (usually UX people) get together to try out differen tgame structures in a "safe" (i.e. non-operational) environment.
Really busy right now but happy to discuss it all sometime.
I am going to move this to the news section - it fits right in next to Rosalind.
Interesting concepts - come to think of it some of these could be applied within lectures
Agreed. These are great ideas. There is a whole field of research in scholarship of teaching and learning about using "active learning" approaches like these sorts of games in lectures.
Hi again,
Yes, the Gamestorming book is popular with certain people here at EMBL-EBI. I also like the Innovation Games book for similar techniques. In addition, there are things that I or my colleagues (particularly the two others who are involved in user experience design) have come up with that we know work.
I have some favourites (in no particular order): 3-12-3 for brainstorming (sometimes adapted to 5-15-5, to be honest!); The 4 Cs, for breaking down problems; empathy maps (for better understanding users, funder, other stakeholders); what I call NVA: nouns, verbs, adjectives, again for breaking things down; the speedboat game, to identify things that are holding back an idea of project; basics like post-ups, affinity mapping and dot-voting, of course; I use speed sketching to help people explore design issues (especially UI issues) together; draw the box is a great exercise for working on the "vision" for something; elevator pitch is then very useful for helping people describe it; various kinds of ice-breaker;
Enough?! :)
// Francis
Hello Francis, thank you very much for answering! If you want, you can copy the answers below, so people will vote you for them.
I am glad to know that other scientists use these techniques and that they are useful. I practiced the elevator pitch in the past, and use a lot of post-its and maps. I definitely have to try the 3-12-3 and the 4Cs :-)
Dear Francis, Are all of these techniques you mention found in the Gamestorming book mentioned above?
//Genbio64
Hello,
Sorry for the very long delay in response! No, they aren't all in Gamestorming.
See also the Innovation Games: http://innovationgames.com/
I describe speed sketching here: http://ebiinterfaces.wordpress.com/2011/11/10/speed-sketching-diy/
I should probably write a description of NVA, too! It's very simple. You have a kind of "focus group" to discuss a particular topic or design problem. As facilitator, you listen to the discussion, perhaps prompting and questioning occasionally. Filter the discussion for nouns (the things the group are talking about), verbs (actions that may take place), and adjectives (descriptions of things or actions)... write all these up on a big whiteboard or flipchart. It gives the group a way to visualise what they are discussing... I suppose it is a bit like generating a word cloud! Do that for a while (20 - 30 mins, maybe), and then discuss the words you have picked out. The group can begin to explore and prioritise things some more, and it can lead you into other "game" structures.
cheers, Francis
Hi Giovanni, here something about gamestorming: Four Articles & Papers Defining this thing called #Gamification: http://tinyurl.com/c7c3twm
Thank you for the comment mvschneiderg! Although gamestorming and gamification are slightly different. Gamestorming is more about how to organize group meetings, while gamification is more about designing user interfaces inspired by games. Thank you anyway!