Dear reader, it is that time of year. While we all hope for ‘accept, minor revision’ the reality is this time of year brings with it a lot of disappointment: Conference rejections, journal revises and resubmits, grant application abject failures, and, what I have most recently been coping with, scathing student evaluation feedback. Yes, the January blues for academics are a bitter, frustrating, and utterly depressing time of year… if you don’t know how to take feedback.
Some time ago I wrote a blog post about how bad reviews exist because no one teaches academics how to review. I’d like to think of this post as a companion piece about how no one really teaches academics how to read reviews of their work either. Most of the advice I was given in grad school amounted to “Well, if the journal says to revise your article, revise it. Or don’t, and fight them on why the suggested change is nonsense.” While well intentioned, this advice doesn’t work. It doesn’t work not only because it takes a combative- rather-than-collaborative stance and assumes your reviewers are your opponents and not your assistants, but also because not all feedback you get as an academic is revise-and-resubmit. Grants, for example, give you a huge REJECTION and very little feedback about what you could’ve done better. And deal with it, they say. In a similar vein, student end of semester evaluations have left me scratching my head on more than one occasion. I’ve gotten feedback so out of left field it made me question whether or not the student and I were ever in the same room! (We were, I take attendance.)
Feedback on research and teaching is often perplexing, frustrating, and hard to not take personally. So what can we do about it? I’ve developed some sanity-preserving strategies to process even the most beastly feedback and listed them here. Now obviously I am no psychologist, so these strategies should not be taken as therapy, medical advice, or anything of the sort. Think of the advice listed here as if you and I were out for coffee and you asked about how I take bad reviews and I happened to have a list prepared of the following 5 easy steps:
1. Check your perspective
2. Take the good with the bad
3. Remember that all criticism is valid- to varying degrees
4. Summarise the takeaways- leave the rest
5. Drink a beer, pet a dog, commiserate with a trusted friend
1. Check your perspective
I tend to focus on the negative. Not just in feedback, but also in life. (Goth4eva!!) But I think that’s relatable, no? We all know that we can have 10 people compliment our new jeans and one person say it makes our booty look flat and that’s a day ruiner. Not only a day ruiner, but we also might think about putting those trousers in the bin when we get home.
Since I know I tend to focus on the negative, I know that I need to pull back and look at the big picture of the review as a whole. Rather than think, “It only takes one apple to spoil a whole bunch,” I focus less on the spoilage and more on the whole barrel of fruit. In the above example, the new jeans were a smash hit! If a product had 10 five star reviews on Amazon and only 1 one star review, would I still buy the product? Probably. I would probably disregard that one star review as a salty outlier- and rightly so.
But how do you gain perspective with journal article reviews, teaching evals, peer evals, comments from well-meaning-but-pushy relatives over Christmas? Well, my own personal answer is a spreadsheet because, ya know, data… No, seriously, I really do put all comments into a spreadsheet with the total positive, neutral, and negative comments at the top (I’ve included an example screenshot below). Why talk about the positives first? Well, I am glad you asked because that’s pro-tip #2! But we will get to that in a minute.
What do I do if the feedback isn’t written down? Like, what if I am in a verbal meeting and being given feedback? Well, I take notes. Taking notes in feedback meetings is vitally important- especially if it is a performance review- and generally considered to be good professional practice. You can type up the notes, any goals/guidelines and expectations and send them to whomever was giving you feedback to check for clarity and ensure no miscommunication has transpired. Plus, if you get feedback which you feel is unfair, unkind, or unprofessional, having a record of what was said and why it made you feel that way is very important if you go to file a formal complaint (but hopefully that will never happen).
2. Take the good with the bad.
I’m sure my very bright psychology colleagues and games user researcher friends (pinging Rachel Kowert, Ben Taels, Elizabeth Zelle) would have a fancy way of saying this, but I’ll be blunt: It is easier to hear a Nice Thing™ first and then a Bad Thing™. If I hear a Bad Thing™ first, then I focus on the bad thing while the Good Things™ are (hopefully) being said. If I hear a Good Thing™ first, then it colours the Bad Things™ as less Bad™. Make sense? No? Oooookay. Here’s a chart:
Put another way, putting the positive feedback first in my chart (example above) not only helps me keep a balanced perspective (like we talked about in #1 above), but also helps me get through the stuff that’s less pleasant to hear.
It is also just good practice. If you want to revise or revamp a course you are teaching, it is as important to keep the parts that are working as it is to change the parts that are broken. If you only listen to what’s bad, you may be at risk of messing up, changing, or removing the parts of the class that were very effective for learning.
3. Remember that all criticism is valid- to varying degrees.
This is the point I personally struggle with the most. Getting criticism on your work that comes out of nowhere, from a different paradigm, or feels completely irrelevant is frustrating and irritating. But we all know that you should never outright disagree with a review or be anything less than grateful for the feedback. Publicly. (We will get to what to do with bad feedback privately in #5).
Just last year I received a negative review of a paper I had submitted to an Unnamed-but-Well-Known conference. This Strongly Reject review claimed my paper should be, well, strongly rejected for a litany of sins. And although I do enjoy a good sin, I did not feel that the feedback in this specific case was warranted. In particular, the review claimed the paper lacked clear methodology. This review comes despite the fact I had a subheading entitled “Methods” where I detailed the methodology I used to collect the data for the study. If you cannot tell, dear reader, I am still salty about this review.
Going back to my own points #1 and #2, I had to read through the other two ‘Accept’ reviews of the paper to gain adequate perspective. Yes, someone probably did piss in Strongly Reject’s cereal the morning they decided to tell me my paper is bad and I am a bad academic, but they had a point buried under all their unprofessional rhetoric. My paper did have a methods section that was very clearly written for anyone coming from a background in qualitative methods or pedagogical studies, but maybe it wasn’t clear to folks coming from a comp sci or a quantitative background.
Now I can sit here and make the argument that Strongly Reject should have identified themselves as lacking expertise in the research method and asked for a paper reassignment. Or I can sit here and make the argument that Strongly Reject should have just Googled the #!@*$ing method. Or I can sit here and say Strongly Reject could have given me the benefit of the doubt that the method was appropriate considering it was well cited within top tier education journals. But I won’t. Nope, I won’t sit here and say any of that because the point still stands: if Strongly Reject was confused by how I described my methods then surely some of the conference audience would be too. I took this into consideration when I revised my paper and presentation for the conference and, honestly? My paper was much stronger thanks to Strongly Reject.
As an aside, I did complain about the review to the conference committee because it was worded in a genuinely negative way that bordered on abusive. The committee assured me they would speak to the reviewer. Hopefully they directed them to my blog post, but who knows. If you’re out there reading this Strongly Reject, it is okay. I forgive you. 😛
4. Summarise the takeaways- leave the rest.
Sometimes a review contains a phrase or sentence that’s a real dagger to the heart. Like, it is so inflammatory either because it is a straight-up personal attack or because it is the antithesis of what you tried to achieve with your work and it gets stuck playing in your head on repeat. For me, a particular phrase will join the chorus of other voices in my head telling me that I am trash, a fraud, and that I don’t belong in academia OR videogames. Oh, that’s right grad students and juniors, the self-doubt and imposter syndrome never goes away. It gets less frequent, sure, but it will never go away. You gotta learn how to manage it.
One of the ways in which I manage my imposter syndrome is by not giving it fuel. I summarise down my feedback to just a few short points (actually, not unlike how I teach my games user research students to summarise feedback for a videogame after a playtest). I use my own words to highlight the key takeaways and then prioritise them according the frequency in the feedback so that when I go to make changes I know how to prioritise. The image below is actual, genuine, bonafide key takeaways which came from a post mortem I did after Fall 2019 student course evals.
Putting the feedback in my own words might seem like an interesting choice. Grounded theory purists would say that using my own words instead of the students’, in the above example, runs the risk of polluting the data set by abstracting the takeaways too far from their original meaning. I counter this critique by saying “yeah, probably, but for my own sanity I can’t stare too long at the blackhole sun of disparaging comments so it is either this or I wont change my classes.” At the end of the day, data is meaningless unless it is actionable so ya gotta do whatcha gotta do.
5. Drink a beer, pet a dog, commiserate with a trusted friend.
On the note of putting feedback into an easier to swallow pill, let’s not forget the social/psychological effects of getting harsh critique. It sucks. It powerful sucks. If you don’t acknowledge that it sucks, you probably aren’t taking the feedback adequately on board. It is important to first emotionally deal with all the bad feels that come from having your work critiqued before you can gain the perspective necessary to make actionable changes.
In fact, in terms of order of operations, I should probably put this pro-tip first in this blog post. I have chosen to put it last because I didn’t want students or colleagues reading this post to skim it and go, “Oh, Ashley is encouraging alcoholism as a way to deal with feedback. Cool.”
I know much has been made about alcohol addiction amongst academics so I cautiously claim a pint as part of my coping strategy. From my years in The Shire (England) I learned that after a rough day there is really nothing better to lift the spirits than an ale, a packet of crisps, and a shoulder to whinge on. Sometimes that shoulder is a dog (or a cat for all you cat people out there) and sometimes it is a trusted friend or colleague.
I highlight the word trusted here because I want to make a clear distinction. If you have a hard day kvetching to a partner, family member, or friend might make you feel better. I’ve found that if I complain about a conference rejection or unfair teaching evaluation to compadres not in academia or games I am met with total bewilderment or well-meaning but ultimately unhelpful advice.
“Who does that journal think they are to tell you your paper was anything less than amazing?” -My Nan.
Great. Thanks Nana. The sentiment is appreciated, but helpful you are not.
In these cases having a wonderful work buddy to QQ at is one of the most beautiful things in the world. In my case this is the amazingly supportive Dr. Kelly Boudreau, former co-lecturer and current bestie. Kelly and I worked together for a couple years back in 2014. She is a games academic with industry experience so she’s seen both sides of the coin and has also seen me at my best and worst. She is the perfect person to talk to about a shitty review because she understands, can relate, and (MOST IMPORTANT) is discrete.
Here’s the part of this blog post that makes me hesitate pushing that post button. Look, let’s be real. We all complain from time-to-time about all sorts of stuff. Even if you have your absolute dream job and are happy 99.9% of the time, there is still going to be that .01% of yuck that you have to deal with and having friends you can talk to about it makes life more bearable. Especially if you are an extrovert like me that needs to process things with other people.
So it seems like everyone complains about their job from time-to-time but everyone also agrees that complaining about work is unprofessional. Here’s my argument: there is a professional way to complain about your profession. To be professional about your unprofessional behavior consider the following components to a successful whinge session: time, place and people. Make sure you are complaining at the right time (AFTER you have had time to process the good and bad parts of the feedback); at the right place (NOT at a venue where concerned parties might overhear); and with the right people (NOT those that might fuel the gossip engine or look down on you/your work/school/journal/conference as a result).
Here’s a handy list (because I love lists!) of dos and don’ts to make your complaining profesh:
Don’t make yourself (or your company, family, friends, teachers, etc) look bad by complaining about something in public. If you are at a restaurant, assume the table next to you knows the editor of the journal you want to complain about. If you are on public transit, assume the person next to you is on the institutional review board for your university and will take offense if they overhear you calling their colleagues fools for rejecting your study. This includes the internet! Please, for all of us, think twice before tweeting.
Don’t complain to the first person you see immediately after getting feedback. I’m very guilty of walking into a co-worker’s office and blowing up about some turd of a review that’s just plopped into my inbox. Learn from my fails and just don’t. Please? Give yourself time to gain perspective and take the good with the bad before you start complaining.
Don’t exaggerate the situation for extra sympathy. “The reviewer actually said you are worse at writing than an untrained monkey?! What journal was this again? I never want to submit there!” Bad reviews are bad enough. Exaggerations and embellishments often drift into accusations about wrong-doings and abusive behaviour. Abusive or unprofessional behaviour should be taken extremely seriously. If the feedback was abusive- which while rare does unfortunately happen from time-to-time- then the appropriate authorities need to be made aware.
Do call a trusted friend on the phone from the privacy of your own home/room/office. You should trust this friend to not be a gossip or explicitly ask them not to share the information contained in your rant.
Do pat a doggo or a kitty. You can also rant to them all ya want. They are great listeners.
Do report reviews/reviewers who are abusive. As a counter to the above point about not exaggerating feedback, it is very important to accurately report abusive reviews to the appropriate authorities. If it is a journal, look up the editor’s contact information and send them an email with the offending review attached. If it is a conference, contact the content chair and inform them. If it is a supervisor/manager giving inappropriate feedback in a verbal performance review take your notes (hooray for writing stuff down!) to their direct supervisor or human resources manager and file a formal complaint.
There. That’s how I cope with bad reviews of my work. If you have anything to add to the list or if you’d like to give me a bad review on my review of bad reviews, then please do! I love getting conversations started and talking about the stuff in academia which is oh so rarely written down.
Until next time (I have a soap box to stand on),
Ashley
Ashley ML Guajardo 