A Taste of Perception: Examining Bias in Side-by-Side Coffee Tastings

By James Scott, Crown Barista

Over here at The Crown, we taste a lot of coffee–so much that we probably spit out more than we swallow (can you believe that?). Between slurps, we describe each coffee in as much detail as possible. We are encouraged to be creative and specific, which sometimes produces grandiloquent and/or superfluous language to delineate our subjective experience of the coffee. Every week, as part of our Crown Jewel analysis, we brew the same coffee and alter one variable (e.g. brew device, grind size, water temperature). We then taste the two side-by-side and list our flavor notes for samples A and B. After numerous cuppings and several of these CJ analyses, I witnessed how we tend to generate completely different flavor notes for the same coffee. For A we may write “Apple, Caramel, and Cardamom”, while for B write “Orange, Melon, and Milk Chocolate”.

It is worth mentioning that the very intention of changing a variable is to see how it affects the flavor and, most of the time, it does. I do not claim that these differences are not present; in my experience, I have noticed that the same coffee can taste different from brew to brew—even using the exact same recipe—because of the complexity of extraction.

Yet, I am more interested in what I perceive to be a learned tendency to differentiate coffees. Most of the time, this ability is extremely helpful and an essential skill of a coffee professional. Nevertheless, I wonder if our large coffee vocabularies and well-developed palates ever get in the way and blind us from accurately assessing coffee? How important is context while tasting coffee? More specifically, would people notice that they were served coffee from the same brew if it were separated in glasses labeled “A” and “B”?

Few people understand the nuances of coffee better than my coworkers, consisting of a few veteran baristas, two Q-Graders, a Q-Instructor, and another who has worked in roasting, QC, and Green Coffee for over 10 years. I figured that this would be the perfect group on which I could test my hypothesis that presenting the same brew of coffee in separate glasses will provoke the taster—consciously or not—to differentiate the two samples.

Before conducting the experiment, I scoured the internet for any research that could elucidate the phenomenon I was exploring. Unfortunately, I did not come across any previous experiments that employed the exact same method; however, I did discover an illuminating article about how taste can change sip to sip—albeit in an article about wine. As with most drinks, coffee gets our salivary juices flowing. This increase in saliva “can reduce bitterness as well as the astringent mouthfeel [of the coffee]”. I therefore hypothesized that the drinker would find “B” to be sweeter, assuming that they took just one or two sips of “A” beforehand. Also potentially impactful is the food or drink that they previously consumed. The author notes that, “If the food was particularly acidic, it may cause the [beverage]—perhaps paradoxically—to taste less acidic.” Nevertheless, I did not control for this consequential variable and would suggest that further studies restrict the intake of any food or drink besides water for 30 minutes before the experiment.

My burgeoning curiosity in calibration, biases, and subjectivity in taste, manifested in the following experiment:

I brewed up CJ1310 Uganda Kapkwai Sipi Falls Raised Bed Natural on Fellow’s Stagg X Brewer using 25 grams of coffee and 400 grams of water (1/16 ratio). I then split the homogenized brew into two carafes, labeled “A” and “B”, that I previously heated to ensure that the coffee would be at about the same temperature when I poured them. I took this step to eliminate—or at least mitigate—the potential that the tasters would perceive flavor differences based only on the temperature of the coffee. After waiting a couple minutes, I poured the coffee from carafes “A” and “B” into separate glasses and asked them to write their tasting notes on a sheet.

I did not disclose that the two glasses of coffee were from the exact same brew until after all of my colleagues finished writing down their notes. Needless to say, they were flabbergasted that they had just tasted “identical” cups of coffee. Table 1 substantiates my hypothesis that the participants would not realize that they were tasting the same thing, leading them to (over)differentiate between “A” and “B”. Moreover, all six of them had a clear preference between the two; all but one preferred “A” over “B”, which contradicts my hypothesis that they would find B to be sweeter after their mouths adjusted to the tannins in coffee. Given that 5 out of 6 had a clear favorite, I would encourage further experiments with a larger sample size to explore what this study revealed as a preference for “B”–i.e. the 2nd of two cups.

After completing the experiment, I came across an article that my colleague, Chris Kornman, wrote in 2016, exploring how cup color impacts flavor perception. He approached the issue by serving four different coffees into five different colored mugs (black, red, blue, white, cedar) and assessing people’s ratings of sweetness, acidity, and viscosity for each cup. Despite our different intentions and methods, we arrived at some similar conclusions about consistency in flavor perception. Chris highlights that the majority of tasters in the experiment scored the same coffee differently depending on the color of the cup. Although it was difficult to draw widespread conclusions, as Chris explains, “Our gustatory perception is most definitely affected by visual stimuli, but not in a consistent pattern depending on the coffee and the evaluator”. In fewer words, he concluded that we struggle to taste things both accurately and consistently.

At the same time, we have several examples of people excelling at tasting and differentiating; look no further than the Cup Tasters Championship or the thousands of Q Arabica Graders around the world to witness incredible achievement in tasting and calibration. How can we explain the paradox that, when it comes to tasting coffee, us humans can be flawless one day, and god-awful the next?

I believe that the key to understanding this contradiction lies in the context in which people are tasting. A couple common scenarios come to mind. In the coffee industry, several of us are familiar with triangulation cupping, which is a key component of the Q Grader Exam. Different from a standard cupping, a triangulation challenges the taster’s ability to perceive small differences in coffee characteristics. In standard triangulations, two of the three cups are the same coffee, while the third is different. In this context, the taster is looking to identify two identical cups. Had I modeled my experiment after this manner of tasting, I am confident that most, if not all, of them would have been able to correctly identify the two cups in the context of a triangulation.

This differs from the context of the CJ Analysis tasting, which I utilized for my experiment. Although I did not prompt or mislead the taster into believing that I altered a variable, through repetitive tastings where they were told that the two brews were, in fact, different, they were probably anticipating that this tasting would be structured the same way. Evidently, my coworkers were blinded by their expectation that they were supposed to differentiate “A” and “B”. My main takeaway from this experiment has been that context is of the essence; if you are expecting something, your mind will do its best to shape reality around that expectation.