Judges were asked to rate the brews in a variety of categories on a scale from 1 to 10. For the purposes of scoring, all non- integer results were rounded up to the next integer. Also, people who submitted out-of-bounds scores (like 0, or 10+) were corrected to the nearest legitimate rating. All derived statistics (such as average or standard deviation) are based on the number of valid entries: if a judge left a category blank, this did not count either for or against the entry; it merely increased the weight of each score that was received.
Bang-for-the-buck (highest overall score for an entry, divided by the price per pound) was only calculated for those entries that included cost information, obviously.
Two entries were disqualified:
Truthfully, we would rather not have had this happen. It's not our objective to bar anyone from entering the competition, but we had way more coffee than we could accept, even without bending the rules to allow more in. Next year's announcement will be more explicit about what constitutes a valid entry, and we apologize to those who went to the trouble of bringing their coffee, only to have it go unsampled.
Entry #13 (Prince of Darkness) deserves some special mention. The entrant, JP (not that JP), was visiting his favorite roaster, Oso Negro, in Nelson, British Columbia. That's in Canada, for those of you following along at home. He described CoffeeWars to them, and they got so excited that they donated his entry. This, truly, was amazing: our first entry backed by corporate sponsorship and our first indication that CoffeeWars is known abroad. Amazing, and exciting. Thanks, Oso Negro, for being so cool! Also: it was good coffee!
This table presents the various entries in order of average overall score. In the event where overall scores tied, "Taste" was used as a tiebreaker category. There is one mysterious entry, to which the ballot form seems to have disappeared. It didn't place too highly, so I don't think there is cause for concern, but nevertheless, I wish I knew what it was. The ballot submitted in Jay Dyson's name may have been for two entries; if that's the case, this is most likely the first of the two.
|Coffee Name||Entered By||Score|
|Ground Zero Papua New Guinea||Sweetypig||7.1|
|Oso Negro Prince of Darkness||JP||6.7|
|Sulawesi Peaberry||Rob Nielson||6.7|
|Abyssinia & JBM Wallenford Estate||Jay Dyson||6.7|
|Lavazza Qualita Oro||Gurney & SpyBoy||6.4|
|Caffie Lobbo||Abby Normal & Annoney Moose||6.3|
|Indian River Roastmaster's Choice||Cheshire Catalyst||6.0|
|Silver Canyon French Roast||Tim||5.3|
|Baby's Private Buzz||Dean||5.1|
|Mystery Entry||(is this the other half of Jay's?)||4.9|
|Hot Lava Java||Dr. Vann Harl||4.8|
|Havana Gold||Dr. Vann Harl||3.7|
For better or worse, the individual comments made by the judges have been omitted. Sometimes they were inscrutable (such as "meh." or "shiny!"). Sometimes they were insightful ("nutty bean flavor."). Sometimes they were harsh ("is this water?"). Likewise, they sometimes contradicted the scores on the ballots (one judge made a comment praising the excellence of the entry's aroma, and then rated both bean and brew aroma low; another wrote that the coffee was very good, but then gave it low marks overall). I don't know what to make of this information, except that we will endeavor to make the scoring process a little more rigorous next year. One metric of accuracy, perhaps, is the standard deviation, and we should strive to supply guidelines where it grows so large that it encompasses, say, 50% of the entries.
This table presents the various entries in order of our bang-for- the-buck metric. Bang-for-the-buck is a derived statistic, calculated by dividing the overall score by the cost per pound (in US dollars). Entries for which no price data was submitted are excluded, of course. In the event of a tie, the raw overall score was used as a tiebreaker.
|Coffee Name||Entered By||Score|
|Ground Zero Papua New Guinea||Sweetypig||1.0|
|Oso Negro Prince of Darkness||JP||0.6|
|Silver Canyon French Roast||Tim||0.6|
|Indian River Roastmaster's Choice||Cheshire Catalyst||0.5|
|Baby's Private Buzz||Dean||0.4|
Truly, this is interesting data. First, the overall winner was also the lowest score, in terms of bang-for-the-buck. This just goes to show you that you get what you pay for, but there may be a point of diminishing returns. Similarly, the second place entry overall was also the highest bang-for-the-buck. This just goes to show you that excellent coffee does not have to cost an arm and a leg. Sometimes just an arm is sufficient. Or a leg. Whichever you'd miss less.
In other categories, the winners are as follows.
|Category Name||Winning Entry||Entered By||Score|
|Bean Aroma||Jamaican Peaberry||Hook||8.7|
|Brew Aroma||Jamaican Peaberry||Hook||8.4|
|Brew Strength||Jamaican Peaberry||Hook||7.3|
|Brew Taste||Jamaican Peaberry||Hook||8.3|
|Brew Acidity||Jamaican Peaberry||Hook||7.8|
There's a moral here, too: the winning entry was the winner overall, in all categories. There wasn't just one score that tipped the scales.
Some have hypothesized that the reason that entry received such off-the-scale marks was just that it was the last entry to be sampled, and the judges were simply going crazy with the emotional release of finally finishing their hard work. Perhaps, to some small degree, this is true, but I think not greatly. This was some outstanding coffee, and it richly deserved to win.
OK. That's it for the results summary. All the data is available for download from this site, in case you want to do your own calculations.
|Simple Navigational Aid|
|Back to the CoffeeWars 4 Index|