While Consumer Reports is an invaluable tool for buying the best among many different types of appliances, cars and widgets, when it comes to beer … not so much. Last year, Consumer Reports Rate[d] Mainstream Beers, with horrible results. More recently, they released their choices for the “Best Craft Beers,” stating that their “experts did blind taste tests of 23 ales and lagers.” The whole report will be in the August 2013 issue. As I’m on a deep deadline, I’m not going to be able to dissect where they went wrong, so I’ll instead happily defer to Jeff Alworth’s take, Consumer Reports on Craft Beer: #Fail, in which he describes how their “hardy band of untrained, eager tasters” chose 23 beers and “put them through their rigorous — if wholly clueless — tasting regime.” Some of their choice conclusions include that “the best ales have intense, complex, and balanced flavors” but “the best lagers are very tasty but not quite complex or intense enough to be excellent,” while they describe the worst beers as “decent but not as balanced, complex, or intense as the others, and some have off-flavors—hinting of cheese, soda water, or even paint.”

And while I wholeheartedly agree with Jeff that anyone with or without formal training can competently judge beer, I also agree that it can expose a “massive blind spot … when untrained people try to assess [the] ‘best’ among a widely differing group of beers with no regard to tradition or style.” Any of that’s fine if you’re sitting around your kitchen table with your buddies sampling the lastest haul from a beer run. But if you’re a massively influential consumer ratings group, you should take it a little bit more seriously and not just assume that the way you rate everything else will work in every instance.

People may laugh, but judging beer can be hard. (I assume at this point, I’ve become the equivalent of the supermodel complaining about standing around and looking sexy while people snap her picture.) But there’s more to it than simply swigging a few beers and saying which one you like best. Anyone can do it, but in order to do it competently and well, you still need to put in the time, gain the experience that’s necessary to calibrate your palate and understand what you’re drinking and why it tastes the way it does. It’s not magic, but you can’t learn to do it overnight, either. Like any learned skill, you have to keep doing it until you gain a certain competence, and then continue doing it to maintain or improve that level of proficiency.

I’ve judged beer on three continents, in local, regional, national and international competitions over a period of decades. No matter how often I do it, it’s always a challenge. There’s always something new to learn and appreciate, new combinations of aromas and flavors to consider against others, and a new understanding of the beers you’ve tasted. As Alworth nicely concludes. “Context matters. If you don’t understand why a beer tastes the way it does, you’re not going to appreciate the flavors you apprehend.” And that, I believe, is why Consumer Reports continues to do a great disservice to their readers when it comes to beer.

Oh, and one last snide remark. They have Samuel Adams listed among “the best ales.” Without identifying it, I can only assume that’s the Samuel Adams Boston Lager they’re referring to, which suggests a certain sloppiness, don’t cha think?

The final “Best of Show” judging tables at this year’s California State Fair.