By Harry T. Lawless, Ph.D.
Professor Emeritus
Cornell University
Once upon a time there was a need to certify the quality of some economically important food commodities that were major exports or important agricultural products of various countries. A good example was the export of butter from 
Cork in Ireland in the middle 1800’s. There is even a butter museum in Cork City today. Product quality would be certified by qualified expert graders, which would insure the appropriate sensory characteristics were present in that commodity as it left the docks for transportation.
This tradition of expert quality grading persisted for some time, sometimes with a government certified symbol of the guarantee of quality to consumers. A good example is the dairy product grading system in place in the U. S. since the late 19th century. These systems persisted, and even so today, in the form of American Dairy Science Association (ASDA) grading systems for fluid milk, butter, cheddar cheese, cottage cheese, yogurt and vanilla ice cream. There are even student judging competitions (collegiate level) of teams trained to recognize defects and score products based on ideal qualities and well-known defects that can occur in production, storage or packaging. Many of these systems for point assessment/deduction date from about 1910 to 1920. They have sometimes been characterized as a “negative QDA” although the relationship to descriptive analysis is tenuous at best. These systems were important in quality control, and training manufacturers to avoid common defects that could occur in processing, packaging and storage.
Historically, these systems were of little use to the food industries that flourished after WWII, whose innovative products bore little resemblance to standard
agricultural commodities. Those industries made highly engineered and processed products such as “instant” stove-top stuffings, soups, stews, frozen dinners, instant fruit beverages (for astronauts), quick-cook rice, breakfast cereals, gelatin desserts, and so on. The standard grading schemes were irrelevant, although for some common and economically important and minimally processed products, they were useful for quality control purposes.
In this environment, a system of modern scientific methods of food sensory evaluation sprang up to replace the old fashioned systems of expert quality judging. The central dogma of modern sensory evaluation was to have a set of methods that matched the research questions at hand. Was the issue one of simple difference/equivalence (discrimination testing), or of specifying the nature of the difference (with trained panels for descriptive analysis), or assessing the acceptability of the product (hence consumer testing)? These three types of tools proved very useful in product development, process modification, packaging studies, storage and shelf life, ingredient substitution, cost reductions, and nutritional improvements (as a few examples).
However, quality scoring systems persisted. Often this was because they were entrenched in various academic institutions or professional organizations. When I came to Cornell, there were several goat farms in the area making cheese and ice cream, and the first reaction to the need for evaluation of goat products was to pull out the ADSA score card for vanilla ice cream (from cow milk!) in order to evaluate the quality of goat’s milk ice cream. Nonsense. “Horse of a different color” entirely.
So a battle ensued between the pioneers of modern sensory testing such as Pangborn, Stone and Sidel, O’Mahony and others, to show the weaknesses of quality scoring, and how it was ill-suited to research problems in processed and engineered foods.
Then the debate arose as to what was in fact the nature of food quality. A whole issue of Food Quality and Preference was devoted to this matter. The consensus, if I can summarize it, was that true quality resided in the perception of the consumer, not experts. The potential discrepancy between experts and consumers was readily acknowledged, and Bob McBride went so far as to describe it as an “inevitable” discrepancy (his example was from cheese). As an example, the ADSA standard for cottage cheese is a curd approximately the size of a pea, uniform in size, and when dropped from the height of the waist upon a tile floor will bounce approx. 6 inches. And so cottage cheese with smaller curds is graded down as “shattered” and points are deducted from the overall quality score. However, and entire market segment has sprung up for “small curd” cottage cheese, because it is well-suited to other uses such as a faux ricotta. One man’s meat is another man’s poison. 
A guy at 3M makes “bad” glue, but the next thing you know we have little yellow sticky notes, which now are an essential office staple. J.-X. Guinard once reported at a Pangborn meeting that U.S. consumers actually preferred olive oil that was somewhat oxidized, although European olive oil graders would deduct points for oxidation.
Another problem with quality scoring methods is the entire notion that there is one Platonic ideal for a given product. As we see in many products, different (sensory) styles of products are preferred by different market segments. There is no single perfect coffee, not one perfect lager beer, and so on. Malcolm Gladwell had a recent Ted-talk segment about this in discussing Howard Moskowitz’s contributions to the notion of market segmentation.
So the sensory evaluation community, at least on this side of the pond, remains quite skeptical of quality scoring schemes that depend upon the opinions of expert graders.
This stands in contrast to the recent demand for PDO-certified products, especially in Europe. Various traditional products, from delimited geographic areas, have been proposed for some kinds of PDO certification. This is not unlike DOC wines from the major European wine producing areas. So the historical precedent is there. Quality index methods also persist in seafoods.
In response, and with all good intentions, various food scientists have established panels (sometimes well-qualified) to set up grading and scoring schemes for proposed PDO items. A surge in activity in Europe has occurred Some of these quality scoring systems have been published in peer-reviewed scientific journals.
The systems published for sensory PDO certification have several moving parts:
- Sensory specification of the so-called typical product. What must it taste, smell, look and feel like? What disqualifies a product if it has other non-typical qualities?
- Who decides what these qualities should be?
- How should a sensory panel be trained and qualified to judge such characteristics?
- How should points be allotted or subtracted in a scoring system to numerically quantify the candidate products for certification or disqualification?
In answer to the last question, some scoring schemes have become quite elaborate (Baroque is the word that comes to mind). In some ways they resemble the old 20-point scoring systems for wines, only much more complex.
Current practice may be summarized as follows:
- Convene a panel of persons familiar with the product in question.
- Exhaustively characterize the sensory properties of the product (not just defects).
- Decide which attribute are important in the unique authentic sensory profile of this product and which other attributes, if present, would make it less authentic.
- Develop a weighted scoring scheme to allot points for various characteristics or subtract points for others that are not appropriate to the product.
So far the industrial sensory evaluation community has had little to say about these methods, but perhaps it is time for serious discussion. There will be no shortage of Italian cheeses and Spanish wines that can be subject to this approach, and I predict a landslide of papers submitted to sensory journals. Editors and reviewers alike need some perspective on these methods and whether they qualify as scientific, or merely arbitrary scoring systems with an illusion of quantitative methodology.
Sample references
[1] PDO will here represent products with “Protected Designation of Origin,” with the implication that this is a food (or beverage) item from a delimited area, with a well-known and characteristic sensory profile. A further implication is that there may be a government-approved method for certification, and furthermore, said designation may carry legal implications for truth-in-labeling.
