The Article in Question
A recent post from Robert McKeon Aloe critiquing my study on espresso filtration methods was brought to my attention this past week. First things first, I’m glad to see thoughtful criticism on my work and especially glad to see that including the spreadsheet of raw data from that experiment was useful to others.
In that post, which I would highly advise you go read before continuing here, Robert breaks down why he feels the data leads him to believe filtering espresso samples is unnecessary. I will confess I went into the article expecting the usual lack of understanding of how refractometers work wanting to measure a “complete” sample, but after reading through the article I feel he makes a good point in certain contexts. I would not, however, recommend going without filters as a blanket statement but rather only for certain specific use cases and I’d like to explain why.
Robert’s Points
Robert feels that by using a simpler plot, in this case a scatter plot, a close correlation between the unfiltered and filtered samples becomes evident. I, however, feel a scatter plot oversimplifies the data by better allowing error to cancel out when taking the data set as a whole. There are several points in Robert’s article that I do not dispute:
- When looking at a set of data, unfiltered samples correlate strongly with filtered and centrifuged samples
- A simple transformation can be used to closely match a set of data taken from unfiltered samples to its expected filtered value
- There are situations where this transformation adequately replaces the role of filtration
Where I disagree is that the transformation is useful enough to discard filtration in most circumstances.
Why I Still Feel Filtration is, Often, Necessary
Robert’s transformation works well on data sets. If you’re using your refractometer to track data, the transformation could be quite useful. The error averages out such that a simple transformation will show you the same trends and the same overall picture as using filtration would have.
Where it falls apart is on data points. If you need to know the details of one individual shot, and there are immediate consequences for that one TDS measurement being even slightly off reality, a transformation useful to find trends does you no good. Take a look at the data on the experiment page and you’ll see that any transformation you could perform on a single sample by itself leaves nearly half of the unfiltered samples as potentially misleading. So let’s look at a few use cases to see where you can get away with not filtering versus ones where you can’t.
Why are you using a refractometer?
In order to determine if filtration is necessary for you, it’s important to know your use case. There are probably others, but I’ll break down what I see as five of the most common use cases.
Quality Control
- Readings are taken randomly throughout some period of time
- Readings are used to spot check shots against a recipe
- Consequences for a bad reading depend on your procedures (acceptable variance, samples taken, etc)
- Design your procedures very carefully if you wish to use unfiltered samples here
- Note that procedures that work will with unfiltered samples will require looser (but probably more realistic) acceptable variance
- I believe this to be the most common use case for a refractometer in a cafe, and therefore in general
Dialing In
- Readings are taken after each shot
- Readings are used to decide how to adjust espresso parameters for the next shot
- A single reading slightly higher or lower than reality could turn you around in the completely wrong direction, wasting lots of time and coffee
- I would never recommend using unfiltered samples to this end
- However, if you’re ok with fuzzier guidelines and plan on less very fine tuning, it should be viable
- I also believe this to be the most common use case for a refractometer in a home setting
Data Analysis / Tracking
- Readings are taken after each shot
- Readings are used to analyze a set of data for some purpose after the fact
- A single reading being off is mostly irrelevant as long as the trends can show through in the end
- I see nothing wrong with using unfiltered samples here at all
Evaluating Coffees / Equipment
- Readings are taken at least several times with the same coffee/machine/grinder/etc
- Readings are used to determine solubility of a coffee, effectiveness of a machine/grinder, etc
- Many readings are a must to have any idea of solubility at all over the noise of shot to shot variance
- I see nothing wrong with using unfiltered samples here either
Showing Off
- Readings are taken after each shot
- Readings are used to showcase high extraction yields
- Any significant chance of your measurement being higher than reality post-transformation destroys its credibility
- Frankly, I find this rather obnoxious unless you have a real compelling new technique to share, but it’s a common enough use case to include
- Actually just don’t do this
Making Data Points Into Data Sets?
Obvious question in regards to the above: Couldn’t you just take multiple samples to make unfiltered samples useful in all applications?
Sure. But it’s going to take a lot longer getting several good readings than sending one sample for a trip around the centrifuge. And it’s all hands on time vs mostly time you could be doing other things too while the sample goes for a spin. If that much time and effort is preferable than the relatively low one time investment of a centrifuge, I see nothing wrong with that course of action.
Conclusions
There’s value in simplification, but it can also miss important details. Averages are sometimes, but not always, useful. Looking at a practical problem from a statistical standpoint is a dangerous endeavor and you need to stay vigilant in recognizing how the context of what data you have in real daily use differs from what data you have in controlled experimentation.