Beer experiment philosophy – is your Control my Control?

General beer discussion, beer talk, pictures, etc...

Moderator: Brandon

User avatar
lupulus
Apprentice Brewer
Posts: 206
Joined: Wed May 11, 2016 8:35 pm

Re: Beer experiment philosophy – is your Control my Control?

Postby lupulus » Sat May 06, 2017 12:35 pm

+1 on Tech's comments
I have done a very thorough review of the Scientific (capital S) literature on HSA, and my observations are similar to what Tech describes, the methods part of their papers is not very good, and most of these papers would have been rejected in other scientific areas on methods alone.

@ Caedus - nice meeting you :-)
I do not think one can make a distinction between capital S Science vs lowcaps science. There is no such a thing (to me). We cannot make antimatter at home, but it is not beyond us to brew very good beer, which is the media in which we test our experiments; and I do think that blog-posts can be considered science, if they follow science rules.
I was not referring to Brulosophy itself (or the IGOR program), but I wish for homebrewers to realize that if they wand to test a hypothesis, they have much better odds if they follow some simple rules (see signal to noise ratio comments) ; and many Brulosophy experiments do not help in this regard, inasmuch as many times they do not critically choose the best style for their experiment (minimize noise) nor they do not assess whether the control beer is a good representation of the style (minimize noise); among other things. Also, if you have done research, you know that sometimes you mess up; if you mess up, just do it again, do not proceed to tasting, and do not publish it. On the other hand, some Brulosophy material is very good; the first Brulosophy podcast on kettle trub has an excellent literature review by Malcolm Frazer.
Ich trinke Bier nur an Tagen die mit G enden , und Mittwochs
caedus
Apprentice Brewer
Posts: 167
Joined: Mon Apr 25, 2016 3:27 pm

Re: Beer experiment philosophy – is your Control my Control?

Postby caedus » Sun May 07, 2017 12:32 pm

Well said techbrau.
TheHairyHop
Posts: 49
Joined: Tue Oct 25, 2016 12:26 pm

Re: Beer experiment philosophy – is your Control my Control?

Postby TheHairyHop » Wed May 10, 2017 10:17 am

I routinely see three issues with brewing experiments. I won't be shy to admit that all of these are routinely present in Brulosophy articles. Being the King is not easy.

1) Being unable to prove correlation does not equal disproving correlation. This bleeds into the "sum of the parts" arguments. If my process or beer is able to hide a flaw from an introduced variable (and I am unaware of this philosophy), I could be quick to say "X variable does not change my beer"

2) Triangle tests only prove so much and the result is often heavily interpreted. For instance, trying to decide if people can tell the difference between two hops. It's totally possible that drinking these two beers back to back would make it harder, not easier, to tell the difference between the two. If I woke up one morning and was asked, is this beer singly hopped with Citra or Mosaic? I would probably perform better than if I was given three glasses of heavily hopped beer and asked to tell them apart. The human tongue (and brain) is complicated.

3) If you are searching for "statistical significance" it is certainly possible that you will not find it (even if it truly is), especially when using a single metric. Statistics is more than just a one or two or even three trick pony. To go back to the hop example, if I fail to reach statistical significance on a group of blind triangle testers, I may be inclined to say there is no significance. However, what happens when you re-test those who got the difference? Do they repeat the feat? If so, this is very valuable data. Perhaps a majority of people cannot taste the difference, but some indeed can. A single triangle test does not cover this.
User avatar
Weizenberg
German Brewing
Posts: 843
Joined: Wed Nov 11, 2015 2:11 pm
Contact:

Re: Beer experiment philosophy – is your Control my Control?

Postby Weizenberg » Wed May 10, 2017 6:17 pm

I leave this to the academics in the respective departments who have the necessary intellectual foundations and equipment.
The Quest for Edelstoff - http://edelstoffquest.wordpress.com
User avatar
Owenbräu
German Brewing
Posts: 1196
Joined: Tue Dec 22, 2015 8:23 pm

Re: Beer experiment philosophy – is your Control my Control?

Postby Owenbräu » Wed May 24, 2017 9:21 am

I think one other thing to consider is a repeated measures design. One snapshot in time is just that. It doesn't capture the evolution of flavors or help reveal at what point in the process flavors change.
- The best do the basics better -
User avatar
Weizenberg
German Brewing
Posts: 843
Joined: Wed Nov 11, 2015 2:11 pm
Contact:

Re: Beer experiment philosophy – is your Control my Control?

Postby Weizenberg » Wed May 24, 2017 12:50 pm

Well, what else can an amateur do? Triangle tests are reasonably easy to arrange (people love free beer), but when it comes to accurate and OBJECTIVE data (ie correct measurements, real and reliable data), then that becomes astronomically expensive really fast.

Not that I am against triangle tests or homebrewers doing experiments for as long that the inherent limitations are understood and accepted.
The Quest for Edelstoff - http://edelstoffquest.wordpress.com
User avatar
Owenbräu
German Brewing
Posts: 1196
Joined: Tue Dec 22, 2015 8:23 pm

Re: Beer experiment philosophy – is your Control my Control?

Postby Owenbräu » Wed May 24, 2017 4:46 pm

I'm not advocating any particular test, be it analytical or organoleptic. Whichever you select, then repeatedly measure using that method over time with a batch to get multiple data points along the process. If you select triangle tests, for example, then perform them in the mash, post-boil, post-ferment, after spunding (or force carbonating), after lagering for 1 week, 2 weeks, 4 weeks and 8 weeks. Sometimes, you may lose quality long before the variable you were originally testing. Say for example, you want to test Narziss cold vs Narziss warm fermentations, but you've already worked over, oxidized and/or heat stressed the wort. If you tested repeatedly along the process, then you would know the results were affected long before you even got to the fermentation. If your test is analytical, say measuring DO or sulfite consumption, then the same applies. You would see problems with the system long before you got to the intended control vs treatment experiment. This harkens back to the holistic approach to experimental design and testing.
- The best do the basics better -
User avatar
Weizenberg
German Brewing
Posts: 843
Joined: Wed Nov 11, 2015 2:11 pm
Contact:

Re: Beer experiment philosophy – is your Control my Control?

Postby Weizenberg » Wed May 24, 2017 5:03 pm

:tu
The Quest for Edelstoff - http://edelstoffquest.wordpress.com

Who is online

Users browsing this forum: No registered users and 11 guests