Dr. Nydam and Dr. LeBlanc recently presented a Real Science Lecture series webinar on August 7, 2024. You can find the link at balchem.com/realscience.
Dr. Nydam and Dr. LeBlanc recently presented a Real Science Lecture series webinar on August 7, 2024. You can find the link at balchem.com/realscience.
Dr. Nydam begins with a brief overview of the concepts from the webinar, all based on understanding and applying information from different types of studies on dairy cow health and performance. Dr. LeBlanc adds that their goal was for the webinar to be useful for people with a practical interest in feeding and managing dairy cows. (4:12)
Dr. Nydam discusses different kinds of bias in research. All studies have some bias in them to some extent, so acknowledging, understanding, and trying to control for that is critical. Dr. LeBlanc describes survivor bias. In the simplest sense, survivor bias can be thought of as who’s alive to be counted. Several examples of treatments causing animals to be removed from a study or a disease-causing animal to be culled are reviewed. (8:24)
Both guests give their perspectives on p-values. A p-value tells us the likelihood that a difference we observe is due to chance. There is active discussion among statisticians about the value of the p-value. Both guests suggest that readers should also assess if the study achieved its stated objective and if there are adequate numbers and statistical power to accomplish the objective. P-values help us understand risk. A p-value does not tell us how big a difference was or how important it was. (18:54)
Dr. Nydam reviews that there are two kinds of study validity: internal and external. Internal validity centers around whether the study was done well. Was bias controlled for and acknowledged? External validity centers around the applicability of the study to the population. Is a study about mastitis treatment in water buffalo in Pakistan applicable to a dairy farm on Prince Edward Island? Peer review usually takes care of assessing internal validity. External validity is more up to each reader to decide for themself and their situation. (29:01)
Scott asks about the validity of field trial data. Both guests acknowledge the inherent challenges of field studies and give some tips for success. Field studies can often have good external validity because they are done under real-world conditions and at scale. (34:23)
The group dives into the topic of industry-funded research. Some skepticism and cynicism about industry-funded research exists. Industry-funded studies are not inherently biased and often answer important and tangible questions for decision-makers. Government funding is rarely going to be awarded to that type of research, but the industry is interested in funding it. If an industry-funded study is well done by a reputable researcher, has gone through the peer review process, and has appropriate methods and statistics, Dr. Nydam sees no reason to discount it. (44:56)
Dr. LeBlanc reminds the audience when looking at different kinds of studies and different types of evidence, it’s not that one type of study is good and others are not. For a lot of health-related research in dairy cows, we don’t have good (or any) experimental models to reproduce things in a white-coat-science sort of way. At the end of the day, dairy managers and industry professionals want to know if a particular piece of science, whether experimental or observational, helps them make decisions on the farm. There’s a place for all types of research as long as it’s done well and in its own right. (42:08)
Dr. Nydam’s key takeaway is that it’s important to remember to keep some faith in science and have open discourse about it as we move forward in dairy science and as a society. Dr. LeBlanc reminds the audience that even if listeners are not in the business of designing, conducting, and analyzing their experiments, they do not need to feel powerless as consumers of scientific information. It can and should be something they can engage with and use to answer questions in their day-to-day jobs. (52:26)
Please subscribe and share with your industry friends to invite more people to join us at the Real Science Exchange virtual pub table.
If you want one of our Real Science Exchange t-shirts, screenshot your rating, review, or subscription, and email a picture to anh.marketing@balchem.com. Include your size and mailing address, and we’ll mail you a shirt.
Scott Sorrell (00:07):
Good evening everyone, and welcome to the Real Science Exchange, the pubcast where leading scientists and industry professionals meet over a few drinks to discuss the latest ideas and trends in animal nutrition. Hi, I'm Scott Sorrell. I'm gonna be your host here tonight at Real Science Exchange, and tonight we welcome not one, but two professors Dr. Daryl Nydam and Dr. Stephen LeBlanc to discuss the webinar that they gave recently on the Real Science Lecture series. We're gonna be talking about Epidemiology 101, I believe was the title of the the Talk. It might have been titled Epidemiology 101, but I it, it, it seemed like a graduate course to me Dr. Nydam that was some pretty in depth stuff and would encourage our audience to go back and listen to that. So we're gonna be looking forward to a, a deeper dive into that topic. But first Dr. Nydam, I'd like to welcome you to the Real Science Exchange. I believe this is your first trip to the pub, is that correct?
Dr. Daryl Nydam (01:07):
This is my first trip to this pub, but not my first trip to
Scott Sorrell (01:11):
A Pub There you go. I love it. Well, this, this happens to be my favorite pub, and so I welcome you here in the theme of that what, what's in your glass tonight?
Dr. Daryl Nydam (01:22):
I've got a, a very robust Macallan single malt, 16 years.
Scott Sorrell (01:29):
Okay. We have a theme going here tonight. I can tell already. Well, welcome Dr. LeBlanc welcome. And this is not your first time to the pub. You've been here before, I think early on, perhaps three years or so ago. That's great. Apologize, it's taken so long to get you back here. You're, you're a popular guest and, and, and welcome once again. So what's in your glass tonight?
Dr. Stephen LeBlanc (01:55):
Well, I, I swear we didn't coordinate this ahead of time, but I also have a glass of Macallan. But, you know, because I'm Canadian and, and don't get paid like a Cornell professor, mine's only 12 years old.
Scott Sorrell (02:11):
With that gentlemen looking forward to a great podcast tonight. Great discussion. Cheers.
Scott Sorrell (02:25):
The economics of Feeding reassure Precision Release choline reassures fed during the transition period. And because it's fed for such a short period of time, it costs just $15 per cow, and yet the benefits will continue to generate income throughout the year, cows fed reassure produced five pounds more colostrum, which pays for your reassure investment on the very first day of lactation. Cows fed reassure also produce five pounds more milk per day every day. That means, after the first day, every day is payday invest in reassure during the transition period and recoup your investment on the very first day of lactation. After that, you got it. Payday.
Scott Sorrell (03:15):
We have found that our, our audience here at the, the pub cast is different than the audience that we have at the webinar. And so what I usually like to do is to invite our, our podcast audience to go back and listen to the webinar. And, and I think this is especially true with this one. And so what I'd like to do is first ask Dr. Nydam, if you will, to kind of give us an overview of what did you cover during the webinar, just kind of the treetops. And then I'm gonna come back and ask Dr. LeBlanc to kinda make the case as to why the audience ought to go listen to that webinar Dr. Nydam.
Dr. Daryl Nydam (03:54):
Sure. At the 20,000 foot level, or at least at treetops, we wanted to get to some of the fundamental concepts when people are consuming quantitative information about dairy studies. And if we title it EPI 101, probably no one will go back and listen to it, or, and we will probably shut off this podcast right now. So it is subtitled cause and consequence, a little bit more cheekily understanding and applying information from different types of studies in dairy cow health and performance. So some of the fundamental things we wanted to really get to were the kinds of studies that are out there. They're not all created equal, and you can take different kinds of information from those different kinds of studies. We also wanted to think about how to interpret some of the precision of the estimates or the numbers that come out of there.
Dr. Daryl Nydam (04:47):
So confidence intervals what, you know, how confident are we in something that we see? And then most science papers also have p values in them. So we wanted to discuss a little bit about how and why to interpret P values. And, and even in the, the statistical community, it's, it's a controversial subject, and, and you wouldn't think it would be. So those are the, you know, sort of the textbook things we wanted to get to. And then we've tried to weave through some transition cow, particularly those things around energy balance to make some tangible examples of these things
Scott Sorrell (05:23):
Very well. Dr. LeBlanc, you wanna make your case for, for why we ought to go back and, and watch rewatch that again? I've seen it twice myself. I think I'm gonna go back at least one more time.
Dr. Stephen LeBlanc (05:34):
Well, there you go. See, it's, it's working and hope, hopefully it's I hope it's interesting and it's not painful. And Daryl reminded me that years ago here at the university of Guelph Vet School you know, one of the first year courses used to just be called epidemiology. And, and almost automatically the students hated it when we changed the name to health management that, that already made medicine go down a little more smoothly. So I think what I can promise is, is that this is not some painful super geek seminar, but like Daryl said we weighed into I hope some practical stuff around things that would be of interest for people who have a practical interest in feeding dairy cows and managing dairy cows and, and, and helping managers and owners to, to do that.
Dr. Stephen LeBlanc (06:25):
We weighed into, so there's a, you know, kind of a, a little bit of a hot controversy about dogmas and what is or isn't a dogma and, and whether ketosis is a disease or whether some cows no cows or, you know, all cows that have a blood BHB above X have a glycol deficiency or a, you know, condition that benefits from some intervention and so on. So we weighed into some of that, and ultimately, it's, it's not about, you know, saying this is right or that's wrong, but, but hopefully equipping people with some of those filters and lenses to, to make some sense out of the pieces of data that are coming at them as they're trying to make practical decisions about running dairies or, or advising people who run dairies.
Scott Sorrell (07:11):
Yeah. Well, I said, you know, it was relatively deep and scientific, but the, the parts that I appreciated about it was, you know, kind of helping me understand some of the background of the different kinds of studies. And, and, and, you know, you know, in my job, I, I consume a fair amount of, of data and, and, and research studies. And, you know, this kinda helped me better understand you know, be a better consumer of that information. I, I don't know if it's a true statement or not, but I found myself kind of contemplating that, you know, this science isn't a, a perfect science, right? There's, there's a lot of confounding factors, and, and one of them is, is, you know bias different biases that can find their way into research study. I was wondering if we could just kind of have a, a brief conversation around that, different kinds of biases and, and how they can confound the data.
Dr. Daryl Nydam (08:06):
And in a book by Ian De who and his co-authors, there's a great chapter on bias, and I think boils it down to three types. If, if you wanna just remember three things, there's selection bias, information bias, and confounding bias. And the trick is not that there is no bias in any of these studies, 'cause all studies have some bias in them to greater lesser extent. It's acknowledging them, understanding them, and trying to control for it as best you can. So, and as we do studies, there are lots of things we want to do in the design and conduct of the study, so you don't have to deal with so much of it afterward, right? Sometimes fancy statistics are what you do after you didn't get the bias dealt with upfront. So yeah, let me kick it over to Stephen LeBlanc as an example of one of the kinds of bias, and this is survivor bias, which is a subset of one of the ones I talked about.
Dr. Stephen LeBlanc (09:14):
Yeah. And, and that's, you know, the, the thing to I think point out is that I think sometimes reflectively people hear about bias and think about the kind of bias we talk about in sort of, I don't know, social or cultural context and where that could be well, it could be conscious or unconscious, but it sort of implies in, in a scientific study that either somebody's kind of cooking the books or, or
Dr. Stephen LeBlanc (10:23):
And so what might happen in this highly controlled or, you know, cell culture or rat model or, or, or even a, you know, a dairy cow in an extremely experimental setting you know, that that's not bad. There's a place for those things. But the, but yeah, there's this sort of sliding scale there. And, and then the, the particular bias that Daryl mentioned survivor bias is, is one of the ones that's kind of a big deal in, in dairy studies, small or large. And, and essentially that's, you know, outcomes. I mean, classically, literally who's, who's alive to be counted. But we, that's, that's a kind of a practical thing for, for dairy field study. So for example if we, you know, feed supplement A or B, and we're looking for a milk response, but maybe there's, there's implications for health or reproduction.
Dr. Stephen LeBlanc (11:19):
Well, who, which cows have a value for peak milk or week four milk, or even whole lactation milk? Well, it's the cows that are around at week four at day 60, or at 305 days, and those are the ones you tally up and analyze and, and do stats on. But it's pretty common, you know, not every cow makes it to week four, today 60, or today 305. And not only that, that that's not random, right? Who, who doesn't make it? Well, typically the ones that had worse health, lower milk production, because producers choose to cull them differentially. And so if our treatment had an effect on health, milk, or fertility, it could knock onto that. And so you might say, well, we saw no difference in milk, or we did, and, you know, one group had more than the other, but at least potentially it's because some of the animals that started out aren't there to be counted anymore, and you, you can't reanimate them or bring 'em back in from the dead. But you do at least need to be aware of that, and you can do whatever stats you want and do what have whatever p value you want on that, that that doesn't eliminate that issue. So that's just something to think about.
Scott Sorrell (12:40):
Yeah, I've got a real world example of that. Stephen, though, in one of my previous lives, we were testing an intervention on its ability to reduce mastitis. And we were actually dosing these animals in a memory with some strep uberis. And many of the control cows got to a level of fever and sickness that we had to remove them from the study. And so at the end of the day, we did not have statistical significance because the, the animals that truly were sick, that, that were, were, were no longer in the study. So I find that very interesting. That's a little different than what you're talking about, but
Dr. Stephen LeBlanc (13:19):
There's another classic example to the different context than what we talked about in the, excuse me, in the webinar. But and that's to do with long-term effects of disease in calves. So quite a few studies have said, you know, what does it mean to a calf to have lived through scours or pneumonia as, as a young calf? Does that have a long-term effect when they grow up and, you know, have their first lactation survive into their second and so on? And most of those studies would say there isn't much of an effect. It's pretty underwhelming as to, you know, how much milk enamel make as a 2-year-old, as a function of whether it was or wasn't sick as a calf. But that's a, that's a great example because typically, you know, if you had a hundred calves born today, a hundred healthy ones and a hundred that got sick about 85% or so of the healthy ones will actually make it to have a, a first lactation only about 65% of the ones that had disease as a calf and survive it in the short term. But they don't survive it in the long term. Somewhere along the way, as a heifer, they wound up getting culled. And so you, you look at first lactation milk on those animals, and it's not really different, but there's this chunk of animals who are missing.
Dr. Daryl Nydam (14:43):
Even paradoxically, sometimes you'll do studies and paradoxically, it'll look like these upstream health events are actually good for milk production, these disorders, because of differential culling, the survival bias, the animals, right? If we're doing something in a non-randomized controlled way, and we're just following the animals, smart dairy farmers are gonna remove low producing animals. So even if they had a mastitis event in a lactating study, or if they had a, as Stephen was talking about pneumonia or scours as a calf, if they're growing well, they'll be kept. We know that growth, the animals often make more milk or cows in a lactating herd. If they're making more milk, they're kept around longer. So sometimes some studies will show, probably wrongly, but paradoxically, that because of this survivor bias or this selection bias, it looks like that disease may have increased milk production. Another example of that is doing things comparing different lactation numbers or different parodies, right. Cows in first parody that do really well are far more likely to have a second one than cows that don't do well in their first parody. They're way less likely to get onto the second one.
Scott Sorrell (16:08):
Yeah. You know, Dr. Nydam, during your webinar, you used an example it was a study done with parachutes to talk about I believe that is to talk about bias. That kind of helped solidify it for me. Would, would you mind maybe kind of sharing that, that example with us?
Dr. Daryl Nydam (16:27):
We so Stephen has used these also to make the similar examples. And we did try to come up with, to illustrate two different things in that one was the lack of evidence from randomized trials. So, you know, the joke was these, this group of researchers combed the literature and couldn't find any difference between those that used parachutes and, and not, right. There was no evidence out there. 'cause It just weren't these trials because they couldn't get anybody to sign up for it. And then in the other example, well, there wasn't any of these trials, so they set out to do one, and they couldn't find an important effect on survivorship among those that did and didn't use parachutes when they jumped out of an airplane. Well, that would be crazy, right? But then if you look at the study they did, right, they enrolled participants who jumped off of stationary airplane wings, essentially, right? They were, you know, three feet above the ground. So of course, a parachute didn't have an effect, but we didn't really study what we wanted to do, even though we really conducted a very rigorous trial, right? We randomly collected people, we randomly assigned them to treatment group with parachute or not. They diligently followed the standard operating procedure of when to jump off the airplane, when to pull the cord to get the parachute to deploy. And yet we found no difference with a very large p-value.
Scott Sorrell (18:02):
Right. Well, well, that leads to a whole different discussion. And another one that I found very interesting, and that, that is around p values, right? We, we look at those, and that's kind of the gold standard of whether or not the data is real or not, at least it was for me. You, you talked about, and you used an example about two feed ingredients. And maybe talk through that one of you guys you know, talk about the relevance of p value and where it's valuable and where it might be misleading.
Dr. Daryl Nydam (18:30):
One, one thing that, that you said there, Scott, is is it real or not? And, and that's a bit of a pet peeve for my, of mine, if the data is rigorously collected and, and we tried to control for bias, it is always real. And what the p value tells us is the likelihood that the difference we observed given some statistical things like the specified model extremities of things, right? If the likelihood of the difference we observed is due to chance it's real, and what is that difference? How likely is that due to chance?
Dr. Stephen LeBlanc (19:09):
Yeah. And, and yeah. And I think that there's sort of the proverbial average person on the street who, who, you know, let's say for the sake of discussion, has no idea what a p value is, doesn't know, doesn't care. And so lots of people who will be listening in as industry professionals have, you know, been to school and, and taken a stats course or, or maybe even gone to grad school and and so on. And so there's a, there's a whole other level of engagement with, you know, numbers and analysis of data and so on. And, and you know, Scott, I think what you said earlier is, is exactly right. For a lot of folks, that's almost the first thing they look at. Whether they're, you know, reading an abstract from a study or, or at a meeting and somebody's presenting results from a trial is to say, well, all right, I'm, I'm gonna be pretty clever here.
Dr. Stephen LeBlanc (19:54):
I'm gonna hone in on the P value. Because that is kind of the arbitrary of, you know, is, is was this, is somebody cut in a corner, is somebody kind of fudging things or, you know, that, that kind of cuts through all that. And that's not necessarily true. In fact, there, that's a whole other discussion. But in the, in, even among hardcore statisticians and nevermind epidemiologists or, you know, dairy industry people, there's a really active discussion literally about whether we ought to report p values at all in scientific studies. I mean, there's not, you know, not, not too many people are like, completely throw that right out the window. But the, the point is, it's not that sort of be all end all. Like this is really what's gonna cut to the chase of, of, of helping us understand the essence of a study or of, of, of some data.
Dr. Stephen LeBlanc (20:49):
It, it, I think, I don't know if Daryl agrees, I, I think it still has its place, but it's not the first thing to look at. It's not the last thing to look at. It's a thing to consider in a, in a context. And that's where it gets a little tricky, right? Is 'cause you can't just zoom to that and be like, okay, there, yes, no above, below, done. Now, now I know this is, this one's, you know, this one had an effect, this one didn't got it. Move on. It's, it's a little more complicated than that. And you gotta look at, you know, what was the actual objective? What did they actually do? Is that relevant for me in my context? You know, if this was a grazing study in New Zealand or Ireland, is, is that relevant for me on, you know, like dairy in Indiana?
Dr. Stephen LeBlanc (21:35):
And, and that kind of thing. You know, how big was the, because that's the other thing that you get into, and I think, you know, that'll loop back to Daryl's. Do I, or don't I add this supplement to my diet example, which we can talk about in a sec. But you know, there's, it's not just the p value, it's, you know, what exactly did they do in the study? You kind of gotta read
Scott Sorrell (22:39):
Yeah. You know, I, you know, practically speaking, I used to have a boss one of my previous stops and, and he'd go on these rants about about p values, you know and an example would be, you know, as scientists, we would thumb our noses at a p value of 0.3, right? But practically speaking, 70% of the time that intervention or whatever is going to have a benefit. And so, you know, I like to play the stock market, right? 70 percent's not bad. I'm, I'm not doing 70%. So, and, and I think that the, the example that that Daryl used on with those two feed ingredients, a lot of it has to do is what's the risk, right? What's the cost and then what's the potential benefit? And, and there's probably a whole other equation that you have to create to, to, to do that, or maybe it's just kind of a gut feel. But anyway, I just kind of, kind of curious your thoughts relative to that.
Dr. Daryl Nydam (23:36):
One of the things that the p value, the magnitude of what we observed in some study, some trial, it gets us away from that gut feel that you just brought up there, Scott, and that that's what we're trying to, particularly as you know, the modern dairy industry ever gets more sophisticated, margins are tighter. It's harder and harder to operate on gut feel. So the more things we can do analytically, I, I think the better off we'll be. However, there is a good friend of mine said dairy farmers like legalized gambling, right? So we do have to accept some risk every day when we go out there to do something on a dairy. And this p-value thing from studies anyway, helps us understand that risk. You, you brought up a p- value of 0.3 and thumbing your nose at it. Well, one thing I can be certain that that 0.3 doesn't tell you, right?
Dr. Daryl Nydam (24:36):
That the study was conducted badly. It may have been a great study. Some people immediately throw out studies with big P- values, it could still be a very important result, right? Another thing that a directly p- value of anything including 0.3 doesn't tell us is necessarily how big the difference was, or certainly not how important it was. Okay? So let's take that 0.3, for example, and let's say we did a very good randomized trial with feed additives, and we came up with a 0.3 and milk was our primary outcome. And in one group we got five more pounds of milk per cow per day. So what that 0.3 would tell us is that that difference of five pounds that we observed, or one greater than that even is due to chance. And as you said, 30% chance that's, you know, not really great betting odds.
Dr. Daryl Nydam (25:42):
I'm, I'm part scotch and part dutch, so I I
Dr. Daryl Nydam (26:32):
And if it costs us a couple cents per cow per day, and we can think of some things there, that is a risk I'd be willing to take. Of course, I'd really like a p value of 0.01, right? Because now it's only 1% chance that that chance, that observed difference is due to chance that's even better. Yeah. But to just try this a little more provocatively, 0.06 and 0.04, I ask this in every PhD exam, what's the difference between 0.06 and 0.04? And the PhD students in their a exams will wax on for about 15 minutes philosophically about things before I stop them. I say, no, listen closely to the question, what is the difference, the arithmetic difference? And they're like, oh yeah, it's 0.02.
Dr. Stephen LeBlanc (27:17):
And that goes back to, to the, you know, for instance if that 0.0 whatever value on the, you know, the, the, the study to look at the supplements, say, I mean, was that done on, you know, 10 cows, a hundred cows, a thousand cows, one dairy, two dairies, or 10 dairies? You know, that probably matters where those dairies like my dairy or my customer's dairy. Those things matter. And, and then it gets into things too, like to follow that example when we want to do those types of studies in, in university herds, that's one thing where we can have cows assigned to individual feeding bins and, and everything's done at the cow level, et cetera. When we want to do those in field studies on commercial dairies essentially those become pen level studies. And, and those also have a place like, there, there's a whole controversy about whether that's valid or not valid. It absolutely can be valid, but you need to understand that we're now counting pens, not counting cows. And that has some big implications for how you analyze the data and yes, ultimately what the p values look like and, and, and so forth. So, so yeah, that's, it's, it's all, it's all gotta be in context, right? That's the point.
Dr. Daryl Nydam (28:38):
There's two kinds there. Internal validity and external validity and internal validity is it's a study done well, right? Did they control for bias as best they can? Did they acknowledge the bias? And then the external validity, right? Is this study applicable to the population to which I want to apply it? You know, and I always make up some silly examples. Well, what if we did a study, you know, about mastitis treatment in water buffalo in Pakistan? Are we going to apply that to you know, a dairy farm at Prince Edward Islands? Probably not, you know, but unless there's no other study, well, maybe it would so that there's no test for external validity. You have to look and say, okay, are these Holstein cows fed at TMR, you know, in a free stall, or are these, you know, open lot corralled jerseys in California? Is that the population? And then you have to decide whether those results apply to results after, though the first thing is you gotta have this internal validity was a study done well?
Dr. Stephen LeBlanc (29:44):
And, and that can be almost a, a never ending process. There's, you know, you can get more and more sophisticated and or critical in that. But, but I think the point is, and and that's what we hopefully tried to bring out a little bit in, in the webinars, at least some first steps. It, it's, that's not just an, an ivory tower thing. I mean, sure, you know, with grad students, we can spend hours pouring over these things in a journal club and, you know, geek out on it. But, but, and, and that's fine. But, but I think the point as you said Steve, is like it, this is having some of those sort of mental checklist items you know, whether you're a veterinarian in the field, a nutritionist in the field or, or, or, or even a, you know, a dairy farmer, dairy manager, where you've got salespeople coming to you and saying, oh, here, look at, here's, we did a trial and look how great, you know, product X is you know, as consumers of, of that kind of information.
Dr. Stephen LeBlanc (30:41):
There's, there's at least a few simple questions and yeah, okay, show me your p- values might, might be one, because if there aren't any, that probably is a little bit of a red flag. But but yeah, beyond that and exactly, you know, it, it can get fancy and you can get into convoluted stats and so on and so forth. But a lot of it is, I don't know, I hesitate, you know, common sense in the sense of like, it, even if this is some fairly sophisticated study about, you know, whatever amino acid metabolism and blah, blah, blah, and does that get me more milk? You know, you don't necessarily need to understand all those details, but if it's clearly presented, you should at least be able to follow. And that, that is actually one of the big litmus tests is like, can, can you follow along? What were they trying to do? What did they actually do? How did they measure the outcomes? You know, those are the kinds of things that a an industry professional or, or a farm manager or owner can absolutely make sense of
Scott Sorrell (31:42):
This maybe a good lit litmus test is that, that it was published in, in a peer reviewed journal because I, I can only assume that most consumers of this kind of data, whether it be a, a dairy farmer or a nutritionist, they're really not gonna sit down and study and try to determine the validity, right? I, I mean, maybe, maybe if it's obvious, but is it fair to assume that the, the reviewers have done a pretty good job of that prior to getting accepted
Dr. Daryl Nydam (32:12):
Most of the time, and particularly in journals where, where Stephen is an editor of like the Journal of Dairy Science Top Flight stuff, and again, splitting this, this validity word into internal and external, the internal stuff should be taken care of by the review process. You know, the study was good. And then as a consumer of the information, you have to ask yourself, okay, is this study, which we'll assume was good? Does it apply to me? Right? Are, are these, you know, Jersey cows in open corral and they're studying heat stress where I'm, you know, in a tunnel ventilated barn in, you know, Manitoba, right? And two good studies, but maybe they aren't equally applicable.
Dr. Stephen LeBlanc (32:59):
Yeah. And, and I would, so to your question, Scott, you know, yeah, I think the fact that, that a study is published in a reputable journal, and yes, I'll put in a shameless plug for the Journal of Dairy Science yeah, that absolutely goes a long way. It's not perfect, you know, that these things are reviewed by humans, but, but by several humans who, who are experts in the field. So yeah, it, it does go a a long way just you know, fun fact in the Journal of Dairy Science a little bit less than half of the papers that get submitted actually ever end up getting accepted and published. And again, that's not to say that the ones that are, are rejected are terrible or misleading or something, but but, but yeah, there, there is a pretty rigorous quality control process in place. So, so that, that, that's, it's not perfect, but it goes a long way.
Scott Sorrell (34:00):
You had mentioned field trials a little while ago, Stephen, and, and you know, over the years of my career, I've had the opportunity to, to try to run some field trials, right? To do our best to, to, to, to do controls and treatments, and probably not as many as 10, and yet none of them turned out well. There was always something that confounded the results. I remember one where we were actually bleeding animals and we were going back, I forget what the frequency was, but we, we, we, we go back for the third round and we can't find the cows
Scott Sorrell (34:58):
'Cause No, you're not gonna be there all the time. I, I just even wonder about the results. So I've kind of told our folks, guys, we just, we can't do feed field trials. But anyway, I, I've kind of rambled enough. What what's your thoughts on that? And, and I'm gonna add this too, right? Every time you go to a dairy and, and, and they're gonna wanna try it on their dairy and see results it's just natural. You're gonna wanna do that. And yet, I'm not sure, I'm not sure they're gonna be valid results. But anyway, again, I've rambled enough.
Dr. Stephen LeBlanc (35:29):
I, I think your experience is, is absolutely typical. I mean my research group and Daryl's as well, and a number of others, we do do field studies and yes, it's, and a bunch of other research groups as well on commercial dairies at scale. Sometimes pen level studies, sometimes individual animal level studies, depending on the treatments and the logistics and so on. You are absolutely right that it is horribly difficult, but not impossible. The way we get it done, typically, we and others get it done are with teams of dedicated, very hardworking graduate students who are able and willing to dedicate months or more of their life to literally being on the dairy you know, weekly, many times a week, daily to, to make sure that things are getting done the way they're supposed to be getting done, samples are getting taken, et cetera, et cetera.
Dr. Stephen LeBlanc (36:31):
So can it be done? Yes, absolutely it can, it's difficult. Which means it's time consuming, which means it can be really costly. But it can be done. And, and sometimes those are really valuable, valuable, hopefully also valid. And, and that helps with that external validity or that applicability because it is done under real world conditions and, and typically at, at scale. So yeah, tough to do. And, and you know, you mentioned dairies often want to, you know, especially large dairies where they have the potential to, to do those kinds of things, say, well, we'll try it for ourselves. And, and that can be a, a very useful thing to, to do. It, it's a little, the challenge is, you know, if we're gonna, while we've got multiple pens, we'll, we'll feed two pens and we won't feed the other two pens with the supplement or whatever.
Dr. Stephen LeBlanc (37:29):
Okay. But you, you know, if, if those two pens aren't equal to start out with that, you know, balance in terms of days in milk and lactation numbers and who's in them and, and so on and so forth, and then right away you're kind of shooting yourself in the foot out of the gate. And, and the other one that's easy, a lot easier, not to say easy, but easier to do on a commercial dairy is just like before and after, right? We'll, we'll go along, we'll put the supplement in and we'll see if we get our two or five pounds of milk, which is absolutely the question they want to answer. So, mm-hmm,
Dr. Daryl Nydam (38:32):
Particularly when we're looking for, what I'll say is what's left or the next step in modern sophisticated dairy production, the low hanging fruit, things where there were really big differences where you could learn from before and after, or sloppy trials. We've kind of figured out a lot of that stuff. Now we're looking at smaller differences, which take a little bit more rigor to sort out.
Scott Sorrell (38:59):
You know, as a real world example there was a recent meta-analysis published on ruminant choline and it showed a five pound increase in, the remarkable thing about that research is that it was consistent across all studies. Every study showed a five pound increase. And so, you know, then you, then, then for dairyman on the farm that wanna measure on the farm, you know, even with a product like reassure that gets a five pound response virtually every time, it, it's hard to see that on the dairy, right? Because of all the variations and, and, and all the the measuring equipment. It's just, even though they're getting the result, it's hard to see it and measure it on the farm. And so, I guess my point is right, of course, try it on the farm, do your best to measure it, but you need to learn to incorporate, you know, real peer reviewed research into your decision making process.
Dr. Stephen LeBlanc (39:56):
Yeah, that's a really good point, is particularly on a smaller or medium sized dairy, but, but even on a large dairy yeah, really isolating an effect of, you know, whether it's five pounds of milk or three, or some other number, but that, that is hard to do. And, and so having a a summary of studies that, that, like in a meta-analysis where numerous studies can be validly pooled together can help to identify effects or, or even if there's enough studies you, you can even get as far as to say, well, under these conditions, it, it looks like we get a benefit, but not under those conditions, hypothetically, not saying for that particular product, but those kinds of things. So yeah, that, that's and again, some people are, well, that's just, you know, statistical, you know, black magic.
Dr. Stephen LeBlanc (40:51):
No. again, it all, it all goes back to, you know, were, were the underlying studies done well, and validly, and is the meta-analysis, like, you're not sort of putting your thumb on the scales by which, which ones you include or don't include, those kinds of things. Yeah, that, that's absolutely in fact you know, one of the things that Darryl showed in the webinar is this evidence pyramid. And meta-analyses are right up near the top because if, if you've got an assembly, a collection of well done, randomized controlled trials, or well done observational studies a meta-analysis can be a really powerful tool to confirm or, or even help identify some insights that you may not have been able to see in any one study by itself.
Dr. Daryl Nydam (41:42):
And there's a fair bit of skepticism and even slides into cynicism about industry funded studies. And I go to some length to teach students, producers, other veterinarians that people like Stephen LeBlanc, if they accept money from an industry partner, Stephen is going to honestly do that study. And in fact, the university, there's boilerplate in the contracts that say that Steven is going to get that data and can do with it what he chooses after some short waiting period often. So industry funded studies are not immediately biased in, in the common use of the word, not necessarily epidemiologic use of the word biasd. In, in fact, lots of these industry funded studies are gonna answer questions which are really important and tangible to decision makers out there. For example, there's, if we wanted to know to use product A or product B or product A versus nothing, the government, the USDA, in our case, is rarely gonna fund that.
Dr. Daryl Nydam (42:52):
However, if you go to a room of veterinarians, nutrition professionals or farmers, and you say, do you wanna use Product A? They're all gonna stick their hand up, right? So who's gonna fund that? Industry's gonna fund that. And if this study is well done by a reputable researcher, gone through this peer review process we talked about with appropriate standard operating procedures, appropriate statistics and things, I don't discount those studies. In fact, I, I welcomly, participate in them. And as a consumer of information, taking that back out to the field, I also look, look to them a lot, you know, as a, you know, I joke around public, you know, servant at a, at a land grant university, alls that we have is, is our reputation and honesty to live on, right? And if, and if we do, you know, one false study, it, it, it's probably the end of career.
Scott Sorrell (43:46):
You know, it kind of reminds me we do quite a bit of research with universities. In fact, I think we had 11 abstracts at this year's ADSA, but it, it's it's a little off topic, but it's getting harder and harder to find the universities have got enough cows that can give you that and kind of going back to the p value discussion, but it, it's getting difficult. The, our, our options are fewer these days. Just kind of curious your thoughts on that and, and where that may be headed someday, because I don't see dairies getting larger and, and universities putting in herds. I see them removing herds.
Dr. Stephen LeBlanc (44:25):
Yeah, no, that, that, that's, that is a challenge. I mean, we're, we're here at University of Guelph, we're fortunate to have a, an excellent research dairy. It's not, it's not huge. We milk about 200 cows or so, but we've got fabulous infrastructure. You know, Cornell has a, a much bigger herd. But, but you're right. I mean you know, we can be encouraged. Like, for example, Michigan State right now is building quite a large shiny new research dairy. But you know, South Dakota state just just announced that they were gonna close theirs down and, and others others as well. So it, it will make those that are still available all the more valuable. And, and again, I think it's, it fills part of what we need. So, you know, typically that's where we can have individual cow feed intakes, or we could do, you know, much more in depth physiology or in depth, you know, sampling or monitoring of cows than would be realistic or even possible on a commercial dairy.
Dr. Stephen LeBlanc (45:26):
The other piece though, that goes with is that, and, and I've done this, and Daryl and lots of other people as well more and more research is getting done on commercial dairy farms. And we've talked about the fact that there's, you know, real challenges that come with that. But if we can muster the resources to meet those challenges, those can be really, really valuable studies that they're not, they're not necessarily a replacement for university research herds because they're, you know, you can do different things there. They, they're complimentary, but I, I, you know, I guess I would say the glass is sort of half full there. As we get into larger and more sophisticated dairies, the opportunities, or at least the possibilities for doing really good research on commercial farms is getting better that that's not a replacement or a substitute, but, but it's certainly is a, a good way potentially to get good research done,
Scott Sorrell (46:27):
Right? Yeah. The, the technology's allowed us to get a lot more data. I think we did a podcast recently, I'm forgetting who it was, but they were actually taking individual cow weights daily because they had that capability with, with the robots, right? And we can get you know, milk weights daily by cow and components, those kinds of things. So yeah, it is getting easier. Yeah, that's a good point. Yeah. Daryl, do you have anything to say on that point?
Dr. Daryl Nydam (46:52):
It, it's important to note that we can't let the, the deans and provosts and, you know, philanthropists off the hook. We need support for university dairies still. As Steven said, those that are left become ever more important. However, you know, tangibly to the point, I think among the commercial dairies that today I've got field trials going on, probably representing 10,000 cows today, right? With this, you know, host, I got, you know, postdoc and PhD student out there with a group of technicians out, making sure that we are collecting good data. So that, that, that ability, and that gets us a little higher up, that pyramid of evidence. In fact,
Scott Sorrell (47:34):
Gentlemen, this has been extremely interesting. And we're, we, we're almost at the hour mark which is usually where they, they flicker the lights and call last call. But I, I'm curious, I don't wanna I don't wanna leave if there's still some, some key areas that we've left to uncover.
Dr. Stephen LeBlanc (47:51):
I'll maybe just put in one more shameless plug for the, the webinar. 'cause I think we've, we've talked about some nice complimentary topics here. You know, one thing I'd like to sort of connect to that, that we talk more about in the webinar, and so we don't need to do it again here, but is that as we look about the look at these different kinds of studies and different types of evidence it, it's not one type of study is good and the others are not so good
Dr. Stephen LeBlanc (48:42):
And so we don't want to just sort of hold ourselves up to, well, that that's the gold standard, and if we, you know, if we can't get there, then, you know, other things are very poor cousins and don't, don't, don't get to where we want to go. And, and that was one of the themes was, you know, what's your question? And, and at least I, I kept sort of banging on about, you know, it, it's interesting to talk about how does it work, what's the mechanism, et cetera, et cetera. But at the end of the day for, for dairy owners, managers, industry professionals and so on, it's, you know, is is it useful? Is, is is this something that makes sense for, for me in my dairy? Does this bit of science, whether it's experimental or observational or whatever is it, it, it, does this help me make decisions that I need to make on, on my farm?
Dr. Stephen LeBlanc (49:31):
And, and, and I leaned on one of my favorite analogies, which is the, you know, the blind men around the elephant, and, you know, so each one is, is perceiving a different part. None of them's wrong, but none of them has the full picture. And I, I just think that's a, a, a really compelling analogy for what we do as, as science, whether it's observational studies, whether it's small scale experiments, and in a very controlled, sort of white coat kind of environment, or whether it's that, you know, 10,000 cows worth of field data. There's a place for all of those things as long as they're well done in their own right.
Scott Sorrell (50:10):
Yeah. Yeah. Boy, that's a, that's a great wrap up, Stephen. And with that, I think we will flicker the lights and call last call. And, and one of the things we like to do is, is go around the horn here and just kind of provide kind of a key takeaway that you think that the audience should get from the conversation this evening.
Speaker 4 (50:38):
New research is changing everything we thought we knew about Coline's impact on the cow and her calf and top scientists have a lot to say about it. They're presenting new research that supports choline as a required nutrient to optimize milk production choline as a required nutrient to support a healthy transition choline as a required nutrient to improve calf health and growth, and choline as a required nutrient to increase colostrum quantity. This new research is solidifying Cholines role as a required nutrient for essentially every cow, regardless of health status, milk production level, or body condition score. Learn more about the science that is changing the game and the choline source that is making it happen. Reassure precision release choline from Balchem, visit balchem.com/scientistssay to learn more.
Scott Sorrell (51:39):
Dr. Nydam any thoughts
Dr. Daryl Nydam (51:43):
Related to all of the above? I think, you know, as we move forward in dairy science and as society, I think it's important to remember to keep some faith in science and, and have open discourse about it, acknowledging the biases, and in this case, I mean both social and epidemiologic biases, and continue to discuss things through, through this lens, because, you know, it, it's unfortunate. I think some people are moving away from science Mm-hmm,
Scott Sorrell (52:28):
Yeah, yeah. Great answer. Dr. LeBlanc, can you, can you top that one or your previous answer?
Dr. Stephen LeBlanc (52:35):
Scott Sorrell (53:46):
Yeah. Excellent. Gentlemen, this has been a fun one. I've enjoyed it. A lot of great information. You guys have been great guests. It's been a great conversation and enjoy you guys sharing your insights and and expertise with us. To our loyal listeners as always we really appreciate you. Thank you for coming along and joining us here at the Real Science Exchange where we hope you learned something. We hope you had some fun, and we hope to see you next time here at the Real Science Exchange, where it's always happy hour and you're always among friends.
Speaker 4 (54:20):
We'd love to hear your comments or ideas for topics and guests. So please reach out via email@anh.marketing@chem.com with any suggestions, and we'll work hard to add them to the schedule. Don't forget to leave a five star rating on your way out. You can request your Real Science Exchange t-shirt in just a few easy steps, just like or subscribe to the Real Science Exchange. And send us a screenshot along with your address and t-shirt size to anh.marketing@balchem.com. Balchems real science lecture series of webinars takes place on the first Tuesday of every month with the top research and nutrition topics that will impact your business. We also include small ruminant, monogastric, and companion animal focused topics throughout the year. Visit balchem.com/realscience to see the upcoming topics and to register for future webinars. You can also access past webinars and search for the topics most important to you.