Getting Started With Some Data Crunching
January 10th, 2019
Hi everyone! The first increment of my agricultural internship is off to a running start in the NMSP where I am learning more than I had ever imagined I would about data processing. Right off the bat, I would be the first person to shy away from A: simply sitting at a desk all day and B: the idea of crunching numbers through a computer. However, in this case, what the numbers have the potential to tell you is so incredibly interesting (but maybe that’s my personal bias towards any and all things agriculturally related talking)!
So, to break it down, many farms are equipped with harvesting machinery that has a wide variety of sensors to help keep track of and measure yield. GPS guidance and positioning, speedometers, crop moisture sensors, width of the section being harvested, flow or the thickness/amount of the crop being pushed through the feed rolls, and many other bells and whistles that give off readings.
Many variables go into just simply measuring yield. Take this idea for instance: a bushel of corn has been standardized to a weight of 56lbs, however as a unit of measurement a bushel is actually a volume of something in its dry capacity (8 gallons of that ‘something’ in imperial units). But corn can be dried to different moisture contents, and some corn kernels can be on average denser than others even at the same moisture capacity! Now, add in a giant machine rolling through a field of corn, speeding up and slowing down, flow delay (A.K.A. the time it takes for the crop to be cut and pass through the chopper/combine head to get the sensor and be measured, but now the chopper is 20 feet further down the pass!) missing the perfect swath, a wet vs dry spot, or any other number of crazy things the machine operators do, and you have a huge number of variables that can affect how much crop is actually coming off the ground as opposed to what yield the sensor is reading off the field in a given spot. Now you have a mess of numbers that are all jumbled around when all the poor farmer wants to know is a round-a-bout idea of how his land is producing, and maybe which spots on the fields are the good spots or the bad spots.
Using two kinds of software, the NMSP has been working on fine-tuning a way of “cleaning” all this data that is recorded by the machine sensors and turning the numbers into a usable data set that farmers can work with.
USDA ARS – Yield Editor
SMS by AgLeader
…And the ins-and-outs of those whole concepts was about as much as I could learn in one week! You’ve got the “why,” coming soon—the “how” data is moved around and cleaned.