Pioneering a big data revolution in agriculture

The campus of CIAT’s headquarters can be a noisy place. First-time visitors are often struck by the symphony of tropical birds, the half-dozen or so languages being spoken, the passing of large farm equipment, and the sound of expresso being made at the coffee bar.

But in the past 3 years, a new noise has joined the orchestra at CIAT, changing the auditory landscape and the research. Crossing the threshold of the Decision and Policy Analysis (DAPA) building, the hum of dozens of small fans becomes audible. The fans are cooling an array of processing servers and data banks, which form the backbone of CIAT’s big data operations.

Around the world, big data science is allowing for the discovery of solutions and answers to some of the world’s most complicated problems that previously were hidden in immense datasets. From more accurately forecasting the weather, to predicting disease outbreaks, to making political forecasts, big data is changing the way science is done around the world. The machines humming along in DAPA are the muscle behind CIAT’s very own big data research, and are providing the horsepower that has led to exciting discoveries in agriculture science.

Scientists here at CIAT are applying big data tools to pinpoint strategies that improve the lives of farmers in developing countries who are facing the effects of a changing climate. Traditionally, agronomists conduct small field experiments under delicately controlled conditions, but today CIAT scientists are trying the opposite. Using large, uncontrolled, real-world data sets, scientists at CIAT are applying cutting-edge analytics – in many cases borrowed from fields like biology and neuroscience – to scour the data and produce nuanced, reliable recommendations far more quickly than was previously possible.

These solutions are coming at a critical time. In recent years, rice production in Colombia has inexplicably fallen, even as various international groups have labelled the region “the next global breadbasket”. The reason for the decline is not known, but subtle shifts in rainfall as well as extreme weather due to climate change is believed to be the main culprit.

In the short time that CIAT’s big data operation has been in existence, it has already yielded game-changing discoveries for the Colombian rice industry – solutions that can easily be scaled up, and broadened to include other crops. Using country-wide records of historical climatic conditions, yields and farming practices provided by Colombia’s National Federation of Rice Growers (FEDEARROZ), CIAT scientists developed highly site-specific recommendations to boost rice production.

In one town they found that yields were blunted by solar radiation, while in another town – about 30 kilometers away – they found that it was warm nighttime temperatures that were holding yields back. The extreme localization of the recommendations is providing results not possible through traditional research, and could prove critical as the climate changes both globally and locally.

But it is not just rice production that is being revolutionized by big data science at CIAT. The team has so far applied the methods to 11 crops in total. Through their work they have joined forces with four partners, and their results are being implemented in five countries around the world.

Looking ahead, the scientists are planning to incorporate data on soils, pests, diseases costs and other factors to boost explanatory power, and they are working to develop new ways of capturing and analyzing data that could further strengthen their methods. Their work has caught the attention of partners like the World Bank, which is working with CIAT scientists to scale up successes in Latin America to Africa.

All of it means that in the future, recommendations for crop production will be made more quickly, more accurately, and with greater attention to the unique details that define individual regions. Taken together, big data has the potential to revolutionize crop production, reversing yield, and empowering smallholder farmers going into a future fraught with climatic uncertainty.