Ripe for the Picking

By Robin Ann Smith

On a brisk spring morning in Randolph County, North Carolina, NC State University researcher Jing Zhang strides through a blueberry patch, stops at a bush and pulls out her smartphone.

Blueberry picking season will soon be in high gear in North Carolina, where some 54 million pounds of blueberries are harvested each year — making up nearly 9% of all U.S. production.

It will be another month before the blooms on these bushes transform into plump, juicy berries, but Zhang has been busy developing a way to help farmers better gauge which bushes are the most productive and when they’ll be ready to pick — using computer vision and artificial intelligence.

Zhang snaps a picture of a blueberry bush and uploads it to an app on her phone. Within seconds, an AI system tells her how many berries are on the bush and what percentage of them are ripe.

On this day, the cool temperatures mean few bees are buzzing among the bell-shaped flowers, but some of the first berries have started to swell up.

She glances at her phone’s display. “It sees 112 berries,” says Zhang, a horticultural science professor and N.C. Plant Sciences Initiative faculty affiliate who leads NC State’s Translational Plant Phenomics Lab.

Better Berry Harvests

In using the technology, farmers can better plan their harvests, says Cody Craddock, a Randolph County agricultural Extension agent who has been working with Zhang and growers to beta test the app.

That’s because blueberries don’t ripen all at once. They must be picked several times over the course of the season, and each time requires labor. Blueberries don’t continue ripening after picking, so harvesting too early means they won’t be as sweet, while waiting too long makes for soft or shriveled fruit.

“If farmers can make sure that the bushes are at peak ripeness before sending crews out into the field and estimate how many berries they’ll bring to market, they can maximize their labor,” Craddock says.

“It’s a decision tool,” Zhang adds.

Traditionally, blueberry growers estimate things like ripeness, yield and other traits by walking up and down their fields, eyeballing bushes one by one and then making their best guess on a 1 to 10 scale. Where one person may give a bush an 8, another might see 7 and yet another 9.

“It’s subjective,” Zhang says.

She thought there might be a solution, so a few years ago, she and her team began working on a more precise way to detect differences between bushes.

Cultivating Computer Vision

To build the system, they trained a computer vision model to identify individual berries and distinguish ripe ones from unripe ones by feeding it thousands of labeled images.

To test and refine their models, last summer Craddock and other members of the N.C. PSI Extension Agent Network collected images from 10 commercial blueberry farms across North Carolina using handheld cameras and cellphones.

Afterward, they picked every berry from the bushes, sorting and counting them by hand, and compared their results with the automated counts based on the images.

Zhang uses similar approaches to tackle a range of other farm problems, including Neopestalotiopsis, or neo-p, an emerging disease in strawberries.

While the blueberry app isn’t publicly accessible for anyone to upload their photos yet, she’s working on that. In the meantime, she’s training the model on additional blueberry varieties.

The technology could be useful to breeders, too, she says. That’s because plant breeding is a numbers game.

Let’s say a grower wants blueberries that not only have a higher yield but also ripen earlier for premium pricing. Developing a variety with desired traits often involves an exhaustive search. Thousands of plants must be grown and compared over multiple generations before breeders find ones that are truly exceptional.

Anything that makes it easier to evaluate more plants can go a long way toward helping find those with desired traits, such as disease resistance or the ability to withstand mechanical harvesting.

“The more plants you can look at, the better your chances of finding a winner,” Zhang says.

“The benefit to the tool is that it takes that guesswork out of it,” Craddock adds.