Collaborating with Citizen Scientists

Climbing one of the world’s biggest granite walls is different from climbing trees, as National Park Service botanist Martin Hutten discovered while dangling from a cliff in the spray of Vernal Falls high above the Yosemite Valley. Hutten apprenticed in the logging industry before he started graduate school, so he new how to climb trees. “I could trust myself to a rope,” he recalls, “but I’d definitely never hung off a cliff or collected [samples] from a cliff.”

So when Hutten — who is seeking a Ph.D. in forest ecology at Oregon State University, Corvallis — and his fellow park rangers needed help surveying the park’s lichen, they enlisted experienced rock climbers. Over 2 weeks in 2007 and 2008, those volunteer climbers helped collect 394 lichen specimens. This year, the researchers are asking volunteers to help with other parts of their biodiversity survey, such as counting birds from the valley floor.

Hutten’s project is one of many worldwide in which volunteers help scientists collect data in ways that range from mundane to vital. Such help, although voluntary, is not free. Recruiting volunteers, finding appropriate ways for them to contribute, training them, keeping them motivated, and ensuring the quality of the data they collect requires time, money, and management expertise.

One of the keys is to communicate well and keep the volunteers plugged in to the scientific big picture, says Chris Lintott, an astronomer at the University of Oxford in the United Kingdom and a coordinator of the Galaxy Zoo project which uses volunteers to catalog a database of galaxies. “You really need to treat the volunteers as collaborators,” he says.

What’s in it for you?

Volunteers have long played passive or secondary roles in science, serving as experimental subjects in clinical studies, say, or helping with public outreach. But in some disciplines, nonscientists are increasingly playing roles that scientists themselves usually play: collecting hard-to-get data, for example, or analyzing quantities of data that would be unmanageable without help. There simply aren’t enough scientists or remote sensors to track all the species of interest to ecologists, for instance. “There’s no way we could have collected all that [lichen] data in that amount of time” without volunteer help, says Niki Nicholas, Hutten’s boss at Yosemite.

This article is part of a two-story package on scientists collaborating with the public. In the companion piece, Susan Gaidos writes about working with Inuit, fishermen, and bird watchers to expand our knowledge of climate and ecology.

Another study enabled by volunteer help is the Galaxy Zoo project. By 2007, the robotic Sloan Digital Sky Survey had captured images of more than a million galaxies. But robots and computers are bad at classifying galaxies as spiral or elliptical, and astronomers need this information. The task fell to Oxford graduate student Kevin Schawinski, now a postdoc at Yale University. Schawinski classified tens of thousands of galaxies for his dissertation — but that’s just a tiny fraction of the data his team collected. “We could have made poor Kevin sit down and do the other 850,000 [galaxies], but if you do that you end up with only a set of Kevin’s opinions,” Lintott says — not to mention a stressed-out, graying graduate student.

So Galaxy Zoo enlisted volunteers to render those judgments within an automated system that compares individual evaluations and chooses a classification based on how well individuals have rated previous galaxies. Some people are better at distinguishing spiral galaxies from elliptical galaxies, Lintott says, whereas others are better at detecting merging galaxies: Galaxy Zoo’s volunteers — and their statistical analysis — have improved the accuracy of the taxonomy compared with what scientists could do by themselves. It has also increased throughput: When the Galaxy Zoo project went live, the server hosting the galaxy images caught fire, so great was the volume of data requests from volunteers.

But volunteers can provide more than a pair of eyes. Curious about small green blobs in the background of some galaxy images, one group of Galaxy Zoo volunteers classified them using spectra available publicly in online databases. The blobs turned out to be the most efficient star-forming regions found yet in the universe. By the time the volunteers consulted the experts, “they’d already categorized [the blobs] and collected spectra,” Lintott marvels.

Of course, volunteers do need some help from professional scientists. In the paper they eventually coauthored with the volunteers, the Galaxy Zoo scientists had to provide context and discuss the meaning of the green blobs.

An essential tool: the Internet

In addition to providing scientific guidance, scientists must manage the volunteers and keep them motivated. For that, the Internet has proved a valuable tool.

In the FeederWatch program at Cornell University — part of eBird, a project of Cornell and the National Audubon Society that’s discussed in a companion article — volunteers report birds they observe, via submission forms posted on the Internet. If a volunteer reports species unexpected in their region, a Web-based reporting system flags it. Staff in the Cornell office may follow up, asking for a photo if a report appears particularly unlikely.

The Internet has had a big effect on how scientists handle volunteer contributions. The FeederWatch program has been around for years, collecting reports once a year by old-fashioned mail. The current online interface has many advantages over the old system, including faster analysis times, but it’s not without its challenges. Because many of their volunteers are older and not experienced with the Internet, the FeederWatch interface had to be very carefully designed to ensure usability. Both the Cornell birding project and Galaxy Zoo developed online forums that enable experienced volunteers to answer questions and share expertise with newer ones, reducing the workload for paid staff. Still, David Bonter, an ornithologist at Cornell and director of FeederWatch, must dedicate two full-time staff to handling e-mails from FeederWatch volunteers.

Managing volunteers

Volunteer self-sufficiency is the objective. Bonter and colleagues worked hard to create clear, explicit, simple instructions for FeederWatch volunteers. Unfortunately, “nobody reads them,” Bonter says. They are now experimenting with video tutorials.

Yet scientists must explain not just about how to collect data but also how it’s used if they want to keep their volunteers engaged. “People won’t continue to submit data into a black hole without being provided information about what it’s being used for,” Bonter says.

Carlos Buhler, one of the Yosemite climbing volunteers, appreciates this kind of feedback. “If scientists are trying to get volunteers to help out, making them feel like their work adds to the whole picture [gives them] a tremendous amount of energy,” Buhler says.

Volunteers must be recruited and managed, which requires skills that many scientists struggle with. “We’re a bunch of dumb biologists,” Bonter says. “We know nothing about marketing.” FeederWatch benefits from being a long-term, established program, with a volunteer corps built up over decades. eBird exploits this corps by sharing its members among a number of bird-related research projects: This month, eBird tapped its volunteers for help tracking birds affected by the Gulf of Mexico oil spill.

Like other kinds of networks, networks of volunteers have nodes. One node in the Yosemite network, Hutten says, is Linda McMillan, a volunteer who runs a marketing and communications consultancy. An active member of the American Alpine Club, McMillan recruited most of the Yosemite volunteer climbers and matched them with projects, Hutten says.

Internet communications skills may be crucial for organizing large crowds, but for projects with fewer volunteers who help in person — archeology projects often work this way — old-fashioned management skills are at least as important. Francesc Tuset, director of the excavation at the Roman city of Clunia, in Spain, says, “Your job is a bit like a psychologist’s: If you see somebody is uncomfortable, the work will end badly.” He and Lintott have reached a similar conclusion: One key to good management is matching volunteers to tasks that suit them. Once the matching is done, you have to make sure they know what they’re doing. “I think you really need to spend a lot of time in the early stages developing the protocols,” Bonter says. Finally, if you help the volunteers understand how everything fits together — and make sure they realize they’re making a difference — they’re all likely to be happy with the project. “Everybody likes to feel helpful,” Buhler says.

This feature first appeared in Science Careers [html] [pdf]