The Alaska Fisheries Science Center is using drones, remotely operated vehicles and 3D stereo cameras with Artificial Intelligence to collect information that helps to support sustainable fisheries.
National Fisherman recently published a story about how “Crabbers help scientists gather data about Bristol Bay red king crab”, a project involving the Alaska Fisheries Science Center that allows the scientific community to gather data in Alaska despite the inclement weather and icy seas that make winter research on Bristol Bay red king crab challenging.
Now it’s time to share another story about investigation in Alaska, and other ways scientists are using to collect biological and environmental information managers need to maintain productive, climate-resilient fisheries and coastal communities and healthy marine ecosystems. The Alaska Fisheries Science Center is advancing the use of remote sensing technologies to meet that challenge, and help Alaska’s productive marine ecosystems provide livelihoods and support subsistence communities across the state.
Aware that Alaska seas are vast, remote, and deep and that ships, money, and time are limited, the Alaska Fisheries Science Center is using innovative tools including drones, remotely operated vehicles, eDNA analysis, and stereo camera systems to complement traditional ship surveys. These new tools increase, according to NOAA Fisheries, sampling coverage while minimizing the environmental impact and cost of fisheries monitoring and research.
A unique tool available to scientists is the stereo camera system, developed at the Alaska Fisheries Science Center, which offers additional, unique capabilities. As happens in so many other cases, the development of the stereo camera system appeared as the result of a problem needing a solution: to make sure that fish sampled in survey trawls represented the population.
“Stereo camera imagery reconstructs a 3D environment. The image quality is high enough to identify species. Together these allow us to collect a great deal of information from each sample,” said NOAA fisheries biologist Kresimir Williams, who developed the stereo camera system with colleague Rick Towler.
After having determined that 3D imagery would provide the information they needed, the team developed a first solution, the “Pea Pod”, a set of green floats with cameras in the middle introduced in 2007, which paved the way for the CamTrawl, which Kresimir Williams says, “and our big-time leap into stereo camera sampling.”
Still, it took Towler and Williams six years to perfect the CamTrawl system. Along the development path they decided that Artificial Intelligence was essential to automate image processing, so they partnered with experts at the University of Washington to add AI to the CamTrawl, in order to save an enormous amount of tedious human labor and cost.
NOAA’s Automated Image Analysis Strategic Initiative
CamTrawl has been routinely used in surveys since 2012. The CamTrawl is a sideways facing digital stereo-camera system fitted near the open cod-end of a traditional fisheries survey trawl. The trawl is used as an aggregating device to direct fish in front of the cameras before passing through the end of the trawl. The provided data set shows the fishery target, walleye pollock, as the pass before the cameras.
In response to the need for more accurate, efficient, and timely scientific fisheries stock assessments, NOAA Fisheries is increasingly using camera-based surveys. Cameras can collect millions of high resolution images in a matter of days. These images must be quickly turned into the data needed for stock assessments, such as species, abundance, and size.
The automated analysis can detect fish in images, classify targets into species groups, and estimate fish length. This task is beyond the capability of human analysts.
NOAA’s Automated Image Analysis Strategic Initiative (AIASI) team helped automate analysis of images from cameras mounted inside Midwater Assessment and Conservation Engineering (MACE) program survey trawls (CamTrawl). This reduced two months of work by a human analyst to just 4 days.
The AIASI team also developed Video and Image Analytics for a Marine Environment image analysis software, which has broad potential applications. Due to the importance of the developments, Kresimir Williams of NOAA Fisheries’ Alaska Fisheries Science Center was part of a team that received the 2019 Department of Commerce Gold Medal Award for Scientific and Engineering Achievement.
NOAA’s Automated Image Analysis Strategic Initiative (AIASI) team developed the first open-source toolkit for automated image analysis to harness big data for marine ecosystem science and ocean stewardship.
“Image analysis is hard work. Detecting and identifying fish and other animals in hours of underwater video footage or image sets requires an enormous commitment of expert analyst time,” Williams said in 2019. “The AIASI team set out to change that. We applied the latest machine learning and artificial intelligence algorithms to extract timely and meaningful data from image-based fishery surveys across the nation.”
“With machine learning and artificial intelligence increasingly becoming part of our everyday lives, NOAA has been at the forefront of developing methods of integrating these technologies into our mission of managing the nation’s marine living resources,” Williams added. “I am honored to be able contribute as part of the AIASI team.”
"Kresimir, with MACE program colleagues and other collaborators, has made innovative strides in the use of underwater stereo camera imagery in acoustic-trawl surveys of fish and other species," said Patrick Ressler, MACE Acting Program Manager. "Broad application of underwater stereo cameras and image processing techniques and software have grown out of and continue to multiply the impact of his work."
Since the introduction of the CamTrawl in 2012, Towler and Williams have not only refined the system but also built a second system as a backup, used by other NOAA Fisheries science centers and other agencies across the nation. The team has also helped scientists at the Northwest Fisheries Science Center build a CamTrawl which they use during their West Coast hake surveys. “We can’t make them for everyone, but we made the recipe available so they can make their own,” Williams said.
