Background

Coral reefs were, and still are, rapidly degrading under the pressure of increasing human activities, and at a faster pace than actions to prevent further the degradation. Comprehensive and fully comparable information about the state of coral reefs was, and still is, urgently needed to provide timely advice to management authorities. Under these premises, rapid and standardised survey technologies at a global scale are needed to provide such information and complement current regional efforts. The XL Catlin Seaview Survey combined existing technologies to merge efforts for a high quality and standardised approach.

Approach

The XL Catlin Seaview Survey approach was centred on high-definition imagery, to document and study coral reefs worldwide. Based on the concept that 'a picture is worth a thousand words,' the scientific survey looked at extracting valuable information from coral reef imagery while archiving an unbiased proof of the condition of surveyed reefs. This approach involved rapid acquisition of high-resolution imagery over large extensions of reefs. Using computer vision and machine learning technologies, the imagery was rapidly analysed for coral reef structure across multiple spatial scales, from metres to kilometres.

Technology

The XL Catlin Seaview SVII camera system was specially built for this project. It was designed to be a highly robust underwater vehicle capable of taking 360° high-resolution images in almost all-weather conditions whilst on lengthy remote expeditions. A timer triggers the camera allowing it to take pictures at three-second intervals, which is controlled by an Android tablet, encased in an underwater housing. The head of the camera system is designed to remain sealed whilst on expedition to avoid flooding issues – data and power are transferred through a wet link docking station.


Collection of Shallow Reef Survey Imagery

Lead scientist – Dr Manuel Gonzalez-Rivero

SVII

Using diver-propelled vehicles and customised HD cameras (SVII), divers surveyed unprecedented areas of coral reef over multiple reef sites. A depth sensor logger carried by the diver monitored the survey depth every second. Surveys were set to a standard depth of 10 metres (+/- 2 metres) following the contour of the reef at this depth. By standardising the survey depth, we avoided the inclusion of confounding factors (i.e., light, wave energy) when comparing the structure of coral reef communities amongst multiple spatial scales.

During the survey, the SVII camera captured three high-definition images every three seconds during the fifty-minute dive. These images allowed us to continuously record the full 360° environment along a 2-kilometres stretch of reef.

The transect location was selected by looking at capturing a vast variety of coral reef configurations or habitats to best represent the fore-reefs of a region.

read more...

This variability was captured by looking at multiple drivers of coral reef structure, such as environmental regimes (i.e., hurricane incidence, temperature regimes, ocean colour, etc.), local information, geomorphology (atoll, fringing reef, barrier reefs) as well as anthropogenic influence (i.e., coastal development, fishing pressure).

Every image captured for the AXA XL Global Reef Record has a geographical tag that allows us to accurately describe a specific section of reef that can be revisited in time and compared against previously available information. To achieve this, a GPS unit was synchronised in time to the SVII cameras and tethered from the surface by the divers.

In order to accurately and consistently locate the benthic community structure, image scale was required to standardise the area of the reef to be analysed. A customised altitude logger was fitted to the SVII camera, which also synchronised to the time of the SVII cameras. This altimeter, connected to a doppler transducer, logged the altitude of the camera with respect to the seabed and depth. Using this information and the optical properties of the lens of the SVII camera, the footprint area of each downward facing camera was estimated.

Thus, the Global Reef Record provides two main image data-products. The first product is qualitative 360° panoramas of the reef. The panoramas are created by scaling and stitching the fish-eye imagery and provide an interactive way to explore the complete environment of a reef. The second product is a quantitative analysis of benthic categories (% covers) along the reef transects. To generate these covers, from each downward facing image we obtained quadrats with features required for image analysis (colour corrected and cropped quadrats of ~1x 1 m). The image quadrats were re-scaled to fixed pixel / cm ratio and then automatically annotated as detailed next.

Estimating covers of benthic categories

Lead scientists – Dr Oscar Beijbom and Dr Manuel Gonzalez-Rivero

Every field campaign provided about 30-40 thousand survey images which need to be annotated in order to extract benthic cover estimates (e.g., % cover of coral, algae, & sand). Unfortunately, manual annotation of a human expert requires at least 10 minutes per quadrat, which creates a huge bottleneck between collected images and the required data-product (i.e., 30,000 images would take 3 years to annotate). To address this, we developed state-of-the art automated image annotation methods based on deep neural networks.

A random subset of the obtained quadrats for each region was first manually annotated using the random point annotation tool of CoralNet (coralnet.ucsd.edu). This data was then used to train deep neural networks that map image patches to key benthic categories. Label sets of benthic categories have been established based on their functional relevance to coral reef ecosystems and their ability to be reliably identified from images by human annotators. The labels were grouped into main benthic functional components depending on the coral region.
Names and descriptions of labels can be seen here.

Benthic categories employed for the classification of coral reefs quadrats in the Great Barrier Reef (GBR) and Coral Sea Commonwealth Marine Reserve (CSCMR), Australia.

Functional Group Label Description
Algae Crustose coralline algae Crustose coralline algae
Algae Macroalgae Upright macroalgae > 1 cm in height (all genus and species) including cyanobacteria films
Algae Turf algae Multi-specific algal assemblage of 1 cm or less in height
Coral Acroporidae branching Family Acroporidae, branching morphology (excluding hispidose type branching).
Coral Acroporidae hispidose Family Acroporidae, hispidose morphology
Coral Other Acroporidae Other corals from the family Acroporidae (e.g., Isopora)
Coral Acroporidae plate or encrusting Family Acroporidae, plate and encrusting morphologies
Coral Acroporidae table, corymbose or digitate Family Acroporidae, table, corymbose and digitate morphologies
Coral Favidae or Mussidae, massive or meandroid Families Favidae and Mussidae, massive and meandroid morphologies
Coral Other hard coral Other hard coral including all other groups not represented by the other coral categories of this label set
Coral Pocilloporidae Family Pocilloporidae
Coral Poritidae branching Family Poritidae, branching morphology
Coral Poritidae encrusting Family Poritidae, encrusting morphology
Coral Poritidae massive Family Poritidae, massive morphology
Other Sand Unconsolidated reef sediment
Other Unclear substrate Unclear, cannot make any ID
Other Water Water
Other Invertebrates Other sessile invertebrates Other sessile invertebrates, zoanthids, anemones, clams, tunicates , etc.
Soft Coral Alcyoniidae Soft coral, family Alcyoniidae, genera Lobophytum and Sarcophyton
Soft Coral Sea fans and plumes Sea fans and plumes
Soft Coral Other soft coral Other soft-corals, outside of the common Alcyoniidae and sea fans plumes

Benthic categories employed for the classification of coral reefs quadrats in the Caribbean

Functional Group Label Description
Algae Macroalgae Upright macroalgae > 1 cm in height (all genus and species) including cyanobacteria films
Algae Turf and crustose coralline algae Multi-specific assemblages of filamentous algae and CCA smothering reef surface 1 cm or less in height
Coral Acropora cervicornis Acropora cervicornis
Coral Acropora palmata Acropora palmata
Coral Agaricia,Undaria,Leptoserisspp. Agariciidae, plates and encrusting corals of the genus Agaricia, Undaria and Helioseris
Coral Colpophyllia natans Colpophyllia natans
Coral Coral meandroid Hard coral meandroid including Dendrogyra, Isophyllia, Manicina, Mycetophyllia, Musa, Scolymia
Coral Coral submassive Hard coral sub-massive, including Dichocoenia, Favia, Solenastrea, Stephanocoenia
Coral Diploria labyrinthiformis Diploria labyrinthiformis
Coral Eusmilia fastigiata Eusmilia fastigiata
Coral Madracis sp. Madracis sp.
Coral Meandrina sp. Meandrina sp.
Coral Montastraea cavernosa Montastraea cavernosa
Coral Orcbicella complex Orbicella complex, O. Annularis, O. faveolata, O. franksi
Coral Porites astreoides Porites astreoides
Coral Porites branching P. porites,P. divaricata,P. furcata
Coral Pseudodiploria sp. P. strigosa, P. clivosa
Coral Siderastrea siderea Siderastrea siderea
Other Other, invertebrates, fish Other sessile and mobile invertebrates, fish
Other Sand Unconsolidated reef sediment
Other Terrigenous sediments Terrigenous sediments
Other Unclear Unclear, cannot make any ID
Other Seagrass Seagrass
Soft Coral Other soft corals Other soft corals including erect and encrusting forms
Soft Coral Sea fans Sea fans ,Gorgonia sp.
Soft Coral Sea plumes Sea plumes and whips
Sponge Aplysina fistularis Aplysina fistularis
Sponge Callyspongia vaginalis Callyspongia vaginalis
Sponge Cliona viridis complex Cliona viridis complex
Sponge Encrusting sponges < 5 cm height (for whole individual); e.g. S. ruetzlen, Haliclona sp.,Monanchora sp., Chondrilla sp., Clathria sp., C. varians
Sponge Erect sponges >1 cm height; height >>basal area; projections include: upright narrow tubes, branches, arborescent, e.g.A. compressa
Sponge Ircinia sp. e.g. I. felix, I. strobilina
Sponge Massive sponges Large basal area to body size and height > 5 cm; compact shape; irregular shapes include: chain-tubes, spherical, castle; e.g.E. ferox,Verongulasp.
Sponge Other sponges Other sponges (undfined to species/morphology): e.g. A. wiedenmayeri,Niphates sp.
Sponge Rope sponges Height >> basal area; Spread along the substrate, e.g.S. aura
Sponge Tube sponges Grouped tube species: A. archeri, A. tubulata, C. plicifera, N. digitalis, unknown vase and small barrel group (tube openining ~ height; basal area < opening)
Sponge Xestospongia muta Xestospongia muta

Benthic categories employed for the classification of coral reefs quadrats in the Maldives and Chagos Archipelago

Functional Group Label Description
Algae CCA Crustose coralline algae
Algae CYANO Cyanobacteria
Algae EAM Multi-specific algal assemblage smothering reef surface 1 cm or less in height (Epilithic algal matrix)
Algae MALG Upright macroalgae > 1 cm in height (all genera and species)
Coral ACR-BRA Family Acroporidae; branching morphology (including hispidose type branching)
Coral ACR-OTH Other corals from the family Acroporidae (e.g. Isopora)
Coral ACR-TCD Family Acroporidae; table and corymbose morphologies (no digitate)
Coral BRA_DIG_Ac Acropora with branches resembling fingers (digitate type); e.g. A. humulis
Coral BRA_OTH Other branching genera including: Seriatopora; Anacropora; Echinopora; Montipora; Tubastrea (excluding Pocillopora and Stylophora)
Coral BRA_RND_St Branching Stylophora
Coral FOLI Thin Foliose and Plate colonies (excluding genera Acropora Porites) with/without visible relief structure: e.g. Echinophyllia; Turbinaria ; Montipora; Echinopora
Coral LOBO Lobophyllia sp.
Coral MASE_LRG_O Massive Submassive Encrusting colonies with large visible rounded polyps: Diploastrea; Favia; Favites; Montastraea; Galaxea; Goniopora; Astreopora
Coral MASE_MEA_O Massive Submassive Encrusting colonies with meandering ridges and valleys resembling brain: Platygyra; Leptoria; Goniastrea
Coral MASE_SML_O Massive Submassive Encrusting colonies with small or invisible polyps (including columnar forms): Pavona; Psammocorra; Coscinaraea ; Gardineroseris
Coral OTH-HC Other hard coral including all other groups not represented by the other coral categories in Maldives and Chagos (e.g. Fungia; bleached colonies)
Coral POCI Family Pocilloporidae; all genera (excluding Stylophora) and species
Coral POR-BRA Family Poritidae; branching morphology e.g. P. cylindrica
Coral POR-ENC Family Poritidae; encrusting morphology e.g. P. lichen
Coral POR-MASS Family Poritidae; massive/sub-massive morphology e.g. P. lobata; P. lutea
Other OTHER Other including fish; trash; transect hardware; unknown
Other Sand Sand. Unconsolidated reef sediment
Other Seagrass
Other Invertebrates Cliona Encrusting Cliona spp.
Other Invertebrates COTS Crown of Thorns Sea Star
Other Invertebrates Mille Millepora sp.
Other Invertebrates MINV_OTH Other mobile invertebrates: i.e Echinoderms; lobster
Other Invertebrates OTH-SINV Other sesile invertebrates: bryozoa; clams; tunicates (ascidians); soft hexacorrallia; hydroids
Other Invertebrates SpMass Massive or encrusting sponges
Other Invertebrates SpOTH Other sponges (outside massive/encrusting morphologies and Cliona)
Soft Coral ALC-SF Soft coral family Alcyoniidae; genera Lobophytum and Sarcophyton
Soft Coral OTH-SF Other soft-corals outside of the common Alcyoniidae and erect forms (sea fans plumes whips): e.g. Xeniidae; Nepthtydae; Tubipora; Briareum; Heliopora
Soft Coral SINV_SFC_E Sea fans; whips; plumes

Benthic categories employed for the classification of coral reefs quadrats in the Southeast Asia region and Solomon Islands

Functional Group Label Description
Algae CCA Crustose coralline algae
Algae CYANO Cyanobacteria
Algae EAM Multi-specific algal assemblage smothering reef surface 1 cm or less in height (Epilithic algal matrix)
Algae MALG Upright macroalgae > 1 cm in height (all genera and species)
Coral ACR-BRA Family Acroporidae; branching morphology (including hispidose type branching)
Coral ACR-OTH Other corals from the family Acroporidae (e.g. Isopora)
Coral ACR-TCD Family Acroporidae; table; corymbose and digitate morphologies
Coral BRA_OTH Other branching genera including: Seriatopora; Anacropora; Echinopora; Montipora; Tubastrea (excluding Pocillopora and Stylophora)
Coral BRA_RND_St Branching Stylophora
Coral FOLI Thin Foliose and Plate colonies (excluding genera Acropora Porites) with/without visible relief structure: e.g. Echinophyllia; Turbinaria ; Montipora; Echinopora
Coral LOBO Lobophyllia sp.
Coral MASE_LRG_O Massive Submassive Encrusting colonies with large visible rounded polyps: Diploastrea; Favia; Favites; Montastraea; Galaxea; Goniopora; Astreopora
Coral MASE_MEA_O Massive Submassive Encrusting colonies with meandering ridges and valleys resembling brain: Platygyra; Leptoria; Goniastrea
Coral MASE_SML_O Massive Submassive Encrusting colonies with small or invisible polyps (including columnar forms): Pavona; Psammocorra; Coscinaraea ; Gardineroseris
Coral OTH-HC Other hard coral including all other groups not represented by the other coral categories in Southeast Asia (e.g. Fungia; bleached colonies)
Coral POCI Family Pocilloporidae; all genera (excluding Stylophora) and species
Coral POR-BRA Family Poritidae; branching morphology e.g. P. cylindrica
Coral POR-ENC Family Poritidae; encrusting morphology e.g. P. lichen
Coral POR-MASS Family Poritidae; massive/sub-massive morphology e.g. P. lobata; P. lutea
Other OTHER Other including fish; trash; transect hardware; unknown
Other Sand Sand. Unconsolidated reef sediment
Other Sediment Terrigenous sediments
Other WATER Water
Other Invertebrates COTS Crown of Thorns Sea Star
Other Invertebrates Mille Millepora sp.
Other Invertebrates MINV_OTH Other mobile invertebrates: i.e Echinoderms; lobster
Other Invertebrates OTH-SINV Other sesile invertebrates: bryozoa; clams; tunicates (ascidians); soft hexacorrallia; hydroids
Other Invertebrates SpMass Massive or encrusting sponges
Other Invertebrates SpOTH Other sponges; undefined to species/morphology (outside massive/encrusting/rope/tube morphologies)
Other Invertebrates SpRope Height >> basal area; Spread along the substrate
Other Invertebrates SpTube Tube species: tube opening ~ height; basal area < opening
Soft Coral ALC-SF Soft coral family Alcyoniidae; genera Lobophytum and Sarcophyton
Soft Coral OTH-SF Other soft-corals outside of the common Alcyoniidae and erect forms (sea fans plumes whips): e.g. Xeniidae; Nepthtydae; Tubipora; Briareum; Heliopora
Soft Coral SINV_SFC_E Sea fans; whips; plumes

To validate our method, we generated automated cover estimates on several randomly selected reef segments (roughly 40 consecutive survey images) and compared against estimates by human experts. After verifying that the estimated covers were highly correlated with those by human experts, we deployed the network on the whole set of collected imagery. Automated image annotation using this method was very fast; on average it only required 0.2 seconds, three orders of magnitude faster than manual analysis. Analysis of hundreds of thousands of images from a reef region could therefore be achieved in a manner of days. For example; over 190,000 survey quadrats from the GBR were automatically annotated in one week using a single computational unit (a NVIDIA Titan X GPU).

Further methodological details can be found at:

González-Rivero M., O. Beijbom, A. Rodriguez-Ramirez, T. Holtrop, Y. González-Marrero, A. Ganase, Chris Roelfsema, S. Phinn and O. Hoegh-Guldberg. 2016. Scaling up Ecological Measurements of Coral Reefs Using Semi-Automated Field Image Collection and Analysis. Remote Sens. 8, 1:30; doi:10.3390/rs8010030.

González-Rivero M., P. Bongaerts, O. Beijbom, O. Pizarro, A. Friedman, A. Rodriguez-Ramirez, B. Upcroft, D. Laffoley, D. Kline, R. Vevers, and O. Hoegh-Guldberg. 2014. The Catlin Seaview Survey – kilometre-scale seascape assessment, and monitoring of coral reef ecosystems. Aquatic Conservation: Mar. Freshw. Ecosyst. 24(Supp. 2): 184-198

Beijbom O, Edmunds PJ, Roelfsema C, Smith J, Kline DI, Neal BP, et al. 2015. Towards Automated Annotation of Benthic Survey Images: Variability of Human Experts and Operational Modes of Automation. PLoS ONE 10(7): e0130312. doi:10.1371/journal.pone.0130312

Beijbom, O, P.J.Edmunds, D.I.Kline, B.G.Mitchell, D.Kriegman. 2012. Automated Annotation of Coral Reef Survey Images. IEEE Conference on Computer Vision (CVPR), Providence, Rhode Island, June 2012