Background
Coral reefs were, and still are, rapidly degrading under the pressure of increasing human activities, and at a faster pace than actions to prevent further the degradation. Comprehensive and fully comparable information about the state of coral reefs was, and still is, urgently needed to provide timely advice to management authorities. Under these premises, rapid and standardised survey technologies at a global scale are needed to provide such information and complement current regional efforts. The XL Catlin Seaview Survey combined existing technologies to merge efforts for a high quality and standardised approach.
Approach
The XL Catlin Seaview Survey approach was centred on high-definition imagery, to document and study coral reefs worldwide. Based on the concept that 'a picture is worth a thousand words,' the scientific survey looked at extracting valuable information from coral reef imagery while archiving an unbiased proof of the condition of surveyed reefs. This approach involved rapid acquisition of high-resolution imagery over large extensions of reefs. Using computer vision and machine learning technologies, the imagery was rapidly analysed for coral reef structure across multiple spatial scales, from metres to kilometres.
Technology
The XL Catlin Seaview SVII camera system was specially built for this project. It was designed to be a highly robust underwater vehicle capable of taking 360° high-resolution images in almost all-weather conditions whilst on lengthy remote expeditions. A timer triggers the camera allowing it to take pictures at three-second intervals, which is controlled by an Android tablet, encased in an underwater housing. The head of the camera system is designed to remain sealed whilst on expedition to avoid flooding issues – data and power are transferred through a wet link docking station.
Collection of Shallow Reef Survey Imagery
Lead scientist – Dr Manuel Gonzalez-Rivero
Using diver-propelled vehicles and customised HD cameras (SVII), divers surveyed unprecedented areas of coral reef over multiple reef sites. A depth sensor logger carried by the diver monitored the survey depth every second. Surveys were set to a standard depth of 10 metres (+/- 2 metres) following the contour of the reef at this depth. By standardising the survey depth, we avoided the inclusion of confounding factors (i.e., light, wave energy) when comparing the structure of coral reef communities amongst multiple spatial scales.
During the survey, the SVII camera captured three high-definition images every three seconds during the fifty-minute dive. These images allowed us to continuously record the full 360° environment along a 2-kilometres stretch of reef.
The transect location was selected by looking at capturing a vast variety of coral reef configurations or habitats to best represent the fore-reefs of a region.
This variability was captured by looking at multiple drivers of coral reef structure, such as environmental regimes (i.e., hurricane incidence, temperature regimes, ocean colour, etc.), local information, geomorphology (atoll, fringing reef, barrier reefs) as well as anthropogenic influence (i.e., coastal development, fishing pressure).
Every image captured for the AXA XL Global Reef Record has a geographical tag that allows us to accurately describe a specific section of reef that can be revisited in time and compared against previously available information. To achieve this, a GPS unit was synchronised in time to the SVII cameras and tethered from the surface by the divers.
In order to accurately and consistently locate the benthic community structure, image scale was required to standardise the area of the reef to be analysed. A customised altitude logger was fitted to the SVII camera, which also synchronised to the time of the SVII cameras. This altimeter, connected to a doppler transducer, logged the altitude of the camera with respect to the seabed and depth. Using this information and the optical properties of the lens of the SVII camera, the footprint area of each downward facing camera was estimated.
Thus, the Global Reef Record provides two main image data-products. The first product is qualitative 360° panoramas of the reef. The panoramas are created by scaling and stitching the fish-eye imagery and provide an interactive way to explore the complete environment of a reef. The second product is a quantitative analysis of benthic categories (% covers) along the reef transects. To generate these covers, from each downward facing image we obtained quadrats with features required for image analysis (colour corrected and cropped quadrats of ~1x 1 m). The image quadrats were re-scaled to fixed pixel / cm ratio and then automatically annotated as detailed next.
Estimating covers of benthic categories
Lead scientists – Dr Oscar Beijbom and Dr Manuel Gonzalez-Rivero
Every field campaign provided about 30-40 thousand survey images which need to be annotated in order to extract benthic cover estimates (e.g., % cover of coral, algae, & sand). Unfortunately, manual annotation of a human expert requires at least 10 minutes per quadrat, which creates a huge bottleneck between collected images and the required data-product (i.e., 30,000 images would take 3 years to annotate). To address this, we developed state-of-the art automated image annotation methods based on deep neural networks.
A random subset of the obtained quadrats for each region was first manually
annotated using the random point annotation tool of CoralNet (coralnet.ucsd.edu).
This data was then used to train deep neural networks that map image patches
to key benthic categories. Label sets of benthic categories have been established
based on their functional relevance to coral reef ecosystems and their ability to
be reliably identified from images by human annotators. The labels were grouped
into main benthic functional components depending on the coral region.
Names and
descriptions of labels can be seen here.
Benthic categories employed for the classification of coral reefs quadrats in the Great Barrier Reef (GBR) and Coral Sea Commonwealth Marine Reserve (CSCMR), Australia.
Functional Group | Label | Description |
---|---|---|
Algae | Crustose coralline algae | Crustose coralline algae |
Algae | Macroalgae | Upright macroalgae > 1 cm in height (all genus and species) including cyanobacteria films |
Algae | Turf algae | Multi-specific algal assemblage of 1 cm or less in height |
Coral | Acroporidae branching | Family Acroporidae, branching morphology (excluding hispidose type branching). |
Coral | Acroporidae hispidose | Family Acroporidae, hispidose morphology |
Coral | Other Acroporidae | Other corals from the family Acroporidae (e.g., Isopora) |
Coral | Acroporidae plate or encrusting | Family Acroporidae, plate and encrusting morphologies |
Coral | Acroporidae table, corymbose or digitate | Family Acroporidae, table, corymbose and digitate morphologies |
Coral | Favidae or Mussidae, massive or meandroid | Families Favidae and Mussidae, massive and meandroid morphologies |
Coral | Other hard coral | Other hard coral including all other groups not represented by the other coral categories of this label set |
Coral | Pocilloporidae | Family Pocilloporidae |
Coral | Poritidae branching | Family Poritidae, branching morphology |
Coral | Poritidae encrusting | Family Poritidae, encrusting morphology |
Coral | Poritidae massive | Family Poritidae, massive morphology |
Other | Sand | Unconsolidated reef sediment |
Other | Unclear substrate | Unclear, cannot make any ID |
Other | Water | Water |
Other Invertebrates | Other sessile invertebrates | Other sessile invertebrates, zoanthids, anemones, clams, tunicates , etc. |
Soft Coral | Alcyoniidae | Soft coral, family Alcyoniidae, genera Lobophytum and Sarcophyton |
Soft Coral | Sea fans and plumes | Sea fans and plumes |
Soft Coral | Other soft coral | Other soft-corals, outside of the common Alcyoniidae and sea fans plumes |
Benthic categories employed for the classification of coral reefs quadrats in the Caribbean
Functional Group | Label | Description |
---|---|---|
Algae | Macroalgae | Upright macroalgae > 1 cm in height (all genus and species) including cyanobacteria films |
Algae | Turf and crustose coralline algae | Multi-specific assemblages of filamentous algae and CCA smothering reef surface 1 cm or less in height |
Coral | Acropora cervicornis | Acropora cervicornis |
Coral | Acropora palmata | Acropora palmata |
Coral | Agaricia,Undaria,Leptoserisspp. | Agariciidae, plates and encrusting corals of the genus Agaricia, Undaria and Helioseris |
Coral | Colpophyllia natans | Colpophyllia natans |
Coral | Coral meandroid | Hard coral meandroid including Dendrogyra, Isophyllia, Manicina, Mycetophyllia, Musa, Scolymia |
Coral | Coral submassive | Hard coral sub-massive, including Dichocoenia, Favia, Solenastrea, Stephanocoenia |
Coral | Diploria labyrinthiformis | Diploria labyrinthiformis |
Coral | Eusmilia fastigiata | Eusmilia fastigiata |
Coral | Madracis sp. | Madracis sp. |
Coral | Meandrina sp. | Meandrina sp. |
Coral | Montastraea cavernosa | Montastraea cavernosa |
Coral | Orcbicella complex | Orbicella complex, O. Annularis, O. faveolata, O. franksi |
Coral | Porites astreoides | Porites astreoides |
Coral | Porites branching | P. porites,P. divaricata,P. furcata |
Coral | Pseudodiploria sp. | P. strigosa, P. clivosa |
Coral | Siderastrea siderea | Siderastrea siderea |
Other | Other, invertebrates, fish | Other sessile and mobile invertebrates, fish |
Other | Sand | Unconsolidated reef sediment |
Other | Terrigenous sediments | Terrigenous sediments |
Other | Unclear | Unclear, cannot make any ID |
Other | Seagrass | Seagrass |
Soft Coral | Other soft corals | Other soft corals including erect and encrusting forms |
Soft Coral | Sea fans | Sea fans ,Gorgonia sp. |
Soft Coral | Sea plumes | Sea plumes and whips |
Sponge | Aplysina fistularis | Aplysina fistularis |
Sponge | Callyspongia vaginalis | Callyspongia vaginalis |
Sponge | Cliona viridis complex | Cliona viridis complex |
Sponge | Encrusting sponges | < 5 cm height (for whole individual); e.g. S. ruetzlen, Haliclona sp.,Monanchora sp., Chondrilla sp., Clathria sp., C. varians |
Sponge | Erect sponges | >1 cm height; height >>basal area; projections include: upright narrow tubes, branches, arborescent, e.g.A. compressa |
Sponge | Ircinia sp. | e.g. I. felix, I. strobilina |
Sponge | Massive sponges | Large basal area to body size and height > 5 cm; compact shape; irregular shapes include: chain-tubes, spherical, castle; e.g.E. ferox,Verongulasp. |
Sponge | Other sponges | Other sponges (undfined to species/morphology): e.g. A. wiedenmayeri,Niphates sp. |
Sponge | Rope sponges | Height >> basal area; Spread along the substrate, e.g.S. aura |
Sponge | Tube sponges | Grouped tube species: A. archeri, A. tubulata, C. plicifera, N. digitalis, unknown vase and small barrel group (tube openining ~ height; basal area < opening) |
Sponge | Xestospongia muta | Xestospongia muta |
Benthic categories employed for the classification of coral reefs quadrats in the Maldives and Chagos Archipelago
Functional Group | Label | Description |
---|---|---|
Algae | CCA | Crustose coralline algae |
Algae | CYANO | Cyanobacteria |
Algae | EAM | Multi-specific algal assemblage smothering reef surface 1 cm or less in height (Epilithic algal matrix) |
Algae | MALG | Upright macroalgae > 1 cm in height (all genera and species) |
Coral | ACR-BRA | Family Acroporidae; branching morphology (including hispidose type branching) |
Coral | ACR-OTH | Other corals from the family Acroporidae (e.g. Isopora) |
Coral | ACR-TCD | Family Acroporidae; table and corymbose morphologies (no digitate) |
Coral | BRA_DIG_Ac | Acropora with branches resembling fingers (digitate type); e.g. A. humulis |
Coral | BRA_OTH | Other branching genera including: Seriatopora; Anacropora; Echinopora; Montipora; Tubastrea (excluding Pocillopora and Stylophora) |
Coral | BRA_RND_St | Branching Stylophora |
Coral | FOLI | Thin Foliose and Plate colonies (excluding genera Acropora Porites) with/without visible relief structure: e.g. Echinophyllia; Turbinaria ; Montipora; Echinopora |
Coral | LOBO | Lobophyllia sp. |
Coral | MASE_LRG_O | Massive Submassive Encrusting colonies with large visible rounded polyps: Diploastrea; Favia; Favites; Montastraea; Galaxea; Goniopora; Astreopora |
Coral | MASE_MEA_O | Massive Submassive Encrusting colonies with meandering ridges and valleys resembling brain: Platygyra; Leptoria; Goniastrea |
Coral | MASE_SML_O | Massive Submassive Encrusting colonies with small or invisible polyps (including columnar forms): Pavona; Psammocorra; Coscinaraea ; Gardineroseris |
Coral | OTH-HC | Other hard coral including all other groups not represented by the other coral categories in Maldives and Chagos (e.g. Fungia; bleached colonies) |
Coral | POCI | Family Pocilloporidae; all genera (excluding Stylophora) and species |
Coral | POR-BRA | Family Poritidae; branching morphology e.g. P. cylindrica |
Coral | POR-ENC | Family Poritidae; encrusting morphology e.g. P. lichen |
Coral | POR-MASS | Family Poritidae; massive/sub-massive morphology e.g. P. lobata; P. lutea |
Other | OTHER | Other including fish; trash; transect hardware; unknown |
Other | Sand | Sand. Unconsolidated reef sediment |
Other | Seagrass | |
Other Invertebrates | Cliona | Encrusting Cliona spp. |
Other Invertebrates | COTS | Crown of Thorns Sea Star |
Other Invertebrates | Mille | Millepora sp. |
Other Invertebrates | MINV_OTH | Other mobile invertebrates: i.e Echinoderms; lobster |
Other Invertebrates | OTH-SINV | Other sesile invertebrates: bryozoa; clams; tunicates (ascidians); soft hexacorrallia; hydroids |
Other Invertebrates | SpMass | Massive or encrusting sponges |
Other Invertebrates | SpOTH | Other sponges (outside massive/encrusting morphologies and Cliona) |
Soft Coral | ALC-SF | Soft coral family Alcyoniidae; genera Lobophytum and Sarcophyton |
Soft Coral | OTH-SF | Other soft-corals outside of the common Alcyoniidae and erect forms (sea fans plumes whips): e.g. Xeniidae; Nepthtydae; Tubipora; Briareum; Heliopora |
Soft Coral | SINV_SFC_E | Sea fans; whips; plumes |
Benthic categories employed for the classification of coral reefs quadrats in the Southeast Asia region and Solomon Islands
Functional Group | Label | Description |
---|---|---|
Algae | CCA | Crustose coralline algae |
Algae | CYANO | Cyanobacteria |
Algae | EAM | Multi-specific algal assemblage smothering reef surface 1 cm or less in height (Epilithic algal matrix) |
Algae | MALG | Upright macroalgae > 1 cm in height (all genera and species) |
Coral | ACR-BRA | Family Acroporidae; branching morphology (including hispidose type branching) |
Coral | ACR-OTH | Other corals from the family Acroporidae (e.g. Isopora) |
Coral | ACR-TCD | Family Acroporidae; table; corymbose and digitate morphologies |
Coral | BRA_OTH | Other branching genera including: Seriatopora; Anacropora; Echinopora; Montipora; Tubastrea (excluding Pocillopora and Stylophora) |
Coral | BRA_RND_St | Branching Stylophora |
Coral | FOLI | Thin Foliose and Plate colonies (excluding genera Acropora Porites) with/without visible relief structure: e.g. Echinophyllia; Turbinaria ; Montipora; Echinopora |
Coral | LOBO | Lobophyllia sp. |
Coral | MASE_LRG_O | Massive Submassive Encrusting colonies with large visible rounded polyps: Diploastrea; Favia; Favites; Montastraea; Galaxea; Goniopora; Astreopora |
Coral | MASE_MEA_O | Massive Submassive Encrusting colonies with meandering ridges and valleys resembling brain: Platygyra; Leptoria; Goniastrea |
Coral | MASE_SML_O | Massive Submassive Encrusting colonies with small or invisible polyps (including columnar forms): Pavona; Psammocorra; Coscinaraea ; Gardineroseris |
Coral | OTH-HC | Other hard coral including all other groups not represented by the other coral categories in Southeast Asia (e.g. Fungia; bleached colonies) |
Coral | POCI | Family Pocilloporidae; all genera (excluding Stylophora) and species |
Coral | POR-BRA | Family Poritidae; branching morphology e.g. P. cylindrica |
Coral | POR-ENC | Family Poritidae; encrusting morphology e.g. P. lichen |
Coral | POR-MASS | Family Poritidae; massive/sub-massive morphology e.g. P. lobata; P. lutea |
Other | OTHER | Other including fish; trash; transect hardware; unknown |
Other | Sand | Sand. Unconsolidated reef sediment |
Other | Sediment | Terrigenous sediments |
Other | WATER | Water |
Other Invertebrates | COTS | Crown of Thorns Sea Star |
Other Invertebrates | Mille | Millepora sp. |
Other Invertebrates | MINV_OTH | Other mobile invertebrates: i.e Echinoderms; lobster |
Other Invertebrates | OTH-SINV | Other sesile invertebrates: bryozoa; clams; tunicates (ascidians); soft hexacorrallia; hydroids |
Other Invertebrates | SpMass | Massive or encrusting sponges |
Other Invertebrates | SpOTH | Other sponges; undefined to species/morphology (outside massive/encrusting/rope/tube morphologies) |
Other Invertebrates | SpRope | Height >> basal area; Spread along the substrate |
Other Invertebrates | SpTube | Tube species: tube opening ~ height; basal area < opening |
Soft Coral | ALC-SF | Soft coral family Alcyoniidae; genera Lobophytum and Sarcophyton |
Soft Coral | OTH-SF | Other soft-corals outside of the common Alcyoniidae and erect forms (sea fans plumes whips): e.g. Xeniidae; Nepthtydae; Tubipora; Briareum; Heliopora |
Soft Coral | SINV_SFC_E | Sea fans; whips; plumes |
To validate our method, we generated automated cover estimates on several randomly selected reef segments (roughly 40 consecutive survey images) and compared against estimates by human experts. After verifying that the estimated covers were highly correlated with those by human experts, we deployed the network on the whole set of collected imagery. Automated image annotation using this method was very fast; on average it only required 0.2 seconds, three orders of magnitude faster than manual analysis. Analysis of hundreds of thousands of images from a reef region could therefore be achieved in a manner of days. For example; over 190,000 survey quadrats from the GBR were automatically annotated in one week using a single computational unit (a NVIDIA Titan X GPU).
Further methodological details can be found at:
González-Rivero M., O. Beijbom, A. Rodriguez-Ramirez, T. Holtrop, Y. González-Marrero, A.
Ganase, Chris Roelfsema, S. Phinn and O. Hoegh-Guldberg. 2016. Scaling up Ecological
Measurements of Coral Reefs Using Semi-Automated Field Image Collection and Analysis. Remote
Sens. 8, 1:30; doi:10.3390/rs8010030.
González-Rivero M., P. Bongaerts, O. Beijbom, O. Pizarro, A. Friedman, A. Rodriguez-Ramirez, B.
Upcroft, D. Laffoley, D. Kline, R. Vevers, and O. Hoegh-Guldberg. 2014. The Catlin Seaview
Survey – kilometre-scale seascape assessment, and monitoring of coral reef ecosystems. Aquatic
Conservation: Mar. Freshw. Ecosyst. 24(Supp. 2): 184-198
Beijbom O, Edmunds PJ, Roelfsema C, Smith J, Kline DI, Neal BP, et al. 2015. Towards Automated
Annotation of Benthic Survey Images: Variability of Human Experts and Operational Modes of
Automation. PLoS ONE 10(7): e0130312. doi:10.1371/journal.pone.0130312
Beijbom, O, P.J.Edmunds, D.I.Kline, B.G.Mitchell, D.Kriegman. 2012. Automated Annotation of
Coral Reef Survey Images. IEEE Conference on Computer Vision (CVPR), Providence, Rhode Island,
June 2012