Abstract
Low-impact fishing gear, such as fish pots, could help reduce human’s impact on coastal marine ecosystems in fisheries but catch rates remain low and the harvest of resources used for baiting increases their environmental cost. Using black seabreams (Spondyliosoma cantharus) as target species in the Bay of Biscay, we developed and assessed the efficiency of biodegradable biopolymer-based baits (hereafter bio-baits) made of cockles (Cerastoderma edule) and different biopolymer concentrations. Through a suite of deep and machine learning models, we automatized both the tracking and behavior classification of seabreams based on quantitative metrics describing fish motion. The models were used to predict the interest behavior of seabream towards the bait over 127 h of video. All behavior predictions categorized as interested to the bait were validated, highlighting that bio-baits have a much weaker attractive power than natural bait yet with higher activity after 4 h once natural baits have been consumed. We also show that even with imperfect tracking models, fine behavioral information can be robustly extracted from video footage through classical machine learning methods, dramatically lifting the constraints related to monitoring fish behavior. This work therefore offers new perspectives both for the improvement of bio-baits and automatic fish behavior recognition.
Similar content being viewed by others
Introduction
In our currently changing world, pot fisheries should not be considered “a return to the past, or to a lower technological level but rather a step towards a more sustainable future”1. Fish pots consist of two integral components: a bait to attract the target species and a containment unit. We here focus on the bait component by elaborating novel biodegradable, biopolymer-based baits (hereafter bio-baits) and developing a methodology to evaluate fish interest towards a bait in situ. Novel bait types can act as alternative to be more selective of the catch, to reduce cost, and to minimize its impact in the environment2. The landings of species harvested with pots, traps and crest, represent a small fraction of the total landings3 but these gears’ impacts are significantly lower than of active gears, both on the ecosystems4 and on the amount of greenhouse gas emissions throughout the harvest’s life cycle5. Pot fisheries therefore gain more attention as alternatives to towed fishing gears4 but innovative fish pots remain at their early stages and still require improvement to expand their usage in coastal fisheries.
Pots are highly diverse and different parts of a fish pot can be modified depending on the target species and local constraints (e.g., policy regulations, fishing practice, stock status)1,6. Floating pots with modified bait type help retain cod, Gadus morhua, while reducing bycatch of red king crab, Paralithodes camtschaticus7 while bottom-set pots target crustaceans while reducing bycatch of pelagic fish8. Pots have a lower operating cost and environmental cost than towed gears1,5,9 but designing efficient devices to target finfish is particularly challenging due to lower catch rates related to the quick consumption of baits compared to pots targeting crustaceans or cephalopods6,10.
Various species are currently harvested to obtain baits in order to attract target species. This harvest has been estimated to generate up to 31% of the greenhouse gas emissions generated throughout the life cycle of these passive gears fisheries11,12. In some extreme cases, the amount of fish harvest for baiting can even exceed the amount of target fish landed13. Thus, while passive gears are considered to be generally more sustainable than active gears, manufacturing efficient, eco-friendly bio-baits would enable to take out the harvest of edible species for baiting, further improving their sustainability. Long-lasting baits is among the most potent component for improving the currently limiting catch efficiency6. Biodegradable baits are still under development and have been previously elaborated with proteinaceous medium and water-soluble agents such as polyvinyl alcohol14, or by mixing natural and synthetic agents for longlines, showing encouraging results but only during laboratory trials2,15. Bio-baits based on biopolymers would enable to incorporate any amino-acid within the matrices, thus finding which pairs of matrices and amino-acid trigger attraction in target species would enable to obtain highly-selective baits. Their efficiency at attracting fish in situ has never been tested, while a target species behavior around a gear—or bait—has to be accounted for in each stage of design of a highly selective device with limited impact on non-target species16,17,18.
Even in controlled, laboratory set-ups, fish behavior can be difficult to predict19. This is even more striking in fishing gears environment where most information has been historically inferred from catch comparisons rather than from target species’ fine-scale behavior towards the device19, although recent studies now try to explicitly address species behavior (e.g. Ref.20). Behaviors are generally observed at a fine scale (< 10 m or < 10 min), focusing on how fish react to stimuli (attractive or repulsive) over a few seconds (e.g. Refs.21,22) to minutes of encounter (e.g. Refs.23,24) by scrutinizing swift changes in swimming direction, speed and even mouth or fin movements6. Direct observations of behaviors in fishing gears are thus becoming mandatory over long periods of time but at high spatiotemporal resolution25. These observations generate massive amount of data from which short behavioral sequences have to be extracted, which often dramatically reduces the amount of data actually analyzed for behavioral studies26, or with considerable human effort for manual processing or through citizen science projects27.
Computer vision coupled with deep learning models is now used for automatic object detection and tracking from underwater optics28,29, with a growing interest for automatizing behavioral data through objective quantification and synthesizing huge amounts of information in aquaculture to detect abnormal behavior and improve welfare30. Image quality and environmental factors (e.g., turbidity, luminosity, crowdedness) dictate the accuracy of such models31,32 but, in most cases, metrics can be derived from continuous tracking to provide insights into various aspects of fish behavior.
We here pursue a double objective: (1) develop long-lasting, biopolymer-based biodegradable baits to target black seabream and (2) assess the bio-bait’s efficiency in situ through a framework of artificial intelligence and machine-learning methods to automatically extract species tracks over a hundred hours of videos and process them through trajectory data-mining and movement ecology techniques developed in remote sensing and telemetry for planar data analysis33,34,35,36. This second aspect also aims at demonstrating how the automatization of fish behavior recognition will help in evaluating other types of baits than what we present here and species-specific fish-gear interactions.
Methods
Black seabream case study
In the Bay of Biscay, small-scale fisheries comprised 21% of the national production and black seabream accounted for 520 tons of landings in the area (2022; FOA Subarea VIII, Division A–D). Black seabream abundance peaks in the coastal area during the summer months, when catch of other species are relatively low. Currently, most landing originate from trawling and this seasonal fishery therefore presents an opportunity for the development of pots specifically designed to target this species18. Although there are past studies on the diet37, feeding locations38, and reproduction of black seabreams39 to improve fisheries management, little is known on their interactions with passive gears. Seabream pot development trials showed that a pot with a rectangular shaped entrance, transparent net, and floated circular form is the most optimal configuration to target black seabream18. The bait was chosen among 10 tested baits identified from the literature and interviews of recreational and professional fishers. Cockles, Cerastoderma edule, triggered the longest residence time of fish around the bait (about 2 h; 40). This bait was then used for the development of bio-baits.
Developing bio-bait with a longer temporal span
Bio-baits composition and manufacturing
Three types of long-lasting bio-baits were developed and tested in situ: C17, C600, and Lactips. These bio-baits were produced from a mixture of water-soluble polymers biodegradable in a marine environment and powdered cockle (Cerastoderma edule) flesh (Supplementary Table S1). These materials were selected for their solubilization and biodegradable properties in cold water. Preliminary tests have shown different solubilization kinetics in both freshwater and seawater40. Lactips is a bio-sourced, natural polymer derived from animal proteins, soluble even in cold water and biodegradable in various environments, including the marine environment. The bio-baits C17 and C600 are synthetic polymers based on polyvinyl alcohol (PVOH). Preliminary biodegradation tests conducted in this study on these polymers have revealed that the C17 polymer biodegrades 10% slower than the C600 in a sand-seawater environment, after a 1-year incubation. The three types of biopolymers were chosen from off-the-shelf biodegradable plastic materials tested for their water-soluble properties. They showed a progressive release of particles suitable for in situ testing.
These polymers were combined with an attractant made of cockle flesh, which was chosen because it is easily accessible and used by local fishers while part of the seabream diet37,41. Cockles were dried in a vacuum oven, ground, and sieved to 300 µm. The cockle powder was then mixed with the water-soluble polymers in the molten state to facilitate the dispersion of the attractants in the plastic matrix, in an internal mixer (Brabender® internal mixer; Supplementary Fig. S2a). The samples obtained were then granulated and thermo-compressed (Labtech® LP-S-50 STD Press) to obtain 8 × 8 cm rectangular plates with a thickness of 3 mm (Supplementary Fig. S2b). The proportions of powder incorporated (30% cockle powder or dry weight of 100 g) were identical for the three bio-bait samples.
Bio-bait in situ attraction assessment
The study was conducted in the Bay of Quiberon (northern Bay of Biscay, France) over a 900 m2 rectangular experimental area (Fig. 1a) consisting of sand, mud, coarse sediments, and algae, with depths from 7 to 12 m depending on tides. In situ deployments took place between July 20th and 24th, 2020 when black seabream migrates to coastal areas during the summer. Each bio-bait was mounted in its own observation setup and deployed at the extremities of the study area, with placement times adjusted according to tidal conditions. The observation devices consisted of one GoPro® Hero (4 or 7 Black) camera mounted on a metallic frame weighted with ballasts (Fig. 1b), to maintain a fixed 2 m distance between the camera and the bait. The raw baits were attached to a nylon thread and tied to the metallic rod while the bio-bait plates were placed inside a mesh bag. Recordings were made with a 1920 × 1080 p resolution at a frame rate of 25 fps on a 128 Gb micro-SD. Each camera was connected to a 15,600 mAh battery, inside a waterproof housing, to allow for continuous 9-h recordings (constrained by the micro-SD capacity). Four observation devices were immersed each day, with one control to validate the presence of fish in the study area using raw cockle flesh and three with different types of bio-baits. To enable comparison of the bio-bait attractiveness, the four devices were deployed simultaneously before midday, for video footage to be recorded in daylight (see Fig. 1c for an example). Four trials were carried out per bait, over four days, generating a total of 127 h of raw footage, used to analyze fish behavior around the bait (Fig. 1d).
(a) Maps of the study site, generated in R v4.3.3 with sf v1.0.16. (b) Schematic of the underwater observation set-up, (c) Field of view of the camera with black seabreams swimming around from a pre-processed video frame, (d) Behavioral metrics quantification from a fish trajectory relative to stimuli where B = Bait point, P = Proximity-to-bait zone, A = annotation area, d1 = distance per fish move, d2 = distance to bait, dx = change in x direction, dy = change in y direction, f = frequency in proximity area, R2n = squared net displacement, α = absolute angle, β = relative angle to stimuli, x1 y1 = top left pixel coordinate of A, x2 y2 = bottom right pixel coordinate of A.
Automatic fish behavior detection and quantification
Video processing
The dataset consists of 16 videos, lasting circa 9 h each. Only 15 videos are used for analysis due to a faulty camera on the control bait on day 1. Videos were subsampled at 5 fps (ffmpeg library42), a tradeoff between processing cost while retaining enough frames to capture swift fish movements43. All video processing (annotation, model training, testing, and validation) were performed with the image analytics platform VIAME44. Fish and bait classes were manually annotated frame-by-frame with bounding boxes with the standard COCO format (Common Objects in Context; \({x}_{1} {y}_{1}\), \({x}_{2}\), \({y}_{2}\)). The trained detector used a Cascade Faster R-CNN as the framework for object detection45 coupled with Siamese network46 as the neural network architecture for object tracking. These architectures have shown robust performances in tracking fishes from underwater images as they can address occlusions36,47,48.
Groundtruth dataset for fish detector and tracker (DetTracker)
The groundtruth dataset used to train and test the DetTracker consisted of 54,425 annotations across 13,771 frames, comprising six classes focused on fish-bait interactions (“bio-bait”, “unknown fish”, “Labridae”, “plant”, “raw bait” and “Spondyliosoma cantharus”; Supplementary Fig. S3). There are a total of 46,923 annotations of fish (all species combined) and 6142 annotations of bait (bio-bait and raw bait combined). Classes were unbalanced but sufficient for robust bait detection given their homogeneity throughout the videos. For model training, annotations were gathered from cockle bait videos collected in 2019 with the same devices and protocol18, and which had already been partially annotated for seabream. This allowed us to train and evaluate a model on videos from a previous data collection and assess its performance at handling new data (the videos collected in the current study). For evaluating the DetTracker performance on the bio-bait videos from 2020, annotations were also gathered from random 300-frame sequences from the different bio-baits videos with various light and turbidity conditions to encompass most “fish events”, such as fish swimming in groups, individually, and behind the bait, for a total of 40 min of video validated. The model training and predictions were run on a GPU GeForce RTX3070i with 24 Gb of RAM.
DetTracker evaluation
To evaluate the detection accuracy, Average Precision (AP) and mean Average Precision (mAP) metrics were computed49. To evaluate tracking accuracy (i.e., ID-switch, cut tracks, etc.), the High-Order Tracking Accuracy metric (HOTA) was computed with TrackEval50. These model evaluations were used to evaluate which aspects of the localization, detection, and tracking are best handled and account for these performances when assessing model outputs. The trained model was then used to generate fish tracks in all 15 videos (i.e., over 2,624,000 frames). The tracker attributes the highest classification probability of an object among all instances throughout its track (e.g., when a fish swam alone while well visible, the probability of being a fish tends to be higher than during occlusions). Assuming that a single robust detection throughout a track is sufficient to attribute the predicted classification to the entire track, false positive fish tracks were removed by setting a 0.89 threshold on the maximum detection confidence, discarding most non-fish entities, such as floating plant debris. To ensure reliability, detections above 0.89 threshold underwent manual verification, confirming their identification as fish species.
Quantifying and classifying fish behavior (BeClassifier)
Fish behavior quantification and classification from annotated fish tracks (Supplementary Fig. S3) was achieved through a suite of quantitative metrics extraction and selection, clustering and Random Forest Classifier51 (Supplementary Fig. S4). To quantify and classify fish behavior, behavioral metrics were first calculated from the fish groundtruth to capture various aspects of fish movement and interaction with the bait, encompassing attributes such as proximity-to-bait, swimming, changes in direction, size, and orientation relative to the bait (Fig. 1d). A comprehensive pool of 104 behavioral metrics was considered (full list available in Supplementary Table S5) which was reduced through a step-wise process (Supplementary Fig. S4). Metrics that were highly correlated (Pearson’s r > 0.90) were compared to exclude redundant metrics since they tend to decrease the performance of Random Forest classifiers51 that were used for behavior classification. This resulted in a selection of 72 metrics.
To determine the number of behavior partitions, Gap statistic, Elbow and Silhouette method52 were applied on the 72 metrics, and identified that the optimal number of clusters to group the dataset is two (Supplementary Fig. S6). Two dichotomic fish behaviors were thus initially considered (Interested versus Uninterested, referred to as principal behavior) for the manual behavior annotation of the groundtruth dataset. A total of 301 fish were classified as either Interested or Uninterested and annotated separately by three trained annotators, who were then confronted to reach a consensus on all sequences.
Once the principal behavior dataset had been generated, the number of metrics required to classify them were further reduced to 12 metrics through a recursive elimination approach53. The selected variables were chosen from Random Forest runs with the lowest Out-of-bag (OOB) error rate and with the smallest set of features to account for processing cost. This was achieved through 1000 Random Forest iterations (default hyperparameters) averaged among all iterations, then plotted against model accuracy.
The dataset of groundtruth fish principal behavior along with the 12 most relevant metrics were then used to train and test the behavior classifier (hereafter BeClassifier). Annotations were pooled from manually annotated fish tracks from 2019 and 2020 datasets used for the DetTracker training. For each dataset, a stratified 50–50% split was done to ensure a balanced representation of fish tracks from the two behavior classes. This involves separately applying the split ratio to each behavior class within every dataset. The proportions were then aggregated to form the final dataset pool. However, we deliberately excluded interested fish from the bio-bait videos to test the BeClassifier ability to classify these fish without prior exposure. A total of 204 seabreams were used for training and 97 were used for testing. The split used for the different model training and testing are provided in (Supplementary Table S7).
The 12 metrics selected were then inputted to train a Random Forest model to classify fish behavior with fine-tuned hyperparameters (mtry = 1, ntree = 100, maximum nodes = 50, node size = 2, sample size = 50, class weighting with the inverse of class frequencies, and Gini as the split criterion) obtained by doing hyperparameter optimization with grid search54, as recommended for handling small datasets. The BeClassifier model was tested with a k-fold cross validation method to consider a diverse representation of Interested and Uninterested classes for each fold. Seabreams that were classified as Interested were all manually validated to evaluate the model’s performance at predicting interested fish, both on raw bait and bio-bait videos. Model evaluation was assessed using standard metrics (F1-score, accuracy, precision, and recall). The probability of a fish being interested was set to 0.52 based on the prevalence of incorrect classification below this threshold (i.e., when the probability of being Interested and Uninterested is near even).
Fine-scale behaviors classification
Interested fish exhibited behavior of exploration and different feeding movements where they either feed during a sustained period or bite and nibble the bait then quickly retreat, while Uninterested fish were either passing by or actively retreating away from the bait. We thus tested if accounting for these fine-scale behaviors improved the model’s prediction of the principal behavior and evaluated the output as for the two-class behavior classifier, then assessed the temporal distribution of these fine-scale behaviors around the bio-baits. Eight fine-scale behaviors (detailed in Supplementary Information S8) could be characterized based on the fish behavioral catalog from 55, showing different trajectory patterns (Supplementary Fig. S9). The same procedure was applied to the BeClassifier for the fine-scale behaviors classification, resulting in 30 relevant metrics and fine-tuned (mtry = 5, ntree = 650, maximum nodes = 50, node size = 5, sample size = 150, class weighting with the inverse of class frequencies, and Gini as the split criterion).
Data analysis
All data analysis was conducted in the R statistical software56, with the packages tidyverse v2.0.0, dplyr v1.1.2 and ggplot2 v3.4.2 for data management and graphics, randomForest v4.7–1.1, for the behavioral classifier, and adehabitatLT v0.3.27 for behavioral metric calculation.
Results and discussion
Automatizing fish behavior classification from a large video database
Fish tracking in uncontrolled environment
Most studies aiming at developing low-impact, sustainable fish-targeting gear have been confronted with their low catch rates compared to traditional methods6,10. This case study is no exception as it is also confronted with a low number of observed fish. Still, the DetTracker identified a total of 202 fish tracks were detected above the species probability and behavior probability threshold throughout the 15 videos, with the model’s localization of fish instances presenting good performances at identifying seabream on the videos even with the overall low visibility (localization, Loc = 80.6%; detection, APseabream = 56.3%; Supplementary Fig. S10), consistent with previous studies57,58. Highly accurate trackers are developed on land59,60, and our results (tracking evaluation, HOTAseabream = 23.99%) highlight how underwater imaging remains challenging. Trackers that are trained to follow regular motion and distinct features (e.g., SORT, DeepSORT, Siamese-based trackers) can perform poorly when tracking organisms that look alike and move erratically61. The fish present in the videos have almost uniform appearances, and their movements turn from being regular to erratic as they approach the bait. When tracking fish where movements are expected to be regular, such trackers can suffice59,62,63 but not with non-linear movements61; altogether, these factors explain the moderate model’s tracking accuracy, as is often the case in fish crowds60,64.
Predicting fish behavior from imperfect fish tracks
To evaluate the bio-baits’ attraction power over long periods of time, the fish behavior around the bait needed to be identified among more than 2 million frames. The BeClassifier showed a high accuracy in classifying whether the fish is Interested or Uninterested from fish tracks (BeClassifier on principal behavior in the test set: Accuracy = 91.75%, F1 score = 90%, Precision = 90%, Recall = 90%; Supplementary Fig. S11). Validating the full video dataset further demonstrated a narrow linear relationship between the model's predictions and true interest, even in dense scenes (R2adj = 0.98, p-value < 10−10; Fig. 2) and that the model was able to positively detect nearly all interested fish throughout the video (BeClassifier on principal behavior in the full dataset: Accuracy = 88.91%, F1 score = 94.13%, Precision = 90.72%, Recall = 97.80%). The inflated counts when three or less fish are present are results of split tracks of fish remaining for extended periods of time within the frame of the camera (Fig. 2; Predicted Interest). The moderate HOTA therefore did not propagate over the performance of automatic behavior predictions, and behavioral patterns were conserved even without exact start-to-end fish tracking.
In a static dataset, setting a high threshold to keep only the detections that are accurately predicted may suffice to highlight representative trends65. In extensive in situ recordings however, trackers may inevitably capture noisy fish tracks due to uncontrollable environmental conditions and fish movements66. Tracks can still show general trends when accumulated together60, and our study supports that “imperfect predictions” of fish movement still enable to draw accurate conclusions on spatial and temporal patterns in fish behavior, even with minimal human validation (40 min over 127 h of video, i.e. less than 0.04% of the final dataset) as long as error rate remains acceptable, or controlled for specific classes.
Coarse versus fine-scale behavior prediction
We here applied a cluster-then-classify approach, which highlighted two main behavioral groups from the fish trajectories (Supplementary Fig. S6 of gap statistics, silhouette and elbow results). Yet, categorizing fish behavior into two main, antagonist behavior remains a simplistic approach as its behavioral pattern changes over time, and variability if often more interesting than uniformity67,68. A fish can indeed show a strong interest toward the bait for short periods and then stay in the field of view for a longer duration without approaching the bait. Techniques for splitting animal movement paths have been developed to pinpoint the fundamental segments of motion within a trajectory33, and highlighted that animal interactions with the presence of stimuli are associated with a higher frequency of changes in movement. Nevertheless, the BeClassifier performed poorly at predicting the eight fine-scale behaviors (per behavior prediction statistics available in Supplementary Fig. S12 and Supplementary Table S13, and accounting for the fine-scale behaviors to predict parent behaviors (i.e., Interested for Feeding, Biting, Nibble and Exploring; Uninterested for Passing, Startle, Burst swimming and Fleeing; Accuracy = 84.48%, F1 score = 82.43%, Precision = 79.31%, Recall = 85.19%) did not enhance the model’s ability to discern the overarching patterns (Fig. 3). We thus argue that changes in movement over time can still be depicted by a one-behavior-per-track approach as long as the quantified behavior is related to fixed stimuli. To further improve accuracy, seabreams tracks may need to be segmented into fixed intervals to depict subtle changes in behavior, in particular over long tracks69. However, the objective of the study is to assess the overall interest of seabreams toward the bait where fewer behavior classes suffice.
The importance of objectives metrics and clusters definition for robust fish behavior recognition
The tracker’s shortcomings being superseded for behavioral quantification may also result from the objective cluster-then-classify approach used along with the behavioral metrics selection from a large (> 100) set of metrics (Supplementary Table S5). Defining behavior classes can be subjective70,71, and having a polarity or binary classification of positive and negative response to stimuli can serve as the principal behaviors to initially look for72. Here, the fish tracks were then described by objective metrics used to cluster behaviors, which were, in turn, associated with an explicit behavior (i.e., the cluster-then-classify approach). Behavioral metrics with the highest ranking from the variable importance feature of Random Forest were all locomotion-based and stimuli-relative quantification of the fish’s swimming trajectory. Distance to bait and frequency in close proximity to stimuli are the strongest descriptors to classify behaviors around a cue as it is measured relative to the stimuli. Metrics like the number of fish present or the bounding box area as fish size proxy were uninformative. The cluster-then-classify approach thus enabled to handle the potential confusion of the trained behavior classifier when the fish’s interest manifested in different ways that could be attributed to opposite patterns (for instance, seabreams approaching the bio-bait through exploratory movements or quick approach and retreat; Fig. 3). As highlighted by the behavioral metric selection, locomotive-based metrics provide a more accurate way to identify behavior than relying solely on responses to stimuli and personality traits73,74.
Bio-bait efficiency in situ assessment
The presence and behavior of seabreams were then detected throughout the database to determine the efficiency of the bio-baits, following the assumption that fish interest would increase over time for bio-baits and decrease for raw bait. The count of fish activity is based on model predictions (Fig. 4a,b), while the fish interest is based on actual observations (Fig. 4c,d) since all model predictions for interest behavior have been manually validated. Seabreams were observed every day in the experimental area, and raw cockle bait attracted more fish than bio-baits (Kruskal–Wallis: chi2 = 12.021, p = 0.00731; Dunn test on raw bait vs bio-baits: all p < 0.05; Fig. 4a,b). Yet, peak activity around raw bait was limited to the first hour of immersion and interest afterwards were solely directed towards the nylon, which hold the raw bait until consumed, or the rod. Some fish detected on the raw bait videos during the second half of the deployment, with the bait already consumed, were mostly curious towards the apparatus and camera. Seabream interest toward the bio-baits was detected during the 4th and 5th deployment hours (Fig. 4c,d) where they showed feeding, nibbling and biting behaviors towards the bio-baits. Other species such as the European seabass (Dicentrarchus labrax) also approached and circled around the bio-baits during the 5th hour. Raw bait therefore outperforms bio-baits in terms of overall activity, but bio-baits show potential in terms of longevity as they continued to lure fish several hours post deployment and were also consumed by the target species.
Seabream presence on the footage and interest toward the baits along the 9-h deployments. (a) Count of fish activity in bio-baits based on model predictions, (b) Full-scale showing all counts in raw bait, (c) True interest of fish in bio-baits based on actual observations, (d) Full-scale showing all interest in raw bait.
Bio-bait and its future implications
The duration of baited fish pot deployment is limited by the bait’s efficiency to still lure fish over an extended period of time, which are costly to regularly replace and re-deploy75. Our results do highlight some potential for bio-baits to trigger fish interest several hours after raw bait has been consumed and attracted finfish only with no crustaceans spotted wangling around bio-baits as opposed to raw cockle baits. All bio-baits exhibited an activity peak in between the 4th and 6th hour after deployment while raw baits’ peak interest occurred within the first hour only (Fig. 4). Raw bait still had a much stronger attractiveness during this period since it is more potent and has a wider spread of odor diffusion from natural prey cues the species may be used to76. The results also show that only the bio-bait continued to attract fish several hours after being deployed. In a fisheries perspective, combining the raw and bio-baits would maintain the strong initial attractiveness while increasing odor diffusion over time. A natural bait, having a faster odor diffusion, would initiate the chemical cue, followed by the artificial bait that would gradually release odors after a few hours of immersion, but this would require obtaining natural baits from other sources and may, in turn, reduce the overall sustainability of the fisheries. Economic constraints should also be considered given that, as of now, the manufacturing cost of an artificial bait is higher than that of a natural bait. This may be compensated by reduced cost for professional's maintenance and sourcing of natural baits12. Several initiatives aim to value fisheries discards such as fish head, viscera and tail for baiting2 extracting their amino-acids and incorporating them into bio-bait matrices could be a solution but, again, only if discards are unavoidable.
This study remains exploratory, and several points of improvement remain to be addressed, in particular on the bait’s attractiveness. Even if preliminary experiments had been carried out, formulation optimization would be necessary to improve control bait release and attractiveness, for example by modifying the percentage of attractant in the mixture. While prior studies highlighted the significance of amino acids in water-soluble extracts in activating feeding behavior in fish, other factors like visual cues and the palatability of the bait itself play significant roles in fish responses2,19,77,78. However, synthetic amino acid mixtures may be enticing but seldom match the effectiveness of natural food extracts76. Previous research has identified black seabreams as an omnivorous and opportunistic species, consuming a diverse diet including mysidacea, crustaceans, polychaetes, and algae41,79. Improving long-lasting baits based on biopolymers may therefore not only consider potency but its composition in terms of amino acids. While here only cockles’ amino acids were tested, others could have triggered a stronger response but are yet to be identified. The appearance of the bio-baits can also be improved by having prey-shaped forms with subtly colored appearance78. One of the advantages of plastic materials is their ease of processing. Specific gynoid-type structures, for example, which can be envisaged using 3D printing technology, could improve the release of attractants, while working on the bait's shape to optimize visual appeal. The composition of the bio-bait could thus still be largely improved to diffuse more potent odors along with providing a more desirable texture.
Other amino acids and bio-bait shapes should still be assessed to increase its attractiveness. Bio-bait has the potential to be more sustainable than raw bait as the amount of organic material incorporated in the mixture can be controlled, and thus reduced. Still, it is important to consider how the decrease in organic material affects the mixture process, as well as whether this reduction weakens the distinctive cockle scent, potentially making the baits less appealing to fish. Using cockles from food waste could valorize leftovers that would otherwise end up in landfills. Various types of synthetic artificial baits have been developed that have explored various bait mixtures. Bait with a protein-rich gel interior and an insoluble skin-like exterior, incorporating fish waste to emit natural odors, have been introduced decades ago14. Innovations followed that utilized water-soluble polymers and plasticizers mixed with fish extracts, such as the biopolymer used in this study, to allow for controlled dispersion. Synthetic bait for crustacean capture that releases attractants at a controlled rate, showed effectiveness with various crab species and American lobsters in field trials80. These developments in bait technology aim to minimize the use of natural forage fish. When evaluating the viability and acceptance of these bio-baits by fishers, it is thus crucial to think about the source and quantity of cockles used and their impact on the environment. Here, the preliminary study on the biodegradation of the bio-baits used in this study found that after 180 days, the degradation rate reached 50–60%. However, the rate slowed considerably thereafter, stabilizing between 80 and 90% after 600 days in a controlled marine environment. This suggests a significant but gradual breakdown of the baits over time, which would, over longer time scale, strongly reduce the number of natural resources used for baiting.
Furthermore, for an exploratory study like this one, site selection was conducted randomly. In future trials, deploying the baits should also be done by varying locations while maximizing the distance among the sites81. This would enable to minimize bias from the same fish exploring the bait at different sites, or over repeated days. The fish activity may also be refined by monitoring the behavior of fish outside the field of view of the camera pointing at the bait, to estimate the optimal time needed to consider a fish entering the field of view as a new—versus a repeated—observation of fish (e.g., Refs.82,83).
This study presents new polymer-based and cockle powder bio-baits to attract black seabream over a longer period than classical baits. While the bio-baits efficiency remained low, it showed potential for bio-baits to attract fish after the natural bait has been consumed. To analyze the data, we developed a semi-automatic method to detect, track, and classify seabream behaviors from videos. We have shown that reliable behavior quantification can be achieved when applied to large volumes of data such as continuous video recordings and marginal validation, even with noisy patterns varying conditions on the in situ recording. We emphasize the ability of trackers, that may be deemed to have poor performance, to still have a significant importance in track-based behavior classification. Our approach, therefore, goes further than previous studies in terms of automatic fish behavior classification from tracks over long periods in situ (e.g., Ref.66), and widens our range of observational capabilities in assessing fish—or other aquatic organisms—behavior, interactions or response to stimuli in uncontrolled environment. From a fishing technology perspective, the fine knowledge of species behavior would enable conceiving highly selective, effective and sustainable, fishing gears and practices.
Data availability
The data and codes are available upon request addressed to the corresponding author, R.F., robin.faillettaz@ifremer.fr, or A.S.A., abanganalexa@gmail.com.
References
Petetta, A., Virgili, M., Guicciardi, S. & Lucchetti, A. Pots as alternative and sustainable fishing gears in the Mediterranean Sea: An overview. Rev. Fish Biol. Fisheries. 31, 773–795 (2021).
Masilan, K. & Neethiselvan, N. A review on natural and artificial fish bait. Int. J. Fisheries Aquat. Stud. 6, 198–201 (2018).
European Commission. Joint Research Centre. & European Commission. Scientific, Technical and Economic Committee for Fisheries. in The 2023 Annual Economic Report on the EU Fishing Fleet (STECF 23-07). (Publications Office, LU, 2023).
Kopp, D. et al. The low impact of fish traps on the seabed makes it an eco-friendly fishing technique. PLoS ONE 15, e0237819 (2020).
Hilborn, R. et al. Evaluating the sustainability and environmental impacts of trawling compared to other food production systems. ICES J. Mar. Sci. 80, 1567–1579 (2023).
Anders, N., Fernö, A., Humborstad, O. B., Løkkeborg, S. & Utne-Palm, A. C. Species specific behaviour and catchability of gadoid fish to floated and bottom set pots. ICES J. Mar. Sci. 74, 769–779 (2017).
Furevik, D. M., Humborstad, O. B., Jørgensen, T. & Løkkeborg, S. Floated fish pot eliminates bycatch of red king crab and maintains target catch of cod. Fisheries Res. 92, 23–27 (2008).
Melli, V., Riginella, E., Nalon, M. & Mazzoldi, C. From trap to nursery mitigating the impact of an artisanal fishery on cuttlefish offspring. PLOS One 9, e90542 (2014).
Olsen, L. Baited Pots as an Alternative Fishing Gear in the Norwegian Fishery for Atlantic Cod (Gadus morhua) (UiT The Arctic University of Norway, 2014).
Jørgensen, T., Løkkeborg, S., Furevik, D., Humborstad, O. B. & Carlo, F. D. Floated cod pots with one entrance reduce probability of escape and increase catch rates compared with pots with two entrances. Fisheries Res. 187, 41–46 (2017).
Ziegler, F. & Valentinsson, D. Environmental life cycle assessment of Norway lobster (Nephrops norvegicus) caught along the Swedish west coast by creels and conventional trawls—LCA methodology with case study. Int. J. Life Cycle Assess. 13, 487–497 (2008).
Driscoll, J., Boyd, C. & Tyedmers, P. Life cycle assessment of the Maine and southwest Nova Scotia lobster industries. Fisheries Res. 172, 385–400 (2015).
Winther, U., Hognes, E. S., Jafarzadeh, S. & Ziegler, F. Greenhouse Gas Emissions of Norwegian Seafood Products in 2017. 40 https://www.sintef.no/contentassets/0ec2594f7dea45b8b1dec0c44a0133b4/report-carbon-footprint-norwegian-seafood-products-2017_final_040620.pdf (2020).
Stephan, J. T. & Stephan, K. F. Gelled proteinaceous fish bait having cross-linked exterior and water-soluble interior. (1975).
Masilan, K. et al. Development of fish gelatin-based artificial fish baits incorporating bioattractants from seafood processing waste. J. Indian Chem. Soc. 99, 100376 (2022).
Graham, N., Ferro, R. S. T., Karp, W. A. & MacMullen, P. Fishing practice, gear design, and the ecosystem approach—Three case studies demonstrating the effect of management strategy on gear selectivity and discards. ICES J. Mar. Sci. 64, 744–750 (2007).
Larsen, R. B. et al. Performance of the Nordmøre grid in shrimp trawling and potential effects of guiding funnel length and light stimulation. Mar. Coast. Fisheries. https://doi.org/10.1080/19425120.2017.1360421 (2017).
Méhault, S. et al. Using fish behavior to design a fish pot: Black seabream (Spondyliosoma cantharus) case study. Front. Mar. Sci. 9, 1009992 (2022).
Rowland, W. J. Studying visual cues in fish behavior: A review of ethological techniques. Environ. Biol. Fishes 56, 285–305 (1999).
Chladek, J. et al. Using an innovative net-pen-based observation method to assess and compare fish pot-entrance catch efficiency for Atlantic cod (Gadus morhua). Fisheries Res. 236, 105851 (2021).
De Robertis, A. & Handegard, N. O. Fish avoidance of research vessels and the efficacy of noise-reduced vessels: A review. ICES J. Mar. Sci. 70, 34–45 (2013).
Yochum, N. et al. Evaluating Chinook salmon (Oncorhynchus tshawytscha) response to artificial light in support of bycatch mitigation. Can. J. Fisheries Aquat. Sci. 79, 912–924 (2022).
Olsen, S. & Laevatsu, T. Fish Attraction to Baits and Effects of Currents on the Distribution of Smell from Baits. https://repository.library.noaa.gov/view/noaa/12421 (1983).
DeBose, J. L. & Nevitt, G. A. The use of odors at different spatial scales: Comparing birds with fish. J. Chem. Ecol. 34, 867–881 (2008).
Krag, L. A., Madsen, N. & Karlsen, J. D. A study of fish behaviour in the extension of a demersal trawl using a multi-compartment separator frame and SIT camera system. Fisheries Res. 98, 62–66 (2009).
Robert, M. et al. A methodological framework for characterizing fish swimming and escapement behaviors in trawls. PLOS One 15, e0243311 (2020).
Spampinato, C. et al. Automatic fish classification for underwater species behavior understanding. in Proceedings of the first ACM international workshop on Analysis and retrieval of tracked events and motion in imagery streams—ARTEMIS ’10 45 (ACM Press, Firenze, Italy, 2010). https://doi.org/10.1145/1877868.1877881.
Logares, R., Alos, J., Catalan, I., Solana, A. C. & Ocampo, F. J. del. Oceans of big data and artificial intelligence. in Oceans. CSIC scientific challenges towards 2030 163–179 (2021).
Abangan, A. S., Kopp, D. & Faillettaz, R. Artificial intelligence for fish behavior recognition may unlock fishing gear selectivity. Front. Mar. Sci. 10, (2023).
An, D., Huang, J. & Wei, Y. A survey of fish behaviour quantification indexes and methods in aquaculture. Rev. Aquacult. 13, 2169–2189 (2021).
Yan, Z., Bi, Y., Xue, B. & Zhang, M. Automatically Extracting Features Using Genetic Programming for Low-Quality Fish Image Classification. in 2021 IEEE Congress on Evolutionary Computation (CEC) 2015–2022 (Kraków, Poland, 2021). https://doi.org/10.1109/cec45853.2021.9504737.
Guo, P. et al. An underwater image quality assessment metric. IEEE Trans. Multimed. 25, 5093–5106 (2022).
Thiebault, A. & Tremblay, Y. Splitting animal trajectories into fine-scale behaviorally consistent movement units: Breaking points relate to external stimuli in a foraging seabird. Behav. Ecolo. Sociobiol. 67, 1013–1026 (2013).
Zheng, Y. Trajectory Data Mining. ACM Transactions on Intelligent Systems and Technology (TIST) 6, (2015).
Harmer, A. M. T. & Thomas, D. B. pathtrackr: An r package for video tracking and analysing animal movement. Methods Ecol. Evolut. 10, 1196–1202 (2019).
Lopez-Marcano, S. et al. Automatic detection of fish and tracking of movement for ecology. Ecol. Evolut. 11, 8254–8263 (2021).
Dulčić, J., Lipeg, L., Glamuzina, B. & Bartulović, V. Diet of Spondyliosoma cantharus and Diplodus puntazzo (Sparidae) in the Eastern Central Adriatic. Cybium 30, 115–122 (2006).
Daban, İ. B. Comparative study on the feeding ecology of the White Seabream, Diplodus sargus, and the Black Seabream, Spondyliosoma cantharus (Osteichthyes: Sparidae) in the North Aegean Sea. 68, 126–134 (2022).
Pinder, A. C., Velterop, R., Cooke, S. J. & Britton, J. R. Consequences of catch-and-release angling for black bream Spondyliosoma cantharus, during the parental care period: implications for management. ICES J. Mar. Sci. 74, 254–262 (2017).
Mehault, S. et al. Projet BAITFISH : Behaviour, performAnce, Impacts of Trap FISH. 1–70 (2020).
Pita, C., Gamito, S. & Erzini, K. Feeding habits of the gilthead seabream (Sparus aurata) from the Ria Formosa (southern Portugal) as compared to the black seabream (Spondyliosoma cantharus) and the annular seabream (Diplodus annularis). J. Appl. Ichthyol. 18, 81–86 (2002).
Bellard, F. & Niedermayer, M. FFmpeg. http://ffmpeg.org/.
Porikli, F. & Tuzel, O. Object tracking in low-frame-rate video. in Image and Video Communications and Processing 2005 vol. 5685 72–79 (SPIE, 2005).
Dawkins, M. et al. VIAME: An Open-Source Framework for Underwater Image and Video Analytics. in 898–906 (Santa Rosa, CA, USA, 2017). https://doi.org/10.1109/WACV.2017.105.
Cai, Z. & Vasconcelos, N. Cascade R-CNN: Delving into high quality object detection. in IEEE Conference on Computer Vision and Pattern Recognition 6154–6162 (2018).
Bertinetto, L., Valmadre, J., Henriques, J. F., Vedaldi, A. & Torr, P. H. S. Fully-Convolutional Siamese Networks for Object Tracking. in Computer Vision—ECCV 2016 Workshops (eds. Hua, G. & Jégou, H.) 850–865 (Springer International Publishing, Cham, 2016). https://doi.org/10.1007/978-3-319-48881-3_56.
Mei, Y. et al. Recent advances of target tracking applications in aquaculture with emphasis on fish. Comput. Electron. Agric. 201, 107335 (2022).
Cai, L., McGuire, N. E., Hanlon, R., Mooney, T. A. & Girdhar, Y. Semi-supervised visual tracking of marine animals using autonomous underwater vehicles. Int. J. Comput. Vis. 131, 1406–1427 (2023).
Cartucho, J., Ventura, R. & Veloso, M. Robust Object recognition through symbiotic deep learning in mobile robots. in IEEE International Conference on Intelligent Robots and Systems 2336–2341 (2018) https://doi.org/10.1109/IROS.2018.8594067.
Luiten, J. et al. HOTA: A higher order metric for evaluating multi-object tracking. Int. J. Comput. Vision 129, 548–578 (2021).
Breiman, L. Random forests. Mach. Learn. 45, 5–32 (2001).
Maechler, M., Rousseeuw, P., Struyf, A., Hubert, M. & Hornik, K. cluster: Cluster analysis basics and extensions. (2023).
Darst, B. F., Malecki, K. C. & Engelman, C. D. Using recursive feature elimination in random forest to account for correlated variables in high dimensional data. BMC Genet. 19, 65 (2018).
Probst, P., Wright, M. N. & Boulesteix, A.-L. Hyperparameters and tuning strategies for random forest. WIREs Data Mining Knowl. Discov. 9, e1301 (2019).
Kalueff, A. V. et al. Towards a comprehensive catalog of zebrafish behavior 1.0 and beyond. Zebrafish 10, 70–86 (2013).
R Core Team. R: A language and environment for statistical computing. R foundation for statistical computing 2023 (2023).
Cui, F., Ning, M., Shen, J. & Shu, X. Automatic recognition and tracking of highway layer-interface using Faster R-CNN. J. Appl. Geophys. 196, 104477 (2022).
Li, X., Shang, M., Qin, H. & Chen, L. Fast accurate fish detection and recognition of underwater images with Fast R-CNN. in OCEANS 2015—MTS/IEEE Washington (Institute of Electrical and Electronics Engineers Inc., 2016). https://doi.org/10.23919/oceans.2015.7404464.
Bruch, R., Scheikl, P. M., Mikut, R., Loosli, F. & Reischl, M. epiTracker: A framework for highly reliable particle tracking for the quantitative analysis of fish movements in tanks. SLAS Technol. 26, 367–376 (2021).
Sun, P. et al. DanceTrack: Multi-object tracking in uniform appearance and diverse motion. in IEEE/CVF Conference on Computer Vision and Pattern Recognition 20993–21002 (2022).
Yang, C.-Y. et al. Multi-object tracking by iteratively associating detections with uniform appearance for trawl-based fishing bycatch monitoring. https://doi.org/10.48550/arXiv.2304.04816 (2023).
Negrea, C., Thompson, D. E., Juhnke, S. D., Fryer, D. S. & Loge, F. J. Automated detection and tracking of adult pacific lampreys in underwater video collected at snake and columbia river fishways. N. Am. J. Fisheries Manag. 34, 111–118 (2014).
Rodríguez, Á., Bermúdez, M., Rabuñal, J. R. & Puertas, J. Fish tracking in vertical slot fishways using computer vision techniques. J. Hydroinform. 17, 275–292 (2015).
Dawkins, M. et al. FishTrack23: An ensemble underwater dataset for multi-object tracking. in 7167–7176 (2024).
Faillettaz, R. et al. Imperfect automatic image classification successfully describes plankton distribution patterns. Methods Oceanogr. 15–16, 60–77 (2016).
Bilodeau, S. M., Schwartz, A. W. H., Xu, B., Pauca, V. P. & Silman, M. R. A low-cost, long-term underwater camera trap network coupled with deep residual learning image analysis. PLOS One 17, e0263377 (2022).
Egnor, S. E. R. & Branson, K. Computational analysis of behavior. Ann. Rev. Neurosci. 39, 217–236 (2016).
Chouinard-Thuly, L. et al. Technical and conceptual considerations for using animated stimuli in studies of animal behavior. Curr. Zool. 63, 5–19 (2017).
Gehring, T. V., Luksys, G., Sandi, C. & Vasilaki, E. Detailed classification of swimming paths in the Morris Water Maze: Multiple strategies within one trial. Sci. Rep. 5, 14562 (2015).
Urquiza-Haas, E. G. & Kotrschal, K. The mind behind anthropomorphic thinking: Attribution of mental states to other species. Anim. Behav. 109, 167–176 (2015).
Keefer, M. L. et al. Inter-observer bias in fish classification and enumeration using dual-frequency identification sonar (DIDSON): A Pacific Lamprey case study. Northw. Sci. 91, 41–53 (2017).
Hale, R., Piggott, J. J. & Swearer, S. E. Describing and understanding behavioral responses to multiple stressors and multiple stimuli. Ecol. Evolut. 7, 38–47 (2017).
Hassinan, C. W. et al. A quantitative analysis of locomotor patterns in developing C. elegans. 2023.06.03.543584. https://doi.org/10.1101/2023.06.03.543584 (2023).
Kanno, Y., Locklear, M. L., Platis, N. M. & Lewis, S. T. Body condition metrics explain fish movement in experimental streams. J. Zool. 320, 18–28 (2023).
Suuronen, P. et al. Low impact and fuel efficient fishing—Looking beyond the horizon. Fisheries Res. 119–120, 135–146 (2012).
Utne-Palm, A. C. et al. Feeding response of Atlantic cod (Gadus morhua) to attractants made from by-products from the fishing industry. Fisheries Res. 227, 105535 (2020).
Kawamura, G., Matsushita, T., Nishitai, M. & Matsuoka, T. Blue and green fish aggregation devices are more attractive to fish. Fisheries Res. 28, 99–108 (1996).
Ríos, J. M., Tesitore, G. & Mello, F. T. Does color play a predominant role in the intake of microplastics fragments by freshwater fish: An experimental approach with Psalidodon eigenmanniorum. Environ. Sci. Pollut. Res. 29, 49457–49464 (2022).
Box, A. et al. Diet and physiological responses of Spondyliosoma cantharus (Linnaeus, 1758) to the Caulerpa racemosa var. cylindracea invasion. J. Exp. Mar. Biol. Ecol. 380, 11–19 (2009).
Dellinger, A. et al. A synthetic crustacean bait to stem forage fish depletion. Glob. Ecol. Conserv. 7, 238–244 (2016).
Taylor, M. D., Baker, J. & Suthers, I. M. Tidal currents, sampling effort and baited remote underwater video (BRUV) surveys: Are we drawing the right conclusions?. Fisheries Res. 140, 96–104 (2013).
Langlois, T., Chabanet, P., Dominique, P. & Harvey, E. Baited underwater video for assessing reef fish populations in marine reserves. Secretariat South Pacific Community Fisheries Newsletter 118, 53–56 (2006).
Andradi-Brown, D. A. et al. Assessing Caribbean shallow and mesophotic reef fish communities using baited-remote underwater video (BRUV) and diver-operated video (DOV) survey techniques. PLOS ONE 11, e0168235 (2016).
Acknowledgements
The authors would like to thank Léa Bonniec for helping with the manual annotations, and the two anonymous reviewers for their insightful comments that significantly improved the manuscript.
Funding
This work was part of the BAITFISH project supported by France Filière Peche (FFP) and the Fonds Européens pour les Affaires Maritimes et la Peche (FEAMP) (PFEA390017FA1000006), and the IFREMER funded PhD of Alexa Abangan (Grant DS/2021/10).
Author information
Authors and Affiliations
Contributions
A.A., D.K., R.F. and S.M. designed the study, M.D. designed and produced the bait; D.K., K.B., R.F. and S.M. collected the data; A.A., K.B. and R.F. analyzed the data; All authors wrote the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Abangan, A.S., Bürgi, K., Méhault, S. et al. Assessment of sustainable baits for passive fishing gears through automatic fish behavior recognition. Sci Rep 14, 13110 (2024). https://doi.org/10.1038/s41598-024-63929-5
Received:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1038/s41598-024-63929-5
This article is cited by
-
Computer vision based approaches for fish monitoring systems: a comprehensive study
Artificial Intelligence Review (2025)






