1. Introduction
One of the advantages of recent index standardization models is that they provide reliable measures of uncertainty through innate error propagation or improved model specification (
Thorson and Barnett 2017;
Luo et al. 2022;
Yalcin et al. 2023), which can then be propagated into subsequent modelling steps. Error propagation methods have received a lot of attention in other fields (
Girard 2004;
Yen et al. 2014;
Bachi et al. 2021;
Zhang 2021) and stock assessment models are increasingly built with the aim of capturing and/or propagating these uncertainties into final population estimates (e.g., estimating index uncertainty in state-space models,
Nielsen and Berg 2014;
Aeberhard et al. 2018, incorporating the index uncertainty within Bayesian priors,
Yin et al. 2019, etc.). While these approaches have improved stock assessments, there is still further work to be done.
Information obtained from resource assessment surveys typically represent a primary source of data that populate stock assessment models. However, it is often impossible to measure all fish in a survey tow, especially for small species. In these cases, it is common to only measure a subsample and assume it is representative of the whole population (
Nasmith et al. 2016;
Hardie et al. 2018;
DFO 2024). Length–weight (LW) models, often based on a cubic relationship (
Froese 2006), are fitted to these subsamples and then used to predict the weight of all other sampled fish. These weights are then summed up to obtain a total “observed” biomass for each tow. However, most index standardization models (
Thorson and Barnett 2017;
Luo et al. 2022;
Yalcin et al. 2023) or design-based estimators (
Smith 1996;
Kimura and Somerton 2006) assume that these biomass estimates are actual data without associated uncertainties. Doing so may lead to inaccurate indices and associated uncertainty metrics, which can have serious consequences for fisheries management.
Our goal here is to develop a reliable biomass index for the Georges Bank sea scallop (Placopecten magellanicus) that takes into account all known sources of uncertainty. To achieve this goal, we test six different LW methods to determine which provides the most reliable biomass predictions. We compare three aggregation approaches for quantifying the accompanying uncertainty, and run a simulation experiment to assess their accuracy and precision. We conclude with some broader insights into appropriate methods for both uncertainty quantification and error propagation when working with LW data.
4. Discussion
We found a substantial reduction in the size of CIs by incorporating the raw data directly, and the errors therein, rather than using aggregated tow-specific sums assumed to have been directly observed, i.e., without errors. The increased sample size (instead of 234 tows, the sample size is 5594 scallops) and the efficiency gains from modelling individual data points rather than sums and means thereof are responsible for this reduction in uncertainty. This suggests a relatively straightforward gain in efficiency easily available for most fisheries assessments that could lead to more reliable survey indices.
For selecting the most appropriate model, fisheries science has provided varying advice, whether favouring parsimony (
Thorson 2019) or flexibility (
Bolker et al. 2009). There is therefore an argument for selecting both the
Spatial or
Spatial Both method. Given their similar diagnostics and predictive capabilities, the main difference is their performance with the two different CI approaches. For a correctly specified model with a large sample size, theoretical
Direct CIs and nonparametric
Bootstrapped CIs should be similar in size as confirmed by our simulations. A loss of efficiency in the
Direct CIs in our case study for the
Spatial Both model might therefore be an indication of model misspecification, pointing toward the
Spatial method as the more accurate method. This is further supported by the simulations results mimicking the case study results when setting the
Spatial method as the data-generating model. However, this is contradicted by the superior performance of the
Spatial Both method in terms of AIC, BIC, and cAIC, meaning that guidance around model selection might instead suggest the
Spatial Both as the appropriate method.
Irrespective of the model selected, incorporating spatial random effects substantially improved results according to every measure considered in this study (diagnostics, RMSPE, AIC/BIC). These results substantiate earlier recommendations to account for spatial dynamics within stock assessment processes (
Punt 2019). Much of the effort to date has focused on incorporating spatial dynamics directly into the stock assessment models (e.g.,
Cadigan et al. 2017;
McDonald et al. 2021), but many other components of stock assessments (growth rates, age-length curves, etc.) can also benefit from the inclusion of spatial dynamics (e.g.,
Cahill et al. 2020;
Yin et al. 2022). Even if the stock assessment model is itself nonspatial, accounting for spatial dynamics in the development of the indices feeding the assessment model can result in improved model estimates and reduced uncertainty (
Thorson et al. 2015). The spatial LW models used here can help identify spatial differences in LW relations, which, as our simulations show, can help reduce bias in both the LW relationship and in biomass-based growth rates that rely on LW models for their estimation (e.g.,
Nasmith et al. 2016). Furthermore, recent computational advances such as the development of sdmTMB (
Anderson et al. 2022) and the starve package (
Lawler et al. 2023) have greatly reduced the complexity of developing spatial models and make it relatively straightforward to operationalize these methods within existing stock assessment frameworks.
Moreover, these methods would benefit any stock assessment that uses design-based estimation to obtain survey indices in their preliminary steps. Indices developed from stratified surveys rely on the underlying strata properly representing the variability in the population being monitored and aggregating areas that tend to be mostly homogeneous regarding the variable of interest (
Särndal 1978). However, this is an often challenging process, and many fisheries utilize simpler method to define their strata (e.g.,
Trzcinski et al. 2009;
Hardie et al. 2018). Using spatial models to standardize catch rates (e.g.,
Luo et al. 2022) should account for spatial variability in the ecosystem and incorporate observed variability in stock productivity. These spatial variables (e.g., spatial growth rates) can then be used to delineate homogeneous areas and improve the identification of these strata. Building on this approach, one could then extend it to identify these homogeneous areas to support the delineation of larger stock units based on biologically-meaningful information (
Cadrin 2020).
These findings reflect a similar conclusion to other analyses incorporating observation uncertainties within stock assessments (
McDonald et al. 2022;
Breivik et al. 2024): ignoring preliminary modelling and analyses can introduce unnecessary uncertainty into the stock assessment process. The analyst should always keep in mind the nature of the data generating process and the nature of the actual data selected in the observed sample so as to properly analyze it. In our example, the sample is not the tow-specific “observed” biomasses, but rather measurements of all individual captured animals from which all subsequent analyses derive.
Our findings suggest confidence intervals for indices in multiple fisheries may actually be smaller than previously thought and their uncertainty could benefit from a reassessment. Specifically for the Georges Bank sea scallop assessment, which currently utilizes a Bayesian model that estimates index variances as captured by credible intervals (
Jonsen et al. 2009;
Hubley et al. 2014), our approach results in a more reliable index forming the basis for the science advice provided to management. More generally, this type of analysis could be undertaken for multiple other components of stock assessments and potentially further improve the uncertainty quantification for analyses including the estimation of age-length curves (e.g.,
Trzcinski et al. 2009), of yearly growth parameters (e.g.,
Koeller 2006;
Maunder et al. 2016), or the inclusion of externally-predicted covariates increasingly used to include the impact of the environment (
Pedersen et al. 2022).
However, one should keep in mind that there are various ways in which an index could be unrepresentative of the actual population dynamics. These include but are not limited to space- and time-varying catchability (
Thorson 2011;
Kotwicki and Ono 2019), confounding between various processes (e.g., between selectivity, growth and fishing mortality (
Maunder et al. 2016), and population movements (
Tanaka et al. 2020)). In addition to these challenges, even when sample sizes appear relatively large, the proportion of the area that these surveys cover is often extremely small. Most standard statistical methods cannot properly account for sampling that might be unrepresentative and assessing how representative a given sample is in an environment as uncertain as the ocean is exceedingly difficult.
While reduced uncertainty may suggest the index well reflects what was observed within a given survey, it also puts into perspective the large number of unknown complex ecological factors impacting both the sampling process and the underlying population dynamics. There are no easy solutions here, although the recent growing interest in new survey methods such as towed or drop cameras (
Stokesbury and Bethoney 2020) might, in conjunction with traditional surveys (essential for stock assessment components such as growth, aging, etc.), improve the ability of surveys to cover more space and more closely track the overall population health. Thus, while these methods reduce the uncertainty by providing statistically sound estimates, the practitioner should remain cognizant of other factors (e.g., poor survey design) that could impact their confidence in the indicator but cannot currently be captured using these techniques.