On the need to avoid apple-to-orange comparisons in microplastic research

Recently, Hataley et al. (2023) provided a microplastic risk assessment for the Great Lakes, claiming to have employed methodologies outlined by Koelmans et al. (2020, 2022), Mehinto et al. (2022), and Redondo-Hasselerharm et al. (2023a). These methods aim to ensure consistent quantitative risk characterization across diverse microplastic datasets (Koelmans et al. 2020). Hataley et al. state that they applied established techniques to evaluate data quality (Koelmans et al. 2019; Redondo-Hasselerharm et al. 2023) and rescaled environmental data to represent appropriate microplastic size ranges, enhancing data set comparability (Koelmans et al. 2020; Kooi et al. 2021). This discussion article seeks to provide constructive feedback on Hataley et al.’s application of these risk assessment and management frameworks. While offering feedback is a standard scientific practice, it is crucial for us to ensure the accurate use of our methods to prevent misguided policy recommendations. We hold reservations regarding the assessment by Hataley et al. (2023), both in terms of calculation accuracy and results communication.

In their risk assessment for Great Lakes surface waters, Hataley et al. (2023) rescaled exposure data using a default power law exponent value of 2.64 for the distribution of microplastic particle lengths in freshwater systems, as specified by Kooi et al. (table S4 in Kooi et al. 2021). In the absence of specific microplastic data for the system of interest, using default values is acceptable. However, for a consistent risk assessment, hazard assessment data should also be rescaled using the parameters pertinent to freshwater microplastic particles. These parameters are a power law exponent of 1.68 for particle volume in the context of “food dilution” (FD) effects and 2.00 for particle area in the context of “translocation-mediated” (TM) effects mechanisms (see table S4 in Kooi et al. 2021). Unfortunately, Hataley et al. instead employed values of 1.48 and 1.50, which correspond to alignment parameters for marine surface waters (see Mehinto et al. table S4A; Kooi et al. table S4). This misalignment stems from their choice to use the effect thresholds reported by Mehinto et al. (2022), which are specific to marine systems. Mehinto et al. (2022) state that their alignments pertain to marine surface waters, mandated by California Legislature to develop microplastics management strategies for coastal ocean and estuaries (Coffin and Weisberg 2022). These effect thresholds also rely on marine particle data to calculate average particle heights, essential for estimating volumes and surface areas needed for FD and TM effect mechanisms, respectively. Moreover, the power law exponent used to correct for unavailable microplastic distribution fractions should be consistent with the exponent used for exposure data rescaling (2.64 for surface freshwater). However, Hataley et al. (2023) rescale effect thresholds using a power law exponent of 2.07, specific to marine surface waters (Mehinto et al. 2022), causing further inconsistency. In essence, Hataley et al.’s risk assessment aligns exposure data using freshwater particle size distribution data with effects data that have been aligned using marine particle size distribution data. This leads to an invalid risk characterization and erroneous risk conclusions.

Hataley et al. also state that they rescaled each data point to the size range relevant to FD, i.e., 1–5000 µm using methods developed by Koelmans et al. (2020) and Kooi et al. (2021). However, these methods do not use this size range for this purpose. After all, FD refers to ingested particles, and the bioavailability criterion for FD is the mouth opening of the organism, which varied from 36 to 400 µm in Koelmans et al. (2020), for example. The assumption that all particles up to 5000 µm can be ingested in the context of FD may lead to an overestimation of the exposure. Furthermore, the authors also implement a rescaling of the exposure data to a range of 1–83 µm to allow comparison of exposure with translocation-based effect thresholds for marine surface water reported by Mehinto et al. (2022), and for translocation-based effect thresholds for freshwater sediment reported by Redondo-Hasselerharm et al. (2023). However, these latter thresholds have already been rescaled from a size range of 1–83 µm to 1–5000 µm, which means that the bioaccessibility corrections are actually performed twice and the exposure data are no longer aligned with the effect data.

Calculation of propagating errors and recalculation of risk values are beyond the scope of this correspondence article. However, due to the nature of the parameters being exponents, the calculated risk values are highly sensitive to errors in their values. To illustrate this point, let’s consider the impact of using the correct power law exponent α_x = 1.68 for volume instead of the erroneous value of 1.48, in the context of an organism with an ingestion bioaccessibility limit (i.e., mouth size opening) of 100 µm.

Employing eq. 1 (for detailed explanation, refer to Kooi et al. 2021), this adjustment results in a value for μ_x,poly that is 15 times lower. Given that the effect threshold EC_poly for polydisperse microplastic adheres to the relationship EC_poly × μ_x,poly = EC_mono × μ_x,mono (Koelmans et al. 2020), where the term EC_mono × μ_x,mono remains unchanged, the threshold effect concentration EC_poly becomes 15 times higher. A swift computation using the ToMEx database and RShiny application (Thornton Hampton et al. 2022), the Mehinto et al. (2022) framework, and freshwater alignment values from Kooi et al. (2021) yields thresholds for the FD mechanism as follows: 20 (T1), 89 (T2), 194 (T3), and 880 (T4) particles/liter. These thresholds surpass the aligned thresholds for marine surface water, as employed by Hataley et al. (2023), by factors ranging from 26 to 67.

We offer two examples where risk assessments are conducted using appropriately aligned data. First, Coffin et al. (2022) evaluated the risk of microplastics in San Francisco Bay, utilizing the thresholds from Mehinto et al. (2022). Given that San Francisco Bay is a marine system, exposure data were scaled using parameters relevant to marine surface waters, resulting in a proper alignment with the Mehinto thresholds designed for marine systems, thereby ensuring consistent risk characterization. Second, Koelmans et al. (2023a) present an example in the context of freshwater. They assessed the risk of microplastics in the Great Lakes for both surface water and sediments, rescaling exposure data using parameters specifically tailored to surface freshwater, akin to the approach pursued by Hataley et al. However, recognizing the inapplicability of Mehinto et al.’s (2022) thresholds, Koelmans et al. (2023a) calculated new thresholds grounded in freshwater-specific data. They also incorporated more recent data, surpassing the data used by Mehinto et al. (2022).

A robust risk assessment originates from the protection goal, which in the case of the Great Lakes is safeguarding freshwater aquatic communities. Ideally, species sensitivity distributions (SSDs) should include data from species within the lakes. If data are lacking, generic SSDs for freshwater species should be used. Based on the same considerations, Mehinto et al. (2022) aimed to create SSDs based on marine species data as much as possible. However, due to data limitations, they had to merge marine and freshwater species data. It was demonstrated that their HC₅ (hazardous concentration affecting 5% of species) largely depended on the most sensitive species, which were marine, diminishing the influence of freshwater species. Consequently, while defensible for marine systems, the application of such SSDs for freshwater systems, as done by Hataley et al. (2023), raises concerns (Koelmans et al. 2023a). Hataley et al. (2023) justified their use of Mehinto et al.’s thresholds based on past studies; however, their assessment could have been more reliable if they had used new data which is more applicable to freshwater. For instance, recent available studies such as Rico et al. (2023) and Koelmans et al. (2023a) provide SSDs tailored for freshwater systems. Koelmans et al. (2023a) recalibrated thresholds for Great Lakes surface water and sediments, using freshwater-specific data that were first screened and quality-controlled. We recommend that while a risk assessment model or framework can retain its conceptual validity over time, recalculating its outcomes whenever new data become available is advisable. For instance, the HC₅ effect thresholds grounded in SSDs exclusively focused on freshwater species, as reported by Koelmans et al. (2023a), surpass those derived from the combined SSDs reported by Mehinto et al. (2022) by a factor of 500. This discrepancy appears to stem from the heightened influence of a small number of sensitive marine surface water species within the latter dataset.

Transparent and honest communication and restraint in extrapolating the meaning of results are fundamental to maintaining trust in science, and this principle extends to the communication of uncertainty (Harris and Sumpter 2015; Mebane et al. 2019; Wardman et al. 2020). Microplastics constitute a complex suite of contaminants (Lambert et al. 2017; Gouin et al. 2019; Koelmans et al. (2023b), and the risk assessment methodology adopted by Hataley et al. (2023) is inherently complex, yielding quantitative outcomes that inherently carry uncertainty. Regrettably, Hataley et al. (2023) omit the quantification of this uncertainty. Their presentation of risk conclusions and the absence of risk fails to account for the statistical significance of differences between exposure concentrations and effects threshold concentrations.

In contrast, several preceding risk assessments that employed the same alignment methods conducted quantitative evaluations of uncertainty and/or variability arising from alignment procedures (e.g., uncertainty in power law exponents) (Mehinto et al. 2022; Coffin et al. 2022; Koelmans et al. 2023a; Redondo Hasselerharm et al. 2024). Furthermore, these assessments encompassed uncertainties stemming from factors such as sampling volume (Koelmans et al. 2023a), hydrological variability (Koelmans et al. 2023a), and HC₅ effect thresholds (Coffin et al. 2022; Koelmans et al. 2023a), while simultaneously scrutinizing the quality of input data. They demonstrated that the probabilistic uncertainty inherent in risk characterization could span up to 14 orders of magnitude (Koelmans et al. 2023a). Notably, even though their risk characterization ratio (RCR = MEC/PNEC) suggested an absence of risks for sediments in the Great Lakes, this study revealed that a portion of the RCR distribution exceeded a value of 1, indicating a certain likelihood of risks. In contrast, by focusing solely on the RCR and disregarding the associated uncertainty, Hataley et al. conclude that the data suggest a “no risk” scenario for sediments in the Great Lakes. Neglecting uncertainty can consequently result in misleading conclusions concerning ecological risks, and to insufficient or confusing information for risk managers.

Moreover, Mehinto et al. (2022) undertook a quantitative assessment of experts’ confidence in the established thresholds, assigning scores from 1 (indicating very low confidence) to 5 (indicating high confidence). The confidence levels in threshold values were generally low to moderate, averaging between 2.4 and 3.0. Furthermore, these scores exhibited considerable variability among experts, with individual ratings spanning from 1 to 4. It is reasonable to assume that risk managers would prefer to base their decisions on health standards that are beyond contention in terms of their quality. This accentuates the necessity to quantify uncertainty, yet Hataley et al. (2023) overlook any deliberation on this aspect.

Hataley et al. (2023) report a misapplication of the marine framework developed by Mehinto et al. (2022), as they applied it to a freshwater ecosystem. This misapplication has resulted in a risk assessment that lacks both meaning and robustness. As a consequence, the conclusions drawn for management purposes lack a solid scientific foundation. We assert that upholding the credibility of scientific findings is crucial, especially when those findings have the potential to impact policy decisions. Clearly, the avoidance and transparent reporting of such errors are imperative, which is why we initiated this discussion.

While uncertainties persist regarding the actual risk that microplastic particles pose in the Great Lakes region, this should not be construed as an argument for delaying action. While Hataley et al. suggest the inclusion of microplastics under Annex 3 (Chemicals of Mutual Concern) of the Great Lakes Water Quality Agreement (GLWQA), our analysis of available options leads us to propose that decision-makers can already initiate potentially impactful measures aimed at reducing the release of plastic debris into the watershed. These measures would also address the additional sources of microplastics. Specifically, within Annex 2 of the GLWQA, there are several options that can be explored. These options include the development of an integrated nearshore framework, to be collaboratively implemented through the lakewide management process for each Great Lake. This framework should encompass considerations of non-point source runoff, shoreline hardening, climate change impacts, habitat loss, invasive species, dredging and contaminated sediment issues, bacterial contamination, contaminated groundwater, and other factors where they are identified as sources of stress to the nearshore environment. The presence of plastic litter represents a number of concerns to the nearshore environment, which include the potential to adversely impact environmental and human health as well as socioeconomic costs; thus, prevention of plastic litter and microplastic should be prioritised as part of the lakewide management process. Additionally, given the observation that some nearshore environments present a potential risk (Koelmans et al. 2023a), it would be beneficial and effective to initiate management actions aimed at supporting research, monitoring, and other scientific priorities. These actions should be directed towards the assessment and identification of mitigative strategies to address current and future potential threats to water quality.

The above recommendation to take certain management actions in the Great Lakes region in light of remaining uncertainties of microplastics risks follows a recent precedent established in the State of California. Specifically, the California Ocean Protection Council (OPC) adopted a “two-track approach” to comprehensively manage microplastics, including multi-benefit solutions the state can act upon now while scientific knowledge further develops (e.g., pollution prevention, pathway intervention, education, etc.), and science to inform future action (e.g., better monitoring and hazard assessment) (OPC 2022). In a more synonymous parallel to the Hataley et al. study, the California State Water Resources Control Board is considering (at the time of writing) placing three waterbodies in San Francisco Bay in Category 3 of the Clean Water Act’s 305(b) list based on the ecological risk assessment in Coffin et al. (2022) (State Water Resources Control Board 2023). This consideration is based on the uncertainty in the risk assessment due to the size rescaling and other data correction techniques involved, which resulted in a determination of these water bodies having “insufficient data and/or information to make a beneficial use support determination but data and/or information indicates benefices uses may be potentially threatened” (State Water Resources Control Board 2023).