Whilst annual fish stock assessments make up the backbone of ICES advice, quality assurance dictates that the assessments themselves are also assessed. This is where the practice of benchmarking – that is, settling on the best assessment and data methodology for a particular fish stock – comes into play.
Since their inaugural year in 2009, benchmark workshops have convened at ICES to determine which stock assessment model (and settings) best fits their needs for future updates. This entails an in-depth analysis of the sort of data available to use and the uncertainty or bias in them. Stock distribution, forecast method and reference points are also reviewed. Once workshop experts decide on a model, the methodology is laid down in a stock annex, becoming the basis for advice for a number of years.
Benchmarking, carried out every three to five years for main stocks to keep pace with situational changes, takes time to come to fruition: groundwork prior to workshops such as preliminary testing of analysis models takes about five to seven months. This may include a specific data compilation workshop held beforehand to catch any blips there might be in the statistics and to allow stakeholders to put forward any new data or material.
With such a process, continual transparency and credibility are of course vital. ICES Advisory Programme Professional Officer Barbara Schoute described independent involvement in the workshops.
"What we try and do is involve stakeholders – industry and NGO representatives," she said. "We have a group of external experts who come from outside the ICES area, or at least outside ICES realm of the stocks involved. The experts are invited based on the benchmarking issues at hand."
Ian Gatt, Chief Executive of the Scottish Pelagic Fishermen's Association, was one of the stakeholders in the recent Benchmark Workshop on Pelagic Stocks (WKPELA 2014).
"This was my first benchmark as a representative of the Pelagic RAC," he stated. "The process was entirely transparent with industry attending as a full partner. Of course much of the debate was hugely complex but clearly all aspects of the individual strands of work were minutely scrutinized."
Moving towards ecosystem benchmarking
Another dimension of benchmarking revolves around the question of how the process looks beyond single fish stocks and out onto the wider landscape of full ecosystems and multispecies considerations.
"Right now we do benchmarking in single species," continued Schoute. "We had a Baltic benchmark (last year, WKBALT 2013), where all the main Baltic stocks – herring, sprat, cod – were benchmarked. This meant they looked at multispecies implications and links between stocks."
"What comes out of benchmarking is a single species stock annex – what we call the recipe book for the stock assessment itself. Single species reference points are part of the stock annex, but if we have multispecies considerations they are also taken into account in relevant reference points, for instance."
"This is important because we try to study the whole ecosystem instead of just single stocks – and that's why we're moving towards ecosystem benchmarking."
Benchmark groups usually ply their trade around the beginning of the year, before March when the stock assessments themselves generally start.