Measuring Performance of Big Science Infrastructure Reviewing how science facility success is measured

Challenge

The Science and Technology Facilities Council (STFC) ISIS facility at the Rutherford Appleton Laboratory provides Neutrons and Muons to probe materials for research by scientists and industrialists from across the world. On average about 700-800 experiments are completed and 500 scientific papers published each year.
ISIS senior management tasked RED to review the way they manage the facility (their ‘operating model’) in order to compress research times, increase the number of experiments conducted and maximise the value of scientific outputs.
The first step, this Scoping Study, was to develop appropriate measures of effectiveness (MoE) and performance (MoP). The complexity and variety of experiments conducted at ISIS means that measures of effectiveness cannot be simply limited to the number of experiments conducted or the volume of data collected but must take into account the scientific value created by each experiment.

Approach

The RED team conducted on-site interviews with ISIS managers and reviewed ISIS’ recent annual reports and operational data in order to:

  1. Confirm how ISIS is expected to contribute to STFC’s overarching objective of maximising the value of it’s facilities to the UK by enabling world class research, innovation and skills development;
  2. Understand the core activities needed to turn proposals from scientists into experimental data and scientific advances;
  3. Determine the performance and effectiveness data already being produced and reported;
  4. Identify ‘gaps’ where new measures should be considered.

Outputs

The study demonstrated that ISIS operational performance is multi-faceted and cannot be captured by monitoring a single MoE such as the number of peer reviewed articles.
The need for additional measures was recommended in the report to the ISIS Management Committee. The analysis was used to help focus follow-on work to review the operating model for ISIS experimental delivery.

Print, email or share this page