Data Analysis and Scientific Software Group -

Author: Alun Ashton (Data Analysus and Scientific Software Group)

DAWN Steering Committee and Data Visualisation

: Figure 1: DAWN’s new DataVis perspective showing processed data from I15-1 displayed as a waterfall plot (available in DAWN 2.8 released March 2018).

This year saw the formation of the DAWN Steering Committee; a collection of Diamond scientists selected to help guide the fundamental data visualisation features and general usability of the DAWN package. Under the supervision of the committee, a completely new 3D visualisation library has been integrated, greatly adding to DAWN’s existing tools.

The DataVis perspective developed in collaboration with the committee also integrates with the latest version of GDA, and is being used for live visualisation of raw and processed data on the XPDF beamline (I15-1).

Small Angle Scattering

: Figure 2: Mapped scan showing the orientation of collagen in a sample overlaid on an image of a human eye (L. Paletto, Albon Group, Cardiff University, UK, Collected October 2017) (left). A gridless Pilatus 2M image gathered from 3 detector frames at differing positions (Y. Zhang and S. Prabakar, New Zealand Leather and Shoe Research Institute, Collected August 2017) (middle). A wide angle X-ray scattering (WAXS) scan highlighting various levels of background subtraction, made possible by applying data corrections before subtraction (Stawski Group, GFZ Centre for Geosciences, Germany, Collected July 2017) (right).

Building on one of last year’s key successes whereby DAWN gained the functionality to automatically process data following data acquisition; this year’s focus has moved to harnessing this capability, extending the functionality available within DAWN to provide bespoke, in situ, data analysis for users during experiments.

This effort has given rise to a number of processing steps designed to analyse a multitude of data types, providing results or parameters of interest to the user. In particular, for small angle scattering users, this has led to the development of a number of analysis routines for orientational, structural and kinetic characterisations of a sample.

These advances have been underpinned by a rigorous set of data corrections, as described in our recent J. Appl. Cryst. publication (B. R. Pauw, et al. 2017). This has facilitated progress in developing rigorous background subtraction routines, which can also be run autonomously, alongside the development of a methodology for the removal of a detector grid to provide one continuous, uniform, dataset.

Eclipse Science and IoT

This year at Diamond, we hosted one of the Eclipse Science group and Internet of Things (IoT) workshops, which involved several invited talks and an IoT playground. Participants from Diamond, around the Harwell campus and the wider business community all enjoyed the excellent presentations and demos which showcased the state of the art in software for science.

Savu

: Figure 3: A rendering of an insect collected on I13 reconstructed using Savu.

2017 saw the release of Savu version 2.0, which features an improved user interface and significant performance increases. The roll-out of Savu onto Diamond’s mapping tomography beamlines has reduced multi-modal processing from eight hours to ten minutes, and Savu is now able to replicate the standard full-field tomography processing of its predecessor in less computational time.

Now routinely used on Diamond beamlines, Savu provides a larger array of processing methods and more flexible processing than previously available to Diamond’s tomography community. It also provides a platform for beamline staff to contribute their own methods in the form of Savu plugins, which has led to a rise in the sharing of methods and collaboration between beamlines.

After recent integration of Savu with GDA data acquisition software, the Joint Engineering, Environmental and Processing beamline (I12) and X-ray Imaging and Coherence beamline (I13) now perform auto-processing on tomography data as soon as data collection is complete. This allows beamline scientists to immediately identify if there are any problems with the current experimental setup.

Zooniverse

: Figure 4: View of Diamond’s ‘Science Scribbler’ on Zooniverse.

Building on last year’s success with the SuRVoS software for analysing data from the Full Field Cryo-Transmission X-ray Microscope beamline (B24), SuRVoS has gone on to analyse biological data from other imaging beamlines as well, such as the Imaging branchline (I13-2). In addition to this, the scope has been extended further by making use of the Zooniverse platform (www.zoonoverse.org) to host some work on protein misfolding disorders, specifically Huntington’s disease initially, but with the aim to extend to other problems where humans are better than machines at identifying information in complex image data.

ISPyB

The ISPyB database and SynchWeb software services store and present experimental meta-data, generate beamline performance statistics and provide remote access to users and beamline scientists. To reflect its growing importance, the team supporting the ISPyB and SynchWeb software services is increasing. A full time project lead was appointed in September 2017 and will be followed by a full time SynchWeb developer joining the team in early 2018. Additional support has also been contracted to re-establish support for mobile devices.

New features added to SynchWeb in the past year have included the integration of automated sample shipping to the facility. This includes support for users across the EU. Work has also been underway to add support to new and emerging techniques including XPDF beamline (I15-1) and continued refinement of the in situ beamline VMXi.

Diamond and the Scientific Software team continue to support the international ISPyB collaboration through developer, scientific and steering committees. Diamond co-hosted the most recent joint ISPyB/MXCuBE meeting in January 2018. One of the key areas for discussion in the collaboration is the growing field of cryo-electron microscopy. The team at Diamond established the integration of electron microscopy experimental data with ISPyB through the development of Scipion workflows. The development activities continue to enhance ISPyB as it evolves from a bespoke solution for a single technique to a general purpose Laboratory Information Management System (LIMS) for Diamond.

: Figure 5: ISPyB/MXCuBE meeting in January 2018.

Acknowledgements:
We wish to thank all our collaborators, Diamond beamline staff and users for their continued support and input.

On this website

About Diamond

For Users

Industry

Science

Instruments

Careers

Public

Procurement

Software & Tools

Sub-navigation

In This Section

Sub Navigation

Opportunities at Diamond

Subscribe to our Corporate and Science Update mailing list

Data Analysis and Scientific Software Group