The public launch of OpenAI's ChatGPT in November 2022 caused a media sensation and kicked off a rapid proliferation of similar Large Language Models (LLMs). However, the computing power needed to train and run these LLMs and other artificial intelligence (AI) systems is colossal, and the energy requirements are staggering. Training the GPT-3 model behind ChatGPT, for example, required 355 years of single-processor computing time and consumed 284,000 kWh of energy1. This is one example of a task that the human brain handles much more efficiently than a traditional computer, and researchers are investigating the potential of more brain-like (neuromorphic) computing methods that may prove to be more energy efficient. Physical reservoir computing is one such method, using the natural, complex responses of materials to perform challenging computations. Researchers from the University of Sheffield are investigating the use of magnetic metamaterials - structured at the nanoscale to exhibit complex and emergent properties - to perform such computations. In work recently published in Communications Physics, they have demonstrated an ability to tune the system to achieve state-of-the-art performance in different types of computation. Their results show that an array of interconnected magnetic nanorings is a promising architecture for neuromorphic computing systems.
Anyone who has witnessed the majestic and mesmerising flight of a murmuration of starlings has no doubt wondered how a flock of birds can achieve such synchronised behaviour. This is an example of emergence, where the interactions of simple things lead to complex collective behaviours. But emergence doesn't only occur in the natural world, and a group at the University of Sheffield is investigating how the emergent behaviour can be engineered in magnetic materials when they are patterned to have nanoscale dimensions.
Dr Tom Hayward, Senior Lecturer in Materials Physics at the University of Sheffield and author of this paper says,
Life is inherently emergent - with simple entities connecting together to give complex behaviours that a single element would not have. It's exciting because we can take simple things - which hypothetically can be very energy efficient - and make them manifest the kind of complexity we see in the brain. Material computation relies on the fact that many materials that exhibit some form of memory can take an input and transform it into a different output - precisely the properties we need to perform computation. Our system connects a series of tiny magnetic rings into a big ensemble. One individual ring in isolation shows quite simple behaviours. But when we connect them, they interact with each other to give complex behaviours.
Magnets have a number of properties that make them interesting for these kinds of applications:
Key to this research is understanding what's happening to these magnetic nanorings when they're connected together - the way that emergence changes the way they change magnetic states.
Dr Hayward explains;
X-ray PhotoEmission Electron Microscope (XPEEM) is a really good way of measuring magnetic contrast, and the I06 beamline offers this incredibly powerful toolset for measuring magnetic dynamics at this length scale.
The high-quality data the team collected at I06 allowed PhD candidate Ian Vidamour to gain a fundamental understanding of how the material behaved and develop ways to input and output data that "tune" the system to give very different computational properties from the same device.
After adding electrical contacts to the metamaterial to build an electrical device, the researchers used magnetic fields to input data and electrical resistances as the output, demonstrating state-of-the-art performance in diverse benchmarking computational tasks.
Developing a reconfigurable device like this raises the question of what happens when several of them - tuned to express different properties - are networked together. The brain has functional units contributing different things to its overall behaviour; will we see another level of emergence from these connected devices, so that they become more than the sum of their parts?
The team, which includes postdocs Charles Swindells and Guru Venkat, is also developing a single device with multiple inputs and outputs, and looking into more energy-efficient ways of driving the dynamic behaviour that don't rely on large magnets.
Another exciting area of research is the potential for using these magnetic materials as smart sensors.
Dr Hayward adds;
Traditional computers are designed to be stable - you don't want them to react to their environment. But magnetic materials respond to stimuli, and so there's the potential for using them as sensors that can process data - that can simultaneously feel and think. There's some truly exciting potential emerging from these tiny rings.
To find out more about the I06 beamline or discuss potential applications, please contact Principal Beamline Scientist Larissa Ishibe-Veiga: larissa.ishibe-veiga@diamond.ac.uk.
1 nnlabs.org. Power Requirements to Train Modern LLMs (2023).
Vidamour IT et al. Reconfigurable reservoir computing in a magnetic metamaterial. Communications Physics 6, 230 (2023). DOI:10.1038/s42005-023-01352-4.
Vidamour IT et al. Quantifying the computational capability of a nanomagnetic reservoir computing platform with emergent magnetisation dynamics. Nanotechnology 33.48: 485203 (2022). DOI:10.1088/1361-6528/ac87b5.
Dawidek, RW et al. Dynamically driven emergence in a nanomagnetic system. Advanced Functional Materials 31.15: 2008389 (2021). DOI:10.1002/adfm.202008389.
Diamond Light Source is the UK's national synchrotron science facility, located at the Harwell Science and Innovation Campus in Oxfordshire.
Copyright © 2022 Diamond Light Source
Diamond Light Source Ltd
Diamond House
Harwell Science & Innovation Campus
Didcot
Oxfordshire
OX11 0DE
Diamond Light Source® and the Diamond logo are registered trademarks of Diamond Light Source Ltd
Registered in England and Wales at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom. Company number: 4375679. VAT number: 287 461 957. Economic Operators Registration and Identification (EORI) number: GB287461957003.