A four-man team has harnessed a university’s bank of computers to create a supercomputer – at a fraction of the cost of similar projects.
Millions of pounds are being spent by some universities on buying supercomputers to handle intense levels of research but the team at De Montfort University (DMU) in Leicester developed a way of linking their high-spec computers to utilize their unused resources, especially at weekends and night-times.
Since Spring 2012, a string of machines with more than 500 computational cores between them in DMU’s Creative Technology Studios have been linked into a cluster, labelled the Memenet.
Controlled by an external computer across the road in Gateway House in the university’s Centre for Computational Intelligence (CCI), the cluster has been working in the background creating algorithms (problem-solving programs) without affecting work being done by students.
“All the problems of the world are optimisation problems,” explained Professor Ferrante Neri, from the CCI. “To optimize means to make a clever choice. Our role is to build up the intelligence of the machine so that the computer has to autonomously make choices – so it can behave in an intelligent way. It has to solve some optimization problems.
“Part of our project consists of letting the machine design the algorithms itself. The machine analyses the problem and makes the decision.
“In order to do this we need to run very extensive computational experiments – many algorithms, many times, on many problems.
“Other universities have been spending a high budget to purchase supercomputers. Our idea is that we have many little computers and we can distribute our optimisation algorithms over all the available computational cores.
“At the same time, we do not use the full power of each machine because those machines are for the students to use. Our processes are going on in the background using only a little part of the capabilities of each machine.
“We can do this without disturbing the students at all. Actually the students can continue their normal study day without even realising that our algorithms are running in the background.”
The idea came between Professor Neri, who arrived at DMU in spring 2012 from a university in Finland, and Dr Lorenzo Picinali, a senior lecturer in music technology who runs DMU’s Fused Media Lab, whose shared Italian origins had brought them together.
Because the audio, video and multimedia tasks done by Dr Picinali’s students sometimes need heavy and extensive processing, the computers in the
Creative Technology Studios are much more than standard desktop machines. They typically cost about £2,500, have eight to 16 core processors, high bandwidth RAM and multiple hard-drives.
But, they are only really used during weekday working hours, and only during term-times.
The beauty of the Memenet project, which runs on Kimeme (www.cyberdynesoft.it), a software package developed by Professor Neri’s former PhD students in Finland, is it costs little more than the extra electricity used.
And a computational cluster built up in this way can potentially be expanded indefinitely, reaching thousands of cores, if needed, with no extra cost.
In comparison, last year, the universities of Warwick, Nottingham, Birmingham and Queen Mary (London) launched a joint project to set up a £3.5 million supercomputing hub available to small businesses to optimise process design. And earlier this year the University of Loughborough launched its own £1million supercomputer.
Since last spring, one of Professor Neri’s PhD students, Fabio Caraffini, also an Italian, has been the ‘pilot’ of the cluster, with the crucial role of administrating the cluster and taking care of most of the software issues.
Jos Lange was also recruited from the beginning of this June through the university’s Frontrunner student employment scheme to measure the Memenet’s impact on students’ work.
Dr Picinali said: “We needed to test the impact of running our Memenet on a student’s standard work, so we created six local tasks of different degrees of system usage, and measured their impact.
“For example, in week one we ran the six tests without Memenet operating in the background. Then in week two we introduced low-usage tasks from the Memenet side. We found that when Memenet is using the network heavily, there are only a few situations when it might affect the local tasks being done by students.
“But we can synchronize the use of Memenet with the course timetable to avoid those situations and, of course, we can run it overnight and during holidays when the students are not working.”
Professor Neri said it fits with many of the strategies at DMU, especially with the aim of using our existing resources better.
“It also fits with sustainable development. For example, the huge supercomputers have to be disposed of at the end of their lives but the methods of cooling them use toxic materials. We don’t have those toxic materials,” Neri said.
As well as the creation of a prototype supercomputer, the Memenet has already seen the publication of 20 research papers, with more planned to follow.
“We hope we can show other universities what we have done and that they can have a supercomputer without spending £1million” said Professor Neri.