In the engine room

When people talk about artificial intelligence, they are usually referring to the fascinating possibilities for seemingly countless applications. The practical use of AI, however, is merely the tip of a technological iceberg. Hidden beneath the surface are vastly greater layers of infrastructure—from fundamental AI models to software and hardware.

Funded by the Carl Zeiss Foundation, the »Interactive Inference« Research Training Group at the Institute of Computer Science trains the next generation of experts for this »AI engine room«, where there is a massive shortage of skilled professionals.

»The technological foundation of artificial intelligence receives far too little attention in the public discourse, even though this is where the most value is generated,« says Prof. Joachim Giesen from the Research Training Group. »Both the fundamental AI models as well as the software and hardware are mostly not from Europe.« It is therefore vital, he argues, that young computer scientists here learn to understand, analyse, and develop AI infrastructure.

The researchers in Jena are focusing on what is, in simple terms, the cognitive process of AI: inference. When users send a request to ChatGPT, for example, inference software generates the desired response—in this case, a piece of text. This inference software has been taught using vast amounts of data in a process known as training.

Just as it can take some time for humans to learn something, AI training can be a prolonged process, several months in the case of large AI models. However, answers to an inference query should be available much faster—ideally in just a few milliseconds. To answer queries from millions of users at this speed, significant computing power is needed, which consumes a lot of energy and causes correspondingly high costs and CO₂ emissions.

The Jena researchers see a promising method for making inference more efficient in optimizing so-called tensor calculations, which can be used to map out many problems and are therefore frequently used in AI infrastructure. »A tensor is a mathematical concept that allows us to depict complex data in an organized way,« explains Prof. Alexander Breuer from the Research Training Group.

»If a data point consists of several numbers —for example, a wind direction—then this data point is called a vector. If we need several vectors to store data, we get a matrix, a two-dimensional grid. And, if several matrices are used, this is referred to as a higher-order tensor with more than two dimensions.«

This structured representation of data offers many possibilities for accelerating computing processes. »If, for example, many numbers within the tensors are zeros, many calculations on the tensors also result in zero—this can be taken into account in advance to avoid unnecessary computations,« adds Giesen.

Efficient calculation of probabilities

The experts in Jena are researching these and other methods to increase efficiency to accelerate inference queries, for example, on probabilistic AI models. These are models in which probabilities play a major role and which are used in fields such as logistics.

AI, for example, can compute how likely it is that passengers will need a ride at a certain location at a certain time, taking account of the regular traffic volume or potential roadworks. »We expect that such applications will become increasingly important in the future and are therefore focusing on the development of the underlying AI infrastructure,« says Joachim Giesen.

In the years ahead, AI will become an integral part of just about every area of our lives. This makes it even more important that infrastructural issues receive more attention. »The computing effort involved in both training AI models and inference is already massive and will continue to increase,« says Breuer. »While hardware performance is also increasing, its programming will no longer be as straightforward.« Excellent training for early-career computer scientists is therefore essential.