Roland Potthast: Gap between 2 kilometres
The German Weather Service (DWD) use supercomputers that do not use artificial intelligence to produce worldwide weather forecasts every three hours. Correctly read. They still exist, the last refuges that manage without artificial intelligence. We always wanted to know where these islands of bliss are.
What sounds understandable to the initiated, seems strange to laymen. Especially the DWD should be predestined for AI applications. With such a tremendous amount of data, they process daily, hourly and minutely. 30 terabytes being produced a day. Data from all kinds of observation systems such as satellites, aircraft, ground stations, radars and from almost all Central European radar networks flow into the systems.
German version: here
But also the huge amounts of globally collected weather data, i.e. the World Meteorological Organization (WMO) in Geneva, are distributed to the DWD for further processing.
From practice to the theory
From all this, the DWD calculates the global weather. As said before, every 12 hours for 7 days globally (and higher resolution over Europe) in advance, every 6 hours it calculates for up to 45 hours in advance and every 3 hours for 24 hours (high resolution over Germany).
About 80 colleagues of Prof. Dr Roland Potthast are working on this in Offenbach/Main. Potthast is Head of Data Assimilation at DWD. Potthast is also Professor of Mathematics at the University of Reading in England, has done research in the field of mathematics in the neurosciences and computer science and is, therefore, able to evaluate the topic of artificial intelligence in a differentiated way. He is also a physicist, but that doesn’t really matter for now.
Let’s stick to the weather. “We don’t work with artificial intelligence in the narrow sense. The processes we use have actually existed since the 1960s. Of course, they have been developed step by step — one could even speak of a quiet revolution.”
Such massive neural networks, which the DWD needed, cannot be trained at all. Potthast works with the classic models, to which even new AI algorithms are still far inferior, “so we can’t even get close to anything. There are no such complex neural networks. They can be set up, but training them and using them in a meaningful way is impossible so far”. The systems at DWD convert the meteorology and physics of the weather directly into equations, which are then solved by the supercomputer. “This is usually more accurate and sustainable than an AI approximation.”
But the described island of alleged bliss, where the AI has not yet found its way into, does not exist that way of course. Potthast, ineed, sees fields of applications for them at his workplace. He is currently establishing a centre for AI in weather research. For example in the calculation of certain elements of forwarding operators to model satellite observations. Here, the AI could improve the compression rates in order to increase the bandwidth of the phenomena or the precision of the algorithms already in use.
Potthast also sees possible applications for AI in the field of so-called physical parameterization. Keyword: modelling the complex nonlinear effect of droplet numbers within cloud microphysics. This allows a better classification of cloud formation in order to better understand and model how clouds are formed and how they react to temperature, pressure, winds and so on.
But he also sees the large and extensive field of observation errors in strong development with AI. These errors are often caused by very complex dependencies between many predictors like time of day, sun angle, air pressure, location within the annual cycle and clouds. “AI can offer us new approaches for quality control and for the assessment of error behaviour. Artificial intelligence is also used to optimize model output statistics, i.e. to adjust the models. Here, he recognises a large and almost “classic” field of application for artificial intelligence here, where a very positive development is predicted.
But the system is already learning
Why is the DWD actually so relaxed when it comes to AI? Overslept seem to be those who refuse to use AI. The answer lies here: “We have not yet worked with artificial intelligence in the strict sense. The procedures we use actually exist since the 1960s.” To better understand this, we will pick out the aspect of learning, which is exclusively assigned to AI.
At DWD, the algorithms have been learning themselves for more than 50 years. Example bias behaviour at measuring stations. One learns the bias from satellite measurements. Or as a newer development, the system learns whether a station is located on a mountain slope or in a valley and automatically envelopes this into the calculation.
It learns how to correct this already during the processing of the observation data, i.e. before the calculation of the forecast has even started; afterwards, it continues learning. Today, modern AI algorithms are being added and supplement or improve what has long been done in a classical way. “The silent revolution continues.”
Is Potthast proud of it? Yes. He is. He explains the “after-the-calculation” phase in great detail, the professor of mathematics almost falls into a system rapture. The expert characterizes the “after-calculation” phase as post-processing. This is now so good — based on the quality of the forecast itself, of course — that it can hardly be beaten. So so. Who could want to beat post-processing, Mr Potthast? He: “There are always competitions like this. Hobby meteorologists can try to improve the predictions. There are ten situations and the participants have to predict ten times. Then they look to see who was better on average. It is difficult or impossible! If a person has to beat the system ten times, the system is always better. That is exactly the same as if a normal person wanted to beat a chess computer: it is not possible.”.
What has become of it?
Now, something forces itself upon us. The question of what has become of the good old meteorology? Is Potthast, in the end, Head of a Development Department that feeds a supercomputer with self-learning algorithms? He confirms. “That’s how you could sum it up. Somehow everything comes from this computer in Offenbach near the beautiful Main River. Even the storm (warning) weather alerts come from this supercomputer.
“That, too, has long been automated, or at least partially automated. Sure, the warnings are examined by a meteorologist. He can also take corrective action. Such a computer is just blind. I mean, you can build in a lot of checks and balances, but it’s still good to have an expert who can keep an eye on it. Nevertheless, the basis comes from a high-performance computer.
Sure, meteorologists used to make the forecast themselves. Today they ensure that the systems are developed. The meteorologist has been given the gift of new creativity through technology, he states. He even goes so far as to say that the work has become more sophisticated. Meteorologists need to know how the system of computing and the models come about, where its weaknesses lie and, more importantly, what the system cannot do and how it could be further developed.
A mammoth task?
Probably. If you consider that the whole of Germany is provided with virtual points every two kilometres, whose weather-relevant data is fed into the model in question. Such a high-resolution model system naturally has weaknesses, and that is exactly between the two kilometres. There is nothing, only a rough average of the effects is taken into account. And the meteorologist knows these very gaps and can virtually speak for them and fill them in, fed by their experiences. So only humans can judge what is going on in these gaps. Let us calmly consider the meteorologist as a gentle pilot who can intervene at any time and always navigates us through the storms of our weather events.
And finally an AHA-experience. A revolution. The systems have gained one day of reliability per decade. Every decade, the DWD has perfected its accuracy to such an extent that it is possible to predict five days as reliably as 50 years ago one day. And what do you think of weather apps on smartphones, Mr Potthast? “They are fed by the supercomputer. They are as reliable as if you were a layman trying to forecast the weather.” But he also says: “The systems are very different. Better compare them with each other.”
Thanks to the DWD press office, without which this interview would not have been possible. And of course a big thank you to Professor Potthast for his time!
