AI and machine learning are improving weather forecasting, but they won’t replace human experts

A century ago, the English mathematician Lewis Fry Richardson came up with a surprising idea for the time: to build a systematic process based on mathematics to predict the weather. In his 1922 book, “Weather Prediction By Numerical Process”, Richardson tried to write an equation he could use to solve the dynamics of the atmosphere based on hand calculations.

It didn’t work because not enough was known about atmospheric science at that time. “Perhaps one day in the bleak future, it will be possible to advance the calculations faster than the weather advances and at a cost lower than the saving made by humanity thanks to the information acquired. But it It’s a dream,” concluded Richardson.

A century later, modern weather forecasts are based on the kind of complex calculations dreamed up by Richardson – and they have become more accurate than anything he envisioned. Especially in recent decades, steady advances in research, data and computing have enabled a “quiet revolution in numerical weather prediction”.

For example, a forecast of heavy rainfall two days ahead is now as good as a forecast for the same day was in the mid-1990s. Errors in forecast hurricane tracks have been halved over the of the past 30 years.

There are still major challenges. Thunderstorms that produce tornadoes, large hail or heavy rain remain difficult to predict. And then there’s chaos, often described as the “butterfly effect” – the fact that small changes in complex processes make the weather less predictable. The chaos limits our ability to make accurate forecasts beyond about 10 days.

As in many other scientific fields, the proliferation of tools such as artificial intelligence and machine learning holds great promise for weather forecasting. We’ve seen some of what’s possible in our research on applying machine learning to high-impact weather forecasting. But we also believe that while these tools open up new possibilities for better forecasting, many parts of the job are best handled by experienced people.

Australian meteorologist Dean Narramore explains why it’s hard to predict big thunderstorms.

Predictions based on storm history

Today, the main tools of meteorologists are numerical weather forecasting models. These models use observations of the current state of the atmosphere from sources such as weather stations, weather balloons and satellites, and solve equations that govern the movement of air.

These models are excellent at predicting most weather systems, but the smaller a weather event, the harder it is to predict. As an example, think of a thunderstorm that dumps heavy rain on one side of the city and nothing on the other side. Additionally, experienced forecasters are remarkably good at synthesizing the huge amounts of weather information they must consider every day, but their memories and bandwidth are not infinite.

Artificial intelligence and machine learning can help address some of these challenges. Forecasters now use these tools in a number of ways, including making predictions of high-impact weather that models can’t provide.

In a project that started in 2017 and was reported in a 2021 article, we focused on heavy rain. Of course, part of the problem is defining “heavy”: two inches of rain in New Orleans can mean something quite different from what it is in Phoenix. We accounted for this by using observations of unusually heavy rainfall accumulations for every location across the country, along with a forecast history from a numerical weather prediction model.

We fed this information into a machine learning method known as “random forests”, which uses many decision trees to split a mass of data and predict the likelihood of different outcomes. The result is a tool that predicts the likelihood that rains heavy enough to generate flash floods will occur.

Since then, we have applied similar methods to forecasting tornadoes, large hail and severe storm winds. Other research groups are developing similar tools. National Weather Service forecasters use some of these tools to better gauge the likelihood of hazardous weather on any given day.

An excessive precipitation forecast from the Colorado State University-Machine Learning Probabilities system for extreme precipitation associated with the remnants of Hurricane Ida in the mid-Atlantic states in September 2021. The left panel shows the predicted probability of excessive precipitation , available the morning of August 31, more than 24 hours before the event. The right panel shows the resulting observations of excessive precipitation. The machine learning program correctly highlighted the corridor where heavy rain and widespread flooding would occur. Russ Schumacher and Aaron HillCC BY-ND

Researchers are also integrating machine learning into numerical weather prediction models to speed up tasks that can be computationally intensive, such as predicting the conversion of water vapor into rain, snow or hail.

Machine learning models may eventually replace traditional numerical weather prediction models. Instead of solving a set of complex physical equations like models do, these systems would instead process thousands of past weather maps to learn how weather systems tend to behave. Then, using current weather data, they would make weather forecasts based on what they learned from the past.

Some studies have shown that machine learning-based forecasting systems can predict general weather patterns as well as numerical weather prediction models while using only a fraction of the computing power required by the models. These new tools don’t yet predict the local weather details that people are interested in, but with many researchers carefully testing them and inventing new methods, the future is bright.

Maps of an evolving machine learning forecast for an outbreak of severe weather in the US Midwest in December 2021.
A forecast from the Colorado State University-Machine Learning Probabilities system for the December 15, 2021 severe weather outbreak in the US Midwest. Panels show the progression of the forecast from eight days ahead (lower right) to three days ahead (upper left), as well as severe weather reports (tornadoes in red, hail in green , destructive wind in blue). Russ Schumacher and Aaron HillCC BY-ND

The role of human expertise

There are also reasons for caution. Unlike numerical weather prediction models, forecasting systems that use machine learning are not constrained by the physical laws that govern the atmosphere. It is therefore possible that they produce unrealistic results – for example, predicting extreme temperatures beyond the limits of nature. And it’s unclear how they will perform during highly unusual or unprecedented weather events.

And relying on AI tools can raise ethical concerns. For example, locations with relatively few weather observations with which to train a machine learning system may not benefit from the forecasting improvements seen in other regions.

Another central question is how best to incorporate these new advances into the forecasts. Finding the right balance between automated tools and the knowledge of expert human forecasters has long been a challenge in meteorology. Rapid technological advances will only complicate matters.

Ideally, AI and machine learning will enable human forecasters to do their jobs more efficiently, spending less time generating routine forecasts and more time communicating the implications and impacts of forecasts to the public – or, for private forecasters, to their clients. We believe that close collaboration between scientists, forecasters and forecast users is the best way to achieve these goals and build confidence in machine-generated weather forecasts.

The conversation

Russ Schumacher, Associate Professor of Atmospheric Sciences and Colorado State Climatologist, Colorado State University and Aaron Hill, Research Scientist, Colorado State University.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Sherry J. Basler