Forecasting in the Clouds

by Daniel Bartucci 2 Comments

When you’re watching your local meteorologist, does this thought sometimes cross your mind? “I wish I could be wrong more than half the time and keep my job.”

Yet still we watch. Why? The weather is something that we encounter every day and it’s something that we all have in common. For some folks, simply asking about the weather is their go-to icebreaker for any conversation. And, this past year there was no shortage of sensational weather stories, from severe weather abnormalities to new weather terms like “bombogenesis.” It’s no wonder that weather is topping social media newsfeeds — not to mention your most annoying Facebook friend posting pictures from their front window.

Just as weather anomalies have been dominating conversation at the dinner table, another type of cloud has emerged in the mainstream news at a record-setting pace: cloud computing.

See what I did there? I went from weather clouds to cloud computing. Beyond the clever “cloud” word play, there are some other interesting interactions going on with these two different kinds of clouds that you hear about every day.

Some industries have capitalized on using cloud computing and are looking towards a seemingly endless upside to leveraging the technology. Interestingly though, even with the direct impact weather forecasting has in our lives, American weather organizations have not always kept pace with other industries in adopting this new technology. That being said, weather forecasting is poised to undergo a bit of a Renaissance as supercomputing can now leverage an incredible amount of cloud-based data never possible before.

A Historical Perspective of Computing and Weather

In 1996, IBM took the lead accessing and analyzing data via cloud computing to improve weather forecasting with a project that would eventually be called Deep Thunder — weather nerds unite when project names are that cool.  The project was somewhat ahead of its time, many years before the many millions of forecasting data points that exist today were available. This was the birth of the “weather cloud vs cloud computing” confusion. In a Wakefield Research study commissioned by Citrix in 2012, 51% of a sample group of 1,000 people believed that bad weather affected cloud computing. I wish I was joking.

The Data in Today’s American Weather Forecast

To put the weather evolution in better perspective, we have to review what actually goes into a weather forecast. Your local television meteorologist has access to forecast data provided by a broad variety of primarily old-school data sources, including:

  • Twice a day, 365 days a year, 92 weather balloons are released in the United States alone, a collection method that dates back to the late 1800s. Trying to explain this to the youngest generation is equal to explaining the rotary phone and VCR.
  • Our most reliable hurricane data depends on the Air Force Reserve’s 53rd Weather Reconnaissance Squadron.
  • In early April, trained tornado spotters enter the roadways in the Midwest to attempt to add all the minutes they can to public warnings.  

The data collected with these methods is as valuable as ever, but the access to it and analysis of it can benefit from 21st century technology advancements.

U.S. Weather Forecasting is Behind the Times

For the last several years, the U.S. forecasting models have come under harsh criticism for their accuracy. The National Oceanic and Atmospheric Administration (NOAA) and the National Weather Service (NWS) have a lack of funding, resulting in outdated technology and lack of access to multiple important data sources.  

A good example is the storm warning model — “spaghetti lines” — that you have seen on your local forecast.

These models are generated by the North American Global Forecast System (GFS) and the European Centre for Medium-Range Weather Forecasts (ECMWF). But the models make different predictions; often radically different. For example, in 2016, an early prediction of Hurricane Matthew using the North American GFS was off by nearly 325 nautical miles, while the ECMWF was extremely accurate.

The difference? The ECMWF has invested abundantly in technology and enjoy double the amount of forecast data between 2015 and 2017 alone.

U.S. Weather Forecasting Needs to Update

Improvements in the underlying infrastructure of NOAA and the NWS will directly affect the ability to improve forecasting.  Time and accuracy are critical in severe weather scenarios, while long-term modeling helps understand changes in climate.

It’s already happening in the private sector. Nearly 10 years after its original Deep Thunder project, IBM acquired the Weather Company. In October 2015, the Weather Company’s cloud-based app processed more than 26 billion data requests daily using three billion reference points for weather forecasting data. That is in addition to weather data from more than 40 million smartphones and nearly 50,000 airline flights per day.

The Consumer Weather Business is Booming

Back to the tie-in with cloud computing. Microsoft Azure — Microsoft’s cloud computing service — has partnered with numerous private weather software and technology companies. Some of these collaborations are bringing truly innovative services to consumers:

  • Wouldn’t it be helpful if Waze could alert you to dangerous winds ahead on a bridge, in addition to alerting you that Taco Bell is two minutes from your location?
  • Disney has changed the game with wearable technology, making your amusement park experience even better by adjusting to weather changes in real time.

Heading to the cloud, American forecasting models will not be behind for long.

  • Grants and funding have launched the next generation of weather satellites.
  • NOAA started 2018 with a massive upgrade to its supercomputing system. The new system will be one of the fastest in the world, processing 8 quadrillion calculations per second.
  • The total computing power of NOAA will rise to 8.4 petaflops. (Don’t worry, I had to look it up too. It’s a measure of computer performance and is in the top 20 globally.) The computing upgrades also increased storage capabilities by more than 60%.  This gives the developing American model access to more data that will improve forecasting.

The Future is in Cloud Computing

Meteorologists aren’t known for making accurate predictions, but I have one that I know is rock solid: if the American weather industry continues to adopt cloud computing, the next several years will bring unprecedented advancement in weather forecasting.

That prediction doesn’t stop at the weather industry, though. Any industry — especially more traditional fields like banking and hedge funds — can make headway using cutting-edge, cloud-based technology. If you’d like to learn how your business can benefit from cloud computing, contact Agio today.