Learning from Hurricanes: Big Data Analytics, Risk, & Data Visualization
This year, Florida has experienced its 10th consecutive year without a hurricane. It is the longest period without a hurricane strike in modern times and one more remarkable considering that Florida’s more then 1200 miles of coastline account for about 40% of the US landed hurricanes recorded in modern history.
Exploring this long stretch without hurricanes is worthy of some examination, as it offers us many lessons in Big Data Analytics, Risk, and Data Visualization. First, the obvious: how frequent are hurricanes and are hurricanes regular in their arrival? The below graphic from the WSJ of last year nicely shows this.
Indeed, the graph shows that in some decades such as the 1910s, 1920s and 1940s, hurricanes were quite frequent in Florida (nearly annual!). Interestingly, the frequency of hurricanes is less in recent decades, except for a major spat of hurricanes in 2004 and 2005. All of this raises questions that are of great interest to climatologists, disaster recovery planners, risk managers, insurers, and re-insurers. Is the irregular arrival of hurricanes just a manifestation of randomness?
It might be a product of climate change, global warming, or simply a level of variation in the natural cycle not seen before. Indeed, hurricane patterns are complex, and we are rapidly learning more about their formation and occurrence, thanks to improved data collection and analytics. In recent years, climatologists have been able to zero in on factors that are more predictive of high hurricane activity years. The below graphic from NOAA communicates some of the most important factors in a high hurricane occurrence year.
A large number of hurricanes are expected when there is high pressure in Northwest Africa, warm temperature in the Atlantic, and favorable trade winds. That is a complex interaction of variables. And, El Niño is generally shown to result in less of these conditions and less therefore hurricanes. Such insight is valuable to a risk manager and risk insurer. From a risk management perspective, knowing about this beforehand allows for more appropriate risk taking, preparation, and investment. Indeed, owning hurricane insurance risk in Florida over the past few years turned out to be a rather nice investment.
Having grown up in Tampa, Florida, I was acutely aware of the dangers and damages from hurricanes – at least I had heard about hurricanes from my grandparents. In some 22 years in Tampa, I saw only one hurricane come by Tampa in 1985. The interesting phenomenon was that the west coast of Florida had seen many hurricanes in the 1910s and 1920s and then a scrap with a category 5 hurricane in 1960. This irregularity in hurricane arrivals perplexed me. I can recall fishing in the inter-coastal way and seeing passes and breaks formed by hurricanes from the past. Why were there less hurricanes in the 1970s and 1980s than in previous decades? Or why were there more in the past? Did something change?
Risk Management Lessons
This phenomenon interested me so much that I explored it as part of my PhD Dissertation at Cornell University. At the time, we did not have the big data tools of today. In particular, I examined if hurricanes and other large flood events were indeed irregular in their arrival. I found that for the southeastern US, large annual flood events are statistically “clustered in time.” That is to stay that some periods of time show many large annual floods and then there are extended periods of time (many decades) with little to no large floods at all. It is a major finding that challenges the principle assumptions of catastrophic risk analysis. It suggests that risk is dynamic and the underlying assumptions subject to changing conditions.
If the 100-year flood comes on average once every 100 years and it has been seen two times in the last 10 years, it also might mean that a long period of tranquility is ahead. That can potentially be exploited by insurers in issuing insurance during low risk periods. The recent Florida hurricane data suggest that such changes are indeed part of the climate.
Key Point: Risk models are simplifications of the real world. With more data, we can explore, understand, and account for relationships across many variables. Big data analytics is changing how we examine risk, not just in climate, but in finance and healthcare, for instance. Deploy Big Data analytics to leverage large scale and multi-variable data sets to understand risk more precisely.
Data Visualization Lessons: Risk is Dynamic and Complex
I came across a great graphic made by John Nelson of IDVSolutions. 
It graphs hurricanes and tropical storms since 1851 and uses colors (more green is more severe) to show the severity of hurricanes and the progression of the hurricanes along their tracks. With little explanation or climatic training, you can easily see some interesting things about hurricanes in the Gulf of Mexico and Western Atlantic Ocean. At least for me, I see a rather suspicious blank space in west Florida – an indication of less or at less severe hurricanes than other parts of the state and the Gulf Coast. One explanation is good luck; another is more physical in that hurricanes lose strength over land. It is hard to hit the west coast of Florida without hitting some land first. So, the west coast of Florida, may, in particular, be more protected. It might be useful in selecting risks. Buy hurricane risk on the west cost of Florida over the Miami area.
The next observation of this impressive data visualization by IDV Solutions is that the strongest hurricanes do in fact avoid land in their formation, riding through the Florida Straits, skirting south of Cuba, and otherwise strengthening in the Gulf of Mexico. And if you ever thought that New Orleans and the Louisiana Gulf Coast gets more strong hurricanes that elsewhere, this graphic would support your hypothesis. This is a great example of a data visualization that allows for the communication of hurricane tracks, relative strength, and geographic occurrence. It would have been great to have this when we looked at hurricane and flood data some years ago. It is a great example of how data visualization is changing analysis. In a few minutes, complex relationship between location, direction, intensity, and reporting can be understood. It is also a great example of why we need data visualization as part of our analytical and risk toolbox.
We, as humans, cannot easily process complexity in numbers. However, we (or some of us in particular) are quite good at addressing and processing complexity expressed in shapes, colors, and graphics. This strength and weakness of our cognitive skills requires that we be mindful of how to use data visualization as part of an analytical strategy. It makes sense and I am reminded of this every time I park on the purple level of the parking garage at Northwestern University. I can’t as easily recall the number of the level at the garage, but the colors of each level are crystal clear to me in memory, and yet I use the garage nearly everyday. Numbers although necessary for analytics, are not the best form for our cognitive processing. Relative comparison and rate changes are more easily understood through graphics.
Key point: Leverage data visualization to understand and explore complex relationships across many variables in data. Leverage the human mind to look for patterns and ask interesting questions of the graphics. It leverages the best of graphics and our cognitive skills.