Could Big Data analytics pore through through seemingly minor news items to pick up early hints of a looming disaster? Data scientists at Microsoft and the Technion-Israel Institute of Technology believe that Big Data analytics can do exactly that. For example, small news squibs pointing to drought in the region around Angola might have provided advanced warning of cholera outbreaks in that country.

Thanks to Big Daa analytics, our ability to recognize subtle patterns is growing. And we are only beginning to discover the possibilities that this ability could open up for us.

The warning signs were there – if only we could have recognized them! How often do these sad words follow in the wake of a natural disaster? And the fact is that that disasters rarely come entirely without prior warning signs.

In the aftermath of disaster these warning signs are easy to detect. Highsight, we say, is 20/20. But by then of course it is too late. The great challenge remains detecting the warning signs of disaster early enough to act on them.

Now, via Brad Reed at BGR, comes word from MIT Technology Review that a team of data scientists is working on exactly that. The researchers at Microsoft and the Technion-Israel Institute of Technology are developing software that reads news headlines, and looks for patterns connecting them.

For example, reports of drought conditions in and near Angola might have escaped attention by themselves. The headlines came from a multitude of media sources, including local media that draw little notice elsewhere. And the drought stories were buried among many other small news items.

Big Data analytics, however, can detect an increase in such stories, and identify a pattern associated with subsequent cholera outbreaks. So far, the software is only being used to "predict" the past. But the researchers report a success rate of 70 to 90 percent in identifying news reportage patterns that foreshadowed major destructive events.

Along with natural disasters such as disease outbreaks, the software is also able to pick up early signs of social upheavals that have led to extensive loss of life.

The researchers note that the software remains a study project: Microsoft has no current intention to develop a commercial product. And important questions will need to be resolved, such as whether the software generates large numbers of "false positives" – predicting events that turn out to never happen.