Welcome to the Era of Unstructured Data

Welcome to the Era of Unstructured Data"Big Data" is the big buzzword. But the Big Data era is not only, or even mainly, about sheer volume of data. After all, processing power and storage capacity continue to grow steadily, and what is immense now will be modest in a few years.

Far more consequential is the growing importance of unstructured data – that is, data that does not fit the neatly organized model of conventional database records. (Such as name, address, account number, always in the same format and order, perfect for traditional relational databases.)

Even as it is today, about 80 percent of all business data is unstructured. And the proportion is almost certainly growing. Much of this "unstructured" data does in fact have a structure. For example, word processing documents have a format, and the text itself has a linguistic and grammatical structure. But it is not a standardized structure that you can simply crunch through.

And text, spreadsheets, and similar files are the easy end of unstructured data. You can at least search for keywords (or key numbers). Searching image files for relevant information is not so simple, unless text metadata is attached.

Harnessing this unstructured data for effective business benefit, therefore, is not really primarily an IT process. It ends up being mainly about human business intelligence – knowing what information matters to a company, and how to find it. Massive "knowledge management" systems, once popular in the enterprise world, turned out to be no match for the varieties of unstructured data.

Says data management expert Stephen Black: "The right way to do it is not to start with the technology, but understand what the business is about." The details of the inquiry technology matter far less than the sharpness and focus of the inquiries the technology is asked to process.

For a good example of working with unstructured data, just think about how Google's core search business works.

The more our technology progresses, the more it seems to emphasize not machine intelligence, as once imagined, but human intelligence.

The security principles set forth in industry standard ISO/IEC 27002 provide a framework for effective security, built around the cycle of Plan, Do, Check, and Act (PDCA). Many good security products are on the market, but all are designed to meet specific threats – and will not block other threats. At GRT Corp. our security philosophy is built around these words by noted security expert Dr. Bruce Schneier: "Security is not a product, but a process."