Billions and Billions....

Managing unstructured data is one of the largest and longest outstanding problems facing computers today. It has existed in some form since the dawn of the first computers.  It is complained about by customers, written about by the press, claimed to be addressed by the latest storage product announcement, or solved by the cloud[1].  The old standby of just buying more storage to stay in front of the tsunami is no longer viable.    

Everyone seems to have an answer as well.  Some industry leaders define the problem as a capacity scaling limitation so they introduce scalable storage.  Others view it as storing too many copies, so they promote deduplication.  There are those that contend if you would only throw away everything you already use and switch to their new super-duper architecture, all will be well[2].  Yet others propose to address the problem by shoving the data out to some cloud storage provider to let them deal with it.  The majority of storage people seem to be content with simply selling users more and more storage.   No one has investigated the real root cause of the problem.

Unstructured data is a huge problem costing companies, users and investors billions of dollars, and it’s getting worse.  The quantity, value, and importance of unstructured data continue to grow resulting in not only increased costs for the storage but also effects employee productivity, legal standings, customer satisfaction, and company brand.  It has had a massive impact on backup, which is still a big problem even after nearly 50 years of development.   

The problem has made many storage technologies needlessly complex, greatly increasing required development investments.  Many of these technologies have either been rejected by the market (some several times) or have failed to reach their full potential because they were unable to address the unstructured data problem.  It has made every “storage feature” its own separate industry, forcing customers to choose separate products from separate vendors and attempt to integrate these functions into a complete solution, often with limited success.  Unfortunately, it has also made certain highly desirable features either too complex or simply impossible to implement.  Sadly, even having a basic conversation about the business requirements of managing unstructured data between the owners of the data and the IT organization is generally futile.

Why?

This blog will attempt to answer this seemingly simple question.  Stay tuned....

---

[1] The cloud is not a religion, a panacea, or some mysterious technology.  It is just using someone else’s computers.

[2] Hey look!  World hunger is now over because I’ve invented the Pop Tart!