Test your knowledge of storage.

"Take out a clean sheet of paper." 

That statement can strike fear in the hearts of students everywhere. 

Well, how about just a single question?  That shouldn't be too bad.  I'll make it multiple choice as well.  How hard can that be?  OK, here is the question....

A train is traveling west at 50 mph with 20 passengers...  Oh, hold on, wrong question.  Sorry!

Let's say we want to add the capability to perform some type of storage management function on a file.  There are lots of functions to chose from but let's pick archiving just for fun.  There is a bit that needs to be set that will indicate the file should be archived.  Got it?  Great. 

Now, here is the question:

Which one is harder?

A.  Setting the bit.     (If it is set, the file will be archived)

B.  Storing the bit.     (Saving it someplace)

C.  Interpreting the bit.  (Actually doing the archiving function)



Think about this for a minute.....














Come on, a little longer than that!














OK, did you pick answer C?  Most people think that actually performing the storage management function, such as archiving in this case, is the hardest one to do. 

WRONG!

The correct answer is A.  It turns out that setting the bit is astonishingly difficult.  In many cases, it is pretty much impossible. 

The problem is that knowing if the bit should be set and when it should be set (and when it should NOT be set) is hugely complicated. 

First of all, having to make this decision on at the file level means that potentially hundreds of thousands or even millions of files means hundreds of thousands or millions of individual decisions need to be made.  When are all those decisions made?  Once (or if) they are made, what if things change?

The current strategy of the storage industry is to pass the buck onto their customers.  They ship a GUI or some type of policy engine that don't really help.  The problem is that no one has the RIGHT information. 

It turns out that the information needed to properly set the bit is the business context.  It is when the business context changes, say to the "closed" state or something similar, that is the time the bit can be set.  Without the
business context, people are just guessing.

The next posting is going to show that this inability setup the configuration for storage management has reeked havoc on unstructured data.  It can also explain why backup is so bad.