Redundant, obsolete, trivial – déjà vu all over again

A few years ago, a lot of software industry folks (including myself) bet big-time that organizations would invest heavily in the Redundancy, Obsolete and Trivial (ROT) data management use case. 

Simply, the ROT use case involved acquiring first-generation file analytics software to identify and eliminate duplicates, aged out content and non-relevant information from repositories across the enterprise. The benefit was the lower cost of operations and measurable hard dollar cost savings. Savings were calculated per gigabyte and reflected not only actual reduced storage capacity but the operational, compliance and process costs of managing content on-premises. 

The Enterprise Strategy Group, a Boston-based consulting firm with vast expertise in storage management, calculated that the average regulated enterprise was paying up to or exceeding costs of $25 a gigabyte on-premises. So, if you had terabytes or petabytes of redundant, obsolete or trivial data sitting out there in on-premises repositories, file shares and SharePoint, organizations are spending well in excess of millions of dollars to manage data. Eliminating a percentage of that volume saves money… Easy-peasy, right?

Well, the market had a very different idea. Almost universally, organizations said no to the value proposition. First, a lot of organizations were initially exploring moving content to the cloud. And, while it sounded easy, it turned out that moving applications and application data to the cloud was more difficult than thought. Plus, other than cost savings, the ROT use case didn’t present a business-compelling enough reason to invest in the required technology and services. 

Most important, cost reduction seemed to compete with IT initiatives designed to grow revenue. As a result, the ROT use case sunk to a much lower rung on the priority ladder. It also turned out that a pure ROT use case was highly complicated and, in many cases, unable to generate a stand-alone ROI that was worth the effort. As a result, most organizations balked. So, the ROT use case, as a stand-alone cost savings activity, died quietly and without enterprise remorse.

Fast forward to the present day, the ROT use case is coming back into vogue. Eliminating ROT is fundamental to meeting legal requirements from newly evolving global and local privacy legislation, the need to reduce discovery costs and the overt compliance risk that sensitive and high-value information present to the enterprise. Increasing maturity in cyber security mitigation plans also seeks to reduce the surface area of threats to the business if there was a breach.

If you study the General Data Protection Requirement, the California Consumer Privacy Act and similar laws, you will see that there are fines for non-compliance. Those fines can be substantial. However, the real business risk for non-compliance is the potential hit to corporate brand reputation and impact on market share. Essentially, all privacy laws include a similar mandate to delete data that is no longer relevant to an active business process. If you are storing personal customer data for a customer who is no longer your customer, you are in violation of the law, unless, of course, you delete it, or retain it for a legally specified period.  This means you not only have to find it, but you also have to determine, once discovered, how you must keep it. And you also need validated deletion processes governed by policy across the enterprise.

Previously, discovering relevant data for deletion and then determining what to keep has been very difficult and requires high levels of expertise and consultancy to create multi-level data management business processes.  

However, with the advent of artificial intelligence and machine learning, there is a new focus on the O (obsolete) and T (trivial). There is also renewed focus on the more difficult to solve R (redundant) part of the equation. That’s because the correct data version needs to be first discovered and then managed correctly. Think of content that is broadcast across the enterprise – what is original and what is a copy – is sometimes very difficult to determine. 

The bottom line is that organizations now must become very adept at discovering, analyzing, retaining and disposing ROT data from every corner of their business. It’s not about saving money; it’s about lowering risk to protect brand and business reputation and keep customers from leaving because they don’t trust or have faith in your data privacy practices. It’s also about meeting the constructs of multi-jurisdictional privacy laws that have more variations than a complex Bach fugue. Your team also benefits by not having as much content to wade through when they are looking for answers.

The lesson learned is that sometimes a use case is not compelling enough unless it can be attached to a larger initiative that has more weight and relevance to an enterprise IT and business strategy. This is certainly the situation with ROT as its focus is now directly related to broader and more significant uses related to compliance, cost-saving, and risk reduction. Feel free to continue the discussion by reaching out to me directly at [email protected] – I would love to hear your thoughts.