Devising an archival plan for an OLTP(Online Transaction Processing) system may seem to involve a lot of complexions that would most often than not, make one choose a ‘play-safe strategy’ – a strategy in which one ends up having more data in the operational database than what is ideally needed. While the ‘play safe strategy’ addresses the ‘safety’ part of the problem it rarely does answer the ‘cost’ part. Ending up having more data that what is ideally required in the database, the operational and maintenance costs shoot up exponentially over time. In this article, I am going to talk more in detail about the real problem statement here and about how IBM Optim Data Growth Solution can help reduce operational and maintenance costs while at the same time ensuring data safety and availability.
The Problem
For the purpose of this problem I am classifying Data into 3 types based on the usage patterns.
1. Hot Data (typically 0-2 years old data) – operational data that is regularly used.
2. Warm Data (typically 3-6 years old data) – data that is not used as much but nevertheless needed within an acceptable time on demand.
3. Cold Data (typically 6+ years old data) – the data that is retained for historical maintenance, compliance, and auditing purposes.
Typically one always need to have their hot data in their operational system. Also usually the cold data is moved out to offline stores like magnetic devices or tapes. In the absence of a good archival solution, one ends up storing the warm data as well in their operational database. While ‘hot’ and ‘warm’ data may look so much alike, they are very much different. The warm data is read very less frequently, never grows and is never updated. Based on these traits one can easily see that warm data alone requires very less maintenance, doesn’t require costly high speed storages, not much CPU to serve the data requests etc. But by clubbing warm data with the hot data we are effectively increasing operational and maintenance costs of the operational database by approximately 200%. So the question here is – how do I keep my warm data separately in a place where I can access it instantaneously whenever required but yet away from my hot data?
How does the IBM Optim Data Growth Solution Help?
One of the many things that the IBM Optim lets the user do so easily is to extract relational data logically and consistently from pretty much any type of database, create a database archive using the extracted data and letting the user access the archive file in the same way he would access an operational database (using JDBC, ODBC, SQL, XML etc..). Now let us try and solve the problem of storing the warm data discussed in the above section using IBM Optim. By taking the warm data out periodically and storing it in archives using IBM Optim we can easily achieve the following objectives,
1. Store only ‘hot’ data in the operational data store. Benefits: a. Reduced maintenance and operational costs on operational database b. Reduced storage costs as only ‘hot’ data is stored in high speed storages
2. Store the ‘warm data’ as archives and access them using standard access methods like JDBC, ODBC, SQL etc. Benefits: a. Instant access to warm data guaranteed. b. Reduce storage costs for warm data by using less expensive storage and hardware. c. Almost nil maintenance and operational cost for warm data given its static nature
3. Once the ‘warm’ archived data becomes ‘cold’, move them to less expensive storages like magnetic tapes. Benefits: a. Easily move out cold data as and when it gets ‘cold’ b. Maintain regulation and audit compliance by option to selectively restore required data from tapes and use them on request basis c. Reduce storage costs for ‘cold’ data
Why IBM Optim?
IBM Optim is a market leader in its space as reported by Gartner with 46% market share. Also Forrester Research valued IBM Optim highly based on the customer feedback it got. Personally, more than the huge width and breadth of its capabilities, I am very much impressed by the ease in which anyone – even a business user – can so easily use it. Not to forget the reliability and comfort in using such a trusted archival solution.
DISCLAIMER: The case study I have based my blog out is that of a simple and typical OLTP system. The mentioned quantifications of costs may vary largely on a case by case basis.
Click here to know more about how IBM Optim Integrated Data Management solutions can do for you.
by Vishnu V Leelakrishnan
