There is no debating that duplication is one of the hottest topics in IT. The question is if the hype has started to become bigger than the technology. Today, there are two primary use cases driving deduplication in the marketplace. The first is backup to disk and the second is virtual guest operating systems (VMware, Hyper-V, and Xen guests). (I will talk a bit about the disk to disk scenario in this article and the virtual guest topic in the next one.) These are both logical markets to adopt deduplication because they suffer from a common challenge. They both create a tremendous amount of redundant data on the disk array. The goal in both cases is to pack more data onto a disk drive and reduce the cost per GB. This is the first and most obvious use case for deduplication.
Disk drive capacity is growing exponentially, but disk performance is increasing at a much slower rate. In many cases, when helping customers size for their workload, performance drives the spindle count and not capacity. It is easy to meet the capacity needs with large drives, but will they meet the performance requirement? That is the problem. Often performance is what dictates the spindle count. It is no longer sufficient to size a storage device based solely on capacity requirements. This is a general challenge that must be taken into account when sizing a storage array.
Read more…
How many times have you wondered whether you could improve the performance of your storage array by adding additional cache?
Will more cache improve the performance of my storage array? This is what the vendors so often tell us, but they have no objective information to explain why it is going to help. Depending on the workload, increasing the cache may have little or no effect on performance.
There are two ways to know whether your environment will benefit from additional cache. The first is to understand every nuance of your application. Most storage managers I speak with classify this as impractical at best and impossible at worst. Even if you have an application with a very well understood workload, most storage devices are not hosting a single application. Instead, they are the hosting many different applications. It is even more complex to understand how this combined workload will be effected by adding cache.
The second way to measure cache benefit is to put the cache in and see what happens. This is the most common approach I see in the field. When performance becomes unacceptable, the options of adding additional disk and/or cache are weighed and a purchase is made. (I will save the topic of adding spindles to increase performance for a future post.) Both of these options force a purchase to be made with no guarantee it will solve the problem.
NetApp has introduced a tool to provide a 3rd option: Predictive Cache Statistics. It provides the objective data needed to rationalize a hardware purchase. Predictive Cache Statistics (PCS) is available in systems running 7.3+ and having at least 2GB of memory. When it is enabled, PCS reports what the cache hit ratio would be if the system had 2x (ec0), 4x (ec1), and 8x (ec2) the current cache footprint. (ec0, ec1, and ec2 are the names of the extended caches when the stats are presented by the NetApp system.)
Now, let’s drill down into exactly how predictive cache statistics work…
Read more…
As the need for disaster recovery continues to move downmarket from the enterprise to medium and small businesses, the number of IT shops replicating their data to an offsite location is increasing. Array based replication was once a feature reserved for the big budgets of the Fortune 1000. Today, array based replication is a feature that is available on most midrange storage devices (and even some of the entry level products).
This increase in replication deployments has created a new challenge for IT. The most common replication solutions move the data over the IP network. That data puts a significant load on the IP network infrastructure. The LAN infrastructure is almost always up to the task, but the WAN is often not able to handle this new burden. While the prices of network infrastructure have come down over the years, big pipes are still an expensive monthly outlay. So, how do we get that data offsite without driving up those WAN costs? WAN optimization technology provides a potential solution.
Not every workload or protocol can benefit from today’s WAN optimization technology, but replication is one that usually gets a big boost. I gathered some data from a client who is using NetApp SnapMirror to replicate to a remote datacenter and deployed WAN optimization to prevent a major WAN upgrade.
Read more…
With the release of the Sun Storage 7000 line of storage appliances, Sun has included a new “Analytics” toolkit. These analytics are based on DTrace (http://en.wikipedia.org/wiki/DTrace), but essentially hide the DTrace complexity in a cloak of Ajax-based browser graphics. Through the GUI, a storage administrator can determine which clients are causing which files on the server to be “hot”, or resource use-intensive. Also the administrator can see the latency of each request to the blocks of that file, or how many request of each protocol are being processed, or how many cache hits a file had. In this blog I’ll explore the basics of Analytics.
The analytics component of the Sun Storage 7000 line can provide useful information to a storage administrator who is trying to manage and monitor the appliance and the files and blocks stored there. Just like DTrace, the analytics run in real time, and allow quick progression from hypothesis through data gathering to new hypothesis, data and conclusions. Unlike DTrace, the analytics component has a very complete and useful graphical interface and visualization engine.
Read more…
Hierarchical Storage Managementement (HSM), Information Lifecycle Management (ILM), and Data Lifecycle Management (DLM). Everyone wants to manage their data intelligently to reduce their spending on storage infrastructure. The storage vendors and the trade rags would like to convince us that there are magic tools to solve this challenge. The truth is there is no magic tool to manage unstructured data. (I am not talking about the archiving tools that integrate with application here, I am only talking about unstructured data.) I have tried many tools over the years and they are simply not cost effective. Don’t panic though, in most cases, the solution is far simpler and far less expensive than HSM.
File services is a huge consumer of storage capacity. For the purposes of this conversation, let’s consider file services as NFS or CIFS storage whether they be integrated appliances or a servers leveraging back end storage devices. In most environments I visit, the file serving infrastructure is using tier 1 disk drives (fibre channel, SCSI, or SAS). These disk drives are populated with data that is mostly idle and the storage managers want to get that idle data onto a less expensive disk tier. The most common request is to transparently move the idle data to a SATA based devices.
Let’s walk through this the scenarios for an environment with 20TB of unstructured data.
Read more…
My October 2008 column has been published in ;login:. This month it’s about Solaris System Analysis – detailed steps to take to determine why a system is “slow” or “busted”. Some ;login: contents is freely available at ;login:, but my column this month is not one of them. I’ve posted the .pdf here for those without a USENIX membership (although I strongly recommend you get one if you are interested in all things Unix).
The wiki that started with my August 2008 column will be expanded (as soon as I get the time) to include this new content. It’s very lonely having a wiki of one, so please consider contributing your thoughts to what I’ve started. It would be a great advance in systems administration if there was a canonocal source of first-step debugging information, and hopefully you will help make this wiki that source: http://wiki.sage.org/bin/view/Main/AllThingsSun
My August 2008 column has been published in ;login:. This month it’s about Solaris System Analysis – a checklist approach to solving a system being “slow” or “busted”. Some ;login: contents is freely available at ;login: August 2008, but my column this month is not one of them. I’ve posted the .pdf here for those without a USENIX membership (although I strongly recommend you get one if you are interested in all things Unix).
I hope this column will turn into a living wiki about (Solaris) system analysis. I’ve prepopulated a wiki with the contents of the column, so now it’s up to you to add your thoughts to the procedure. It would be a great advance in systems administration if there was a canonocal source of first-step debugging information, and hopefully you will help make this wiki that source: http://wiki.sage.org/bin/view/Main/AllThingsSun