If you’ve decided that your data backup system can benefit from data deduplication, you definitely have plenty of choices. But first you need to figure out where and how to implement dedupe. There are several data backup products that incorporate data deduplication. Some are virtual tape library (VTL) products, others are network-attached storage (NAS) that may be used as a backup target, and still others are backup applications.

In this tutorial, we look at post-processing versus inline deduplication, disk-based backup and dedupe, and compare the popular deduplication products.

A look at post-processing deduplication

For backup/storage administrators looking minimize the time it takes to back up their data, the best option is often to use a post-process method. This has the advantage of backing up data faster, reducing the backup window. The disadvantage of this method is that additional storage space is consumed. Backup data is sent to a temporary holding area in order to speed the backup process. Once that completes, the data is reexamined for duplicates, with duplicate data removed (some post systems start deduping before the whole backup is complete, so they may not require as much storage on the target).

Editor’s Tip:

It’s also important to know how data dedupe impacts the recovery process — specifically, how rapidly you can recall data for restoration.

Inline deduplication

An alternative to deduplicating data after a backup is to perform deduplication inline as data is being sent to the backup device. The advantage with this method is that no extra space is required. Another advantage is that once the data is deduplicated and stored, the process is done, and backup data may be replicated to offsite storage. With post-processing deduplication, data must be written to storage, then deduplicated at a later time, and then replicated to offsite storage. As a result, the time to complete the entire backup process — including replicating to offsite systems — can be longer than systems that deduplicate inline.

Editor’s Tip:

There are several kinds of backup products that incorporate data deduplication, and VTLs are one example.

Disk-based backup systems and deduplication

Data deduplication can dramatically decrease the amount of disk space required for backup data, while retaining the significant performance improvements that disk-based backup devices have over tape. Thus, disk-based backup targets, whether they are NAS devices or VTLs, allow these systems to deliver high service-level objectives, while remaining cost competitive with tape-based systems.

Editor’s Tip:

2008 was a big year for disk-based backup and recovery.

A comparison of deduplication product offerings

There are several vendors that deliver products that incorporate data deduplication. Provided below is a comparison of vendors, products and features.

The future of data dedupe

It is likely that over time, data deduplication will become a service and be offered as a feature in conjunction with multiple product types and deployment scenarios. Until this time, you must carefully evaluate their cost, performance and data retention goals prior to choosing a data deduplication product that will deliver the optimal benefits in their particular environment, or test the product carefully in your environment before you buy it.

IBM Corp., Microsoft Corp., Symantec Corp. and others have recently announced new cloud data storage services. These come on the heels of earlier announcements by companies including EMC Corp., Iron Mountain and Seagate Technology, to name a few. Despite pronouncements by some that cloud backup isn’t just marketing hype, serious doubts remain.

The good news is that to a limited degree, cloud storage is real and can save you money, when used appropriately. However, cloud storage isn’t a magic bullet, and there are several issues to consider.

In this tutorial on cloud backup, you will learn the pros and cons cloud backup, how cloud storage is changing data protection and disaster recovery and the differences in the cost of cloud backup vs. traditional backup.

CLOUD BACKUP TUTORIAL

What is cloud storage?

Cloud storage means different things to different people. For some, it means easy access to secure, inexpensive storage from anywhere. In some cases, these are the same promises provided by past storage buzzwords, many of which failed to live up to their hype. During the last decade we have all witnessed the promise of the Storage Service Provider (SSP), then grid storage, then utility storage as being the next technology that solves all our problems. Here we are today with the next banner being waved, claiming that cloud storage is the answer.

There are several companies providing different services, all using the term “cloud storage,” although they differ in many respects. The majority of these offerings are a variation of online backup services. Although there are other services being offered using the term “cloud storage,” many of these are not yet practical.

Where to use cloud storage

Mostbusiness critical applications will continue to depend on access to high-speed storage. For these applications, any delay in access can significantly impact their performance. Additionally, network bandwidth over long distances will always cost significantly more than connections within the data center. Due to the laws of physics.

Long-distance communications take longer, and hence cost more than shorter connections. For these reasons it is impractical to consider cloud storage for many business critical data processing applications.

So, if cloud storage is practical, it will be used primarily for applications that don’t depend on high-speed, high-bandwidth connections to storage.

This leaves open data sharing and data protection/data backup as the two most likely uses for cloud storage. Typically data sharing and protection are not as time critical as transaction processing. You are more likely to wait one minute to see a picture of your niece uploaded by your sister than you are to wait one minute to bid on a rare item on eBay.

According to major backup vendors, more than 75% of all data restore requests are for a single file, less than 30 days old. Thus, the issue of network latency will not be an issue for the vast majority of file restores. Restoring a few files over a wide-area network (WAN) will typically happen nearly as fast as restoring local files.

Also, a large and growing share of data resides outside of traditional data centers on a variety of devices including distributed on desktop and laptop computers. The exact percentages vary with the type of company, size and location, but one thing is clear — the data being stored outside of data centers is an important issue.

Data protection and disaster recovery in the cloud

Conceptually, there is a large difference between protecting data, and providing disaster recovery. However, the way these capabilities are delivered is often very similar. Conventional data backup and recovery applications typically also offer a disaster recovery option, sometimes referred to as “bare-metal restore.” However, disaster recovery involves much more than simply restoring an application server. Ensuring that the data is protected from natural and human disasters is an important part, which requires off-site storage of data.
For home users and SMBs, DR is less of a focus than is data protection. If a hard drive crashes, or a laptop is lost or stolen, the primary concern is being able to get your data back.

Also, many companies also choose to ignore the problem of distributed data by ignoring it, or by instructing users to “Back up their data to a network drive regularly.” Those of us who have lost data probably do this, sometimes.

Both of these problems can be addressed with the current crop of cloud storage backup offerings available. By providing automated backup, data is actually backed up. By doing so to a remote site, a moderate degree of disaster recovery is also provided. The data is protected from local disasters, including fire, theft, floods, tornados and human errors by maintaining copies of files safely and securely off-site.

Cloud backup vendors

There are an increasing number of cloud storage (or online backup) services available. Currently there are approximately 40 online/cloud storage offerings available in the U.S. and Europe, with more coming online every month. Many of these are designed for home and small office users, although there are more than 15 offerings designed for corporate users as well.

The services designed for enterprises typically offer more features and control, along with a correspondingly higher price. If enterprise control is not required, and you have a lot of data to back up, one of the unlimited backup services such as Backblaze, Carbonite Inc., ElephantDrive, Mozy or others may be your lowest cost option.

Microsoft is even getting into the act. Microsoft has a Windows Home Server, which is really like a place to store information in your home. A Windows Home Server is also able to back up this data to another drive connected to the system. But if you want to backup your important data off-site for DR, you can set up your Windows Home Server to back up your data to the Amazon S3 service, which is a commercial cloud storage service offering from Amazon. So, for home users who want the ultimate protection from floods and fires, the cloud is the answer.

Cloud backup vs. traditional backup

When considering using a cloud storage provider for backing up data, it is important to look at a few factors. These are the amount of data you have to protect, the amount of available internet bandwidth and the amount of data that changes on a regular basis. The table below compares cloud backup vs. traditional backup.

Cloud backup cost comparisons

There are a lot of cloud backup options available, and nearly as many pricing models as there are services. One of the cloud storage offerings that has received a lot of press is Amazon’s Simple Storage Service (S3) service. This service is open to nearly anyone, with a set price for storing, uploading and downloading data.

Not surprisingly, many of the cloud/online backup providers use Amazon as the back-end for their service. Although the Amazon S3 service is not necessarily the least expensive way to back up a lot of desktop or laptop systems, particularly if you have a lot of data to protect. In that case, a service that does not charge by the gigabyte is often a better option. Some of the lowest cost options cost between $50 and $60 per year, which is equivalent to storing approximately 30 GB on Amazon’s S3. So, if you are storing more than 30 GB, an unlimited account will save you money.

One potential issue is archive vs. backup. Archiving allows users to free up space by moving data off the primary system. However, several of the online backup applications are designed exclusively for backup. So once you remove the data from your primary system, it gets deleted from the online backup as well. If you want to store copies of your home movies for safe-keeping, it may be very expensive to store that 1 TB on Amazon S3. Expect to pay $100 to move it to the site, and then another $150 per month to store it. For archiving large amounts of data, the best option remains storing data on tape, or a removable disk drive for smaller environments.

The chart below compares the prices of cloud storage vs. traditional backup.

Prices of cloud storage cost vs. traditional backup

Cloud storage is real, with services available from a number of companies. These services should be considered for any environment that does not already have a data protection plan in place. Many home users and SMBs rely on small systems for all their data, although they may not have an adequate (or any) means of protecting this data. Even many large companies who until now have chosen to ignore distributed data on laptops should consider using the cloud backup.

A removable disk drive developed by Iomega Corporation. The Jaz drive has a 12-ms average seek time and a transfer rate of 5.5 Mbps. The removable cartridges hold up to 2 GB of data. The fast data rates and large storage capacity make it a viable alternative for backup storage as well as everyday use.

A high-capacity floppy disk drive developed by Iomega Corporation. Zip disks are slightly larger than conventional floppy disks, and about twice as thick. They can hold 100 or 250 MB of data. Because they’re relatively inexpensive and durable, they have become a popular media for backing up hard disks and for transporting large files.

A device, like a tape recorder, that reads data from and writes it onto a tape. Tape drive has data capacities of anywhere from a few hundred kilobytes to several gigabytes. Their transfer speeds also vary considerably. Fast tape drives can transfer as much as 20 MB (megabytes) per second.

The disadvantage of tape drives is that they are sequential-access devices, which means that to read any particular block of data, you need to read all the preceding blocks. This makes them much too slow for general-purpose storage operations. However, they are the least expensive media for making backups.

A Compact Flash card is an IC (Integrated Circuit) which is stored in a compact and rugged plastic enclosure. Compact Flash cards are designed to store data and to enable the transfer of data between devices equipped with Compact Flash slots.

Current Compact Flash capacities range up to 4GB.

Compact Flash Type I cards are 43mm long, 36mm wide, and 3.3mm thick.

Compact Flash Type II cards are 43mm long, 36mm wide, and 5mm thick.

The theoretical transfer speed of Compact Flash 2.0 is 16MB/sec.

The Compact Flash standard was introduced by SanDisk Corporation in 1994.

Compact Flash Plus (CF+)

Compact Flash Plus (CF+) extends Compact Flash to provide functionality such as micro hard drives, modems, Ethernet cards, 802.11 Wi-Fi cards, serial cards, Bluetooth cards, and more.

This makes Compact Flash the most versatile of the flash media formats.

SD card, short for Secure Digital card, is a highly-sophisticated and non-volatile memory card format developed by Matsushita, SanDisk, and Toshiba for use in portable devices. It is available in standard and high-capacity formats along with a variety of speed classes. Today it is widely used in digital cameras, handheld computers, PDAs, mobile phones, GPS receivers, and video game consoles. Standard SD card capacities range from 4 MB to 4 GB, and for high capacity SDHC cards from 4 GB to 32 GB as of 2008. Announced at the 2009 CES a new specification (SDXC) (eXtended Capacity) will allow for 2TB capacity cards. For example, A camera that stores images digitally rather than recording them on film. Once a picture has been taken, it can be downloaded to a computer system, and then manipulated with a graphics program and printed. And SD card in camera is used for image storage.

Besides, using decoder software, an SD compatible device can play music, video clips and more without a drive mechanism like those in CD and DVD players. This allows the equipment to be more compact and gives product designers new creative freedom. Reliability is also improved through the elimination of skipping on music and video files.

A Memory Stick is an IC (Integrated Circuit) which is stored in a compact and rugged plastic enclosure. Memory Sticks are designed to store data and to enable the transfer of data between devices equipped with Memory Stick slots.

The Memory Stick standard was introduced by Sony in October of 1998.

Current Memory Stick capacities range up to 512MB.

A Memory Stick is 50mm long, 21.5mm wide, and 2.8mm thick.

An even more compact format, Memory Stick Duo, is 32mm long, 20mm wide, and 1.6mm thick.

The theoretical transfer speed of Memory Stick is 160Mbps.

A small, portable flash memory card that plugs into a computer USB port and functions as a portable hard drive with up to 2 GB of storage capacity. USB flash drives are touted as being easy-to-use as they are small enough to be carried in a pocket and can plug into any computer with a USB drive. USB flash drives have less storage capacity than an external hard drive, but they are smaller and more durable because they do not contain any internal moving parts.

USB flash drives also are called pen drives, key drives or simply USB drives.

A soft magnetic disk. Unlike most hard disks, floppy disks (often called floppies or diskettes) are portable, because you can remove them from a disk drive. Disk drives for floppy disks are called floppy drives. Floppy disks are slower to access than hard disks and have less storage capacity, but they are much less expensive. And most importantly, they are portable.

Floppies come in three basic sizes:

8-inch: The first floppy disk design, invented by IBM in the late 1960s and used in the early 1970s as first a read-only format and then as a read-write format. The typical desktop/laptop computer does not use the 8-inch floppy disk.

5-inch: The common size for PCs made before 1987 and the predecessor to the 8-inch floppy disk. This type of floppy is generally capable of storing between 100 KB and 1.2 MB (megabytes) of data. The most common sizes are 360 KB and 1.2 MB.

3-inch: Floppy is something of a misnomer for these disks, as they are encased in a rigid envelope. Despite their small size, microfloppies have a larger storage capacity than their cousins – from 400 KB to 1.4 MB of data. The most common sizes for PCs are 720 KB (double-density) and 1.44 MB (high-density). Macintoshes support disks of 400 KB, 800 KB, and 1.2 MB.