By switching from Digital Linear Tape to Linear Tape-Open library, researchers on the TerraServer Web site now perform tape backups in one third the time and use one third as many tapes.
Not long after researchers on TerraServer Web site project, a site that serves up aerial and topographic maps of the United States, installed a storage-area network, they realized their aging StorageTek 9710 tape library with its Digital Linear Tape (DLT) technology couldn't keep up in the new environment.
TerraServer's three databases, which total more than 4T bytes, contain more than 300 million aerial photographs and topographic maps provided by the United States Geological Survey (USGS). On a typical day, 50,000 unique users visit TerraServer and retrieve an average of 3.8 million images, making it one of the busiest sites on the Internet. The three databases are backed up entirely every weekend.
DLT is not a particularly fast media, and it's prone to errors," says Tom Barclay, lead developer on the TerraServer project and a researcher in Microsoft's Bay Area Research Center, which runs the site.
The researchers opted to replace the StorageTek 9710 with an ADIC Scalar 1000 tape library with four tape drives and 188 tape slots. The Scalar 1000 is based on Linear Tape-Open technology using media that is capable of storing more data, faster and with greater reliability than DLT.
LTO media can natively store 100G bytes of data or 200G bytes when the data is compressed. "We are able to backup a one-plus terabyte database to 12 tapes for a single backup versus 40," Barclay says. "In addition, we found that the LTO drives can backup data at a rate of 9.8M bytes per second per drive versus 4.7M bytes of data per second per drive on the DLT drives."
The added payoff is a smaller footprint in the data center. The researchers were able to swap a five-foot-long robot containing more than 500 tapes for one that sits in a 19-inch rack containing 188 tapes. "Before installing the Scalar 1000, it was actually looking like our tape storage was going to exceed the size of physical disk storage, and that's crazy," Barclay says.
Veritas Software Corp.'s NetBackup provides the developers with a graphical display of the Scalar 1000 tape robot's physical structure. To schedule a backup, a system administrator simply drags files to a tape device.
The Scalar 1000 also is capable of automatically checking the viability of connections within the SAN and data-path conditioning, which reduces the chance of backup interruptions and improves drive performance through extra error correction and command queuing. "The bottom line is much better performance and reliability with the Scalar 1000," Barclay says.
Barclay designed a highly redundant SAN to ensure the site's databases would be available even during a storage media failure or some other calamity. "We're paranoid like anyone doing e-commerce, and we triple mirror the disks," Barclay says.
Although TerraServer is not a typical e-commerce site, its information infrastructure is designed as if it were, Barclay says. The site's foundation is a SAN running Microsoft SQL Server 2000 on Windows 2000 Data Center Edition. The hardware consists of a cluster of four Compaq ProLiant 8500s, each with eight-way processors and 4G bytes RAM, and four Compaq StorageWorks ESA 12000 subsystems with a total of 18T bytes of disk storage.
The TerraServer site's I/O activity is similar to that of an electronic store that sells an item for every five product impressions, Barclay says. USGS provides the site with a steady stream of new and updated imagery. The site feeds daily 120G bytes of imagery into its databases, while executing an average of 1.6 million insert and delete transactions.
Downtime, even for a free site, is no longer an option, Barclay says. "On the Internet, you have to be up 24 hours a day, seven days a week. Realistically, no one takes their system down any more."
For more information on TerraServer, visit the site.
To learn more about ADIC, visit its Web site.
To get more information about Veritas, visit its Web site.
For More Information:
>>SearchStorage.com Best Web Links: Tape
>>SearchStorage.com Discussion Forums: Storage Networking
>>SearchStorage.com Best Web Links: Storage Area Network (SAN)
This was first published in January 2002