LTFS Bulk Transfer Standard Q&A

Our recent live SNIA Cloud Webcast “LTFS Bulk Transfer Standard” is now available on-demand. Thanks to all the folks who attended the live event. We did not have time to address all of the questions, so here are answers to them. If you think of additional questions, please feel free to comment on this blog.

Q. The LTFS standard seems to support shared extents between files, and by extension, deduplicated files. Is this a correct assessment, and how does it play in the bulk transfer standard?

A. The LTFS Bulk Transfer Standard supports shared extents as supported by the LTFS standard, which can transparently reduce space used by having multiple references to common data stored on tape (deduplication). This typically happens below the bulk transfer layer, by the software used to read and write the LTFS volumes. At this point, few software packages support this feature due to the wear and latency consequences of read seeks resulting from using this feature.

Q. What is the state of the standard in its lifecycle? (e.g., working group draft, public review, published, etc.)

A. The LTFS standard has been around for some time; more information can be found here at http://www.snia.org/tech_activities/standards/curr_standards/ltfs. The LTFS Bulk Transfer Standard is here at http://www.snia.org/tech_activities/publicreview#ltfsbulk, and is in public review.

Q. The standard seems to be based on the idea of moving physical tapes to the cloud. Is there a definition of a virtual LTFS image that can be moved between systems over the network?

A. Not yet, but that is a great idea we’ll be taking forward in the next versions of the proposal.

Q. One of the barriers to greater use of LTFS in the Cloud is the relative lack of enterprise grade management software that ensures that the tape media is refreshed / upgraded as it ages, that its integrity is periodically checked, that reclamation and compaction is done. It needs open standards for support in standard volume management systems as well. Until these things are in place, LTFS will be interesting largely to specialized industries like film/entertainment, seismic, and bulk transfer & bulk storage — but not about the steady-state use of tape as a true additional layer of the cloud storage hierarchy. Tape with LTFS plus proper management could fill this role — but not until the full lifecycle tape management is available and integrated.

A. The management that is always required for a physical product with a well-defined and finite lifetime is not a unique requirement of LTFS. Tape has a long history of use as a backup and archive medium, and there are a number of tape management products that are commercially available from LTO tape suppliers and independent software companies, as well as open source products. A Google search for “tape management software” will provide you with a number of alternative solutions.

Q. Do you have a list or people that sell LTFS based solutions?

A. No we don’t, but it’s a very good idea, and we’ll investigate it further.

 

New SNIA-CSI Webcast: LTFS Bulk Transfer Standard

Mark your calendar for February 10th as we conclude our Cloud Developer’s series by hosting a live Webcast on the LTFS Bulk Transfer Standard. LTFS (Linear Tape File System) technology provides compelling economics for bulk transportation of data between enterprise cloud storage.

This Webcast will provide an update on the joint work of the LTFS and Cloud Technical Working Groups on a bulk transfer standard that uses LTFS to allow for the reliable movement of bulk data in and out of the cloud, and mechanisms for verification, error handling and the management of namespaces. Register now to hear David Slik, Co-Chair of the SNIA Cloud Storage Technical Work Group, discuss:

  • LTFS standard mandate and history
  • LTFS adoption and use cases
  • LTFS bulk transfer to, from, and between clouds
  • Error handling and recovery
  • Security considerations

I’ll be hosting the event, taking your questions, and hopefully shedding some light on the importance of this standard. I hope you’ll join us.