Subject: solaris-10, zfs: how to make backup tapes?  does zfs change anything from "old" ufsdump way?

Silly question, maybe.

But I'm *TOTALLY* new to this zfs stuff.

zfs is wonderful, what with disk-crashes no problem (assuming
you have enough disks), automatic mirrors, snapshots (don't understand
them too well), and so on.

But, you still have to make (tape) backups.

Now, for more years than I can count I've been using ufsdump and
ufsrestore (sunos dump and restore?), and partitions, etc.

Doesn't this zfs change everything, what with these pools and all?

Thanks so much!

David

 

David Combs wrote:
> Subject: solaris-10, zfs: how to make backup tapes?  does zfs change anything from "old" ufsdump way?
> 
> Silly question, maybe.
> 
> But I'm *TOTALLY* new to this zfs stuff.
> 
> zfs is wonderful, what with disk-crashes no problem (assuming
> you have enough disks), automatic mirrors, snapshots (don't understand
> them too well), and so on.
> 
> But, you still have to make (tape) backups.
> 
> Now, for more years than I can count I've been using ufsdump and
> ufsrestore (sunos dump and restore?), and partitions, etc.
> 
> Doesn't this zfs change everything, what with these pools and all?
> 
> Thanks so much!
> 
> David
> 
>  
Hi David,

A Snapshot is a effectively taking an instant copy of live filesystem at 
a particular instant in time.

You can not use ufsdump/ufsrestore with ZFS, if you Google this you will 
find it has been discussed for a while. The recommendation I have seen 
is for using tar.

The premise with ZFS is that because disk is cheap, disk to disk 
archiving is the preferred method. Tape capacity is not keeping pace 
with ZFS or native disk capacity, one tape used to be able to hold the 
contents of more than one disk (hdd were 10MB and tape was 300MB) but 
now a 2TB HDD requires more than 1 tape for archive. If you factor in 
ZFS then the situation becomes worse for tape. Tape arrays can handle 
the large volumes of data but they are in the price range of 
corporations only.

David Combs wrote:
> Subject: solaris-10, zfs: how to make backup tapes?  does zfs change anything from "old" ufsdump way?
> 
> Silly question, maybe.
> 
> But I'm *TOTALLY* new to this zfs stuff.
> 
> zfs is wonderful, what with disk-crashes no problem (assuming
> you have enough disks), automatic mirrors, snapshots (don't understand
> them too well), and so on.
> 
> But, you still have to make (tape) backups.
> 
> Now, for more years than I can count I've been using ufsdump and
> ufsrestore (sunos dump and restore?), and partitions, etc.
> 
> Doesn't this zfs change everything, what with these pools and all?
> 
> Thanks so much!
> 
> David

Good question. As far as I am aware, there is not such an equivalent. 
zfs 'send' and 'receive'. But

http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide

says

"The zfs send and receive commands are not enterprise-backup solutions. 
The receive operation is an all-or-nothing event, you can get all of a 
snapshot or none of it. If you store the output of zfs send on a file or 
on tape, and that file becomes corrupted, then it will not be possible 
to receive correctly and none of the data will be recoverable. See RFE 
CR6736794. Enterprise backup solutions, as well as other copying 
methods, such as cp, tar, rsync, pax, cpio, and so on, are more 
appropriate for backup/restore than zfs send/receive."

Being a wiki, which anyone can edit, I don't know how reliable that is. 
But a decent backup/restore system seems to be lacking to me.



-- 
I respectfully request that this message is not archived by companies as
unscrupulous as 'Experts Exchange' . In case you are unaware,
'Experts Exchange'  take questions posted on the web and try to find
idiots stupid enough to pay for the answers, which were posted freely
by others. They are leeches.

Thanks!

What I'm going to do is, within your reply, insert some
more questions.  (This zfs is pretty hairy stuff -- plus
the *totally unexpected* effects of it on OTHER things I
*truly believed* I understood -- like tape-backups!


In article <EIudncyxVcOnoA_XnZ2dnUVZ8gydnZ2d@pipex.net>,
solx  <nospam@example.net> wrote:
....
>> 
>>  
>Hi David,
>
>A Snapshot is a effectively taking an instant copy of live filesystem at 

OK -- first question: I guess the whole meaning of "filesystem" has
changed, at least its "base" -- where it lives.  Before zfs, a
filesystem lived in ONE partition/slice of ONE disk.

WITH zfs, you have disks, (you have partitions/slices?  yes, I think so),
and out of one OR MORE of those one creates a zfs "pool".

Somehow, I'm guessing, sort of, you build a (ie one) "filesystem" based
in that pool?

Is that correct?   


>a particular instant in time.

And if it is correct, then where does a "snapshot" (of what?) "go"?

This "snapshot", what is it, one humongous file, with filename and
everything, so that you can give its name to, say, tar?
>
>You can not use ufsdump/ufsrestore with ZFS, if you Google this you will 
>find it has been discussed for a while. The recommendation I have seen 
>is for using tar.
>
Not to start a tar-wars thread, but GENERALLY, *which* tar do
people use for this INCREDIBLY IMPORTANT TO BE ROBUST "new" use
of tar -- a company's entire business now being backed up via
tar!

My recollection is that sun tar ain't so good (my own experience being that
it has trouble sometimes reading longish tar-files made by gnu-tar).

I've heard good things about Star.  (lotsa options, I think -- question:
can you simply use the same old cvf and xvt you use with the other two?)


>The premise with ZFS is that because disk is cheap, disk to disk 
>archiving is the preferred method. 

So what do you, take the external tape-drive and try to cram it
into a safe-deposit box?  Like, suppose there's a fire?

(No joke here -- I truly am confused by this relying on backup
onto another disk.  Don't you usually have to take the backup-media
off site somewhere?


>Tape capacity is not keeping pace 
>with ZFS or native disk capacity, one tape used to be able to hold the 
>contents of more than one disk (hdd were 10MB and tape was 300MB) but 
>now a 2TB HDD requires more than 1 tape for archive. If you factor in 
>ZFS then the situation becomes worse for tape. Tape arrays can handle 
>the large volumes of data but they are in the price range of 
>corporations only.


Oh -- you said to google (this newsgroup, right)?  Just what
words (pairs, triples) would you look for?   

THANKS SO MUCH!

(Man, I got lots to learn!)

Cheers!

David Combs wrote at 25.08.2009 06:39
> Somehow, I'm guessing, sort of, you build a (ie one) "filesystem" based
> in that pool?
> Is that correct?   

Somewhat. You have a storage pool with 'vdevs' in it. These can be disks 
(-partitions, as ZFS also works with partitions/slices) or files.

In this storage pool you can create child objects, these can be Volumes 
(container which you can access as raw & block device), or Posix file 
systems, which you can mount. Initially, the pool itself is mounted as a 
Posix file system structure.

>> This "snapshot", what is it, one humongous file, with filename and
> everything, so that you can give its name to, say, tar?
>> You can not use ufsdump/ufsrestore with ZFS, if you Google this you will 
>> find it has been discussed for a while. The recommendation I have seen 
>> is for using tar.

A snapshot is a logical view to tha data stored in the original file 
system. Initially, no space in tha pool is used for the snapshot. It is 
also mounted. You can make it visible, then change to the snapshot 
directory. You have full r/o access to all files. You can make 
incremental snapshots. You can restore the original filesystem to the 
state of a snapshot.

> Not to start a tar-wars thread, but GENERALLY, *which* tar do
> people use for this INCREDIBLY IMPORTANT TO BE ROBUST "new" use
> of tar -- a company's entire business now being backed up via
> tar!
> 
> My recollection is that sun tar ain't so good (my own experience being that
> it has trouble sometimes reading longish tar-files made by gnu-tar).

Solaris tar works quite well, see
  <http://docs.sun.com/app/docs/doc/819-5461/gbscu?l=de&a=view&q=backup>
  <http://opensolaris.org/os/community/zfs/faq/#backupsoftware>

ZFS is really well documented.

Sun Document ID is 819-5461. docs.sun.com, then use the search field.

Michael

In article <h6vpt4$p6a$1@reader1.panix.com>,
David Combs <dkcombs@panix.com> wrote:
>I've heard good things about Star.  (lotsa options, I think -- question:
>can you simply use the same old cvf and xvt you use with the other two?)

I use Joerg Schilling's star.
<URL:http://cdrecord.berlios.de/old/private/star.html>
To perform incremental backups, you need to use additional options.

>My recollection is that sun tar ain't so good (my own experience being that
>it has trouble sometimes reading longish tar-files made by gnu-tar).

You recall incorrectly. By default, GNU tar creates non-standard
tars when encountering long pathnames.

>So what do you, take the external tape-drive and try to cram it
>into a safe-deposit box?  Like, suppose there's a fire?
>
>(No joke here -- I truly am confused by this relying on backup
>onto another disk.  Don't you usually have to take the backup-media
>off site somewhere?

You can mirror your backups on off-site fileservers.

John
groenveld@acm.org

On 2009-08-24 10:03:42 +0100, Dave <foo@coo.com> said:

> "The zfs send and receive commands are not enterprise-backup solutions. 
> The receive operation is an all-or-nothing event, you can get all of a 
> snapshot or none of it. If you store the output of zfs send on a file 
> or on tape, and that file becomes corrupted, then it will not be 
> possible to receive correctly and none of the data will be recoverable. 
> See RFE CR6736794. Enterprise backup solutions, as well as other 
> copying methods, such as cp, tar, rsync, pax, cpio, and so on, are more 
> appropriate for backup/restore than zfs send/receive."
> 
> Being a wiki, which anyone can edit, I don't know how reliable that is. 
> But a decent backup/restore system seems to be lacking to me.

I think most larger organisations will be using NBU or Legato, or some 
other system like that, and were already doing do before ZFS came along 
(after all, ufsdump doesn't work for VxFS either).  I can't speak for 
NBU but Legato supports ZFS now I think,

> zfs is wonderful, what with disk-crashes no problem (assuming
> you have enough disks), automatic mirrors, snapshots (don't understand
> them too well), and so on.
>
> But, you still have to make (tape) backups.

It may be worth re-thinking your backup strategy.  ZFS snapshots can
be used to quickly recover single files/directories without hitting
the trusty tape drive.  With sufficient space for snapshots, the only
event where you'd really need to recover from tape is when the entire
ZFS pool was destroyed.  Of course, in that scenario you'd better have
a true backup somewhere.

If you have a single system and your data will fit on a single tape,
I'd just use tar.  If you're talking about a large amount of data
across multiple servers, enterprise backup software will be worth the
money.  I know NBU and TSM have no issues with ZFS.

ITguy wrote:
>> zfs is wonderful, what with disk-crashes no problem (assuming
>> you have enough disks), automatic mirrors, snapshots (don't understand
>> them too well), and so on.
>>
>> But, you still have to make (tape) backups.
> 
> It may be worth re-thinking your backup strategy.  ZFS snapshots can

Rethink with GREAT CARE.  That data is your company's livelihood and 
therefore your livelihood as well!

ZFS *sounds* wonderful.  I still would not want to use it as the *sole* 
repository of irreplaceable data.  A lightning strike on your substation 
and your whole Data Center may be toast.

Malice and stupidity are always with us!  A recent backup, stored off 
site, has saved a number of people and/or companies!  The lack of same 
has meant the end of a number of jobs and/or companies.

On Aug 25, 1:56=A0pm, Tim Bradshaw <t...@tfeb.org> wrote:
> (after all, ufsdump doesn't work for VxFS either). =A0

Yes, but that's what vxdump is for.

In article <h711l3$ek1a$1@tr22n12.aset.psu.edu>,
John D Groenveld <groenvel@cse.psu.edu> wrote:
>In article <h6vpt4$p6a$1@reader1.panix.com>,
>David Combs <dkcombs@panix.com> wrote:
>>I've heard good things about Star.  (lotsa options, I think -- question:
>>can you simply use the same old cvf and xvt you use with the other two?)
>
>I use Joerg Schilling's star.
><URL:http://cdrecord.berlios.de/old/private/star.html>
>To perform incremental backups, you need to use additional options.
>
>>My recollection is that sun tar ain't so good (my own experience being that
>>it has trouble sometimes reading longish tar-files made by gnu-tar).
>
>You recall incorrectly. By default, GNU tar creates non-standard
>tars when encountering long pathnames.
>
>>So what do you, take the external tape-drive and try to cram it
>>into a safe-deposit box?  Like, suppose there's a fire?
>>
>>(No joke here -- I truly am confused by this relying on backup
>>onto another disk.  Don't you usually have to take the backup-media
>>off site somewhere?
>
>You can mirror your backups on off-site fileservers.
>
>John
>groenveld@acm.org

100 Gigabytes?   At what cost?

  And at what SPEED?

David

In article <tLGdnWHrJc-RGgnXnZ2dnUVZ_hKdnZ2d@giganews.com>,
Richard B. Gilbert <rgilbert88@comcast.net> wrote:
>ITguy wrote:
>>> zfs is wonderful, what with disk-crashes no problem (assuming
>>> you have enough disks), automatic mirrors, snapshots (don't understand
>>> them too well), and so on.
>>>
>>> But, you still have to make (tape) backups.
>> 
>> It may be worth re-thinking your backup strategy.  ZFS snapshots can
>
>Rethink with GREAT CARE.  That data is your company's livelihood and 
>therefore your livelihood as well!
>
>ZFS *sounds* wonderful.  I still would not want to use it as the *sole* 
>repository of irreplaceable data.  A lightning strike on your substation 
>and your whole Data Center may be toast.
>
>Malice and stupidity are always with us!  A recent backup, stored off 
>site, has saved a number of people and/or companies!  The lack of same 
>has meant the end of a number of jobs and/or companies.
>

Much agreement here!

But unless I'm half blind, I sure haven't seen any Sun documentation
(on sol 10) that even considers a true need for tape backup (eg
fire).

It's as if Sun has decided that, with snapshorts and zfs, all
that's old-hat now, just totally irrevelant.

Or have I missed some better Sun-doc somewhere?

(If so, please tell me where to get it!)

Thanks,

David

David Combs wrote:
> In article <tLGdnWHrJc-RGgnXnZ2dnUVZ_hKdnZ2d@giganews.com>,
> Richard B. Gilbert <rgilbert88@comcast.net> wrote:
>> ITguy wrote:
>>>> zfs is wonderful, what with disk-crashes no problem (assuming
>>>> you have enough disks), automatic mirrors, snapshots (don't understand
>>>> them too well), and so on.
>>>>
>>>> But, you still have to make (tape) backups.
>>> It may be worth re-thinking your backup strategy.  ZFS snapshots can
>> Rethink with GREAT CARE.  That data is your company's livelihood and 
>> therefore your livelihood as well!
>>
>> ZFS *sounds* wonderful.  I still would not want to use it as the *sole* 
>> repository of irreplaceable data.  A lightning strike on your substation 
>> and your whole Data Center may be toast.
>>
>> Malice and stupidity are always with us!  A recent backup, stored off 
>> site, has saved a number of people and/or companies!  The lack of same 
>> has meant the end of a number of jobs and/or companies.
>>
> 
> Much agreement here!
> 
> But unless I'm half blind, I sure haven't seen any Sun documentation
> (on sol 10) that even considers a true need for tape backup (eg
> fire).
> 
> It's as if Sun has decided that, with snapshorts and zfs, all
> that's old-hat now, just totally irrevelant.
> 
> Or have I missed some better Sun-doc somewhere?
> 
> (If so, please tell me where to get it!)
> 
> Thanks,
> 
> David
> 
> 

I'm a little perplexed by the absence of ZFSDUMP and ZFSRESTORE.  I'm 
just a hobbyist these days but having been a programmer and/or system 
administrator for most of my working life. I'm well aware of the need 
for backup to media that can be moved off site.

OTOH, the bigger file systems get, the harder it is to copy them to tape 
or some other removable media.

It's no longer MY problem.  I wish you luck in dealing with it.

In article <h864r9$9fr$8@reader1.panix.com>,
David Combs <dkcombs@panix.com> wrote:
>100 Gigabytes?   At what cost?

Sun's marketing wonks quote X4540 at $1/GB.
<URL:http://www.sun.com/servers/x64/x4540/>

With the new 2TB SATA drives, I expect the price per GB to drop.

I don't know what you would pay to rent 4RU along with power and
bandwidth.

>  And at what SPEED?

I tar to a ZFS pool of SAS drives over NFS over Gb ethernet.
Then I archive my tars to offsite, online storage.

IIRC Sun's virtual tape library systems archive to disk and
then moves the bits to slow streamers. Eventually the bits
expire out of the online cache. 

John
groenveld@acm.org

On 2009-09-08 18:44:44 +0100, dkcombs@panix.com (David Combs) said:

> But unless I'm half blind, I sure haven't seen any Sun documentation
> (on sol 10) that even considers a true need for tape backup (eg
> fire).
> 
> It's as if Sun has decided that, with snapshorts and zfs, all
> that's old-hat now, just totally irrevelant.

I would assume that since asymptotically all of their market is using 
some third-party backup product, they've decided not to spend too much 
time on it.