I've got a problem on a Sun Blade 2000 running Solaris 10 update 7
with ZFS file systems. One of the file systems is full, but when I try
to remove a file, I get:

# rm 3.ini
rm: 3.ini not removed: no space left on device

This is happening in single user mode too.

I've tried deleting the smallest file I could, in the hope that any
disk space the deletion uses would be minimal. But I still can't
remove the file. I even found a file of 0 bytes in length on the disk,
but that too could not be deleted.

Short of drilling a hole in one of the platters of the disk, how can I
remove files?

Dave

> # rm 3.ini
> rm: 3.ini not removed: no space left on device

Do you have any snapshots in the ZFS pool ?

On 6 Oct, 10:11, Mark  Round <mark.ro...@gmail.com> wrote:
> > # rm 3.ini
> > rm: 3.ininotremoved:nospaceleftondevice
>
> Do you have any snapshots in theZFSpool ?

Thank you. That did solve it.

I'd previously tried

cat /dev/null > some-big-existing-file

but despite the size of the file then being only 0 bytes, still I
could not remove it.

On 2009-10-06 10:30:41 +0100, David Kirkby <drkirkby@gmail.com> said:

> On 6 Oct, 10:11, Mark  Round <mark.ro...@gmail.com> wrote:
>>> # rm 3.ini
>>> rm: 3.ininotremoved:nospaceleftondevice
>> 
>> Do you have any snapshots in theZFSpool ?
> 
> Thank you. That did solve it.
> 
> I'd previously tried
> 
> cat /dev/null > some-big-existing-file
> 
> but despite the size of the file then being only 0 bytes, still I
> could not remove it.

Copy-on-write thwarts this cunning scheme :-)

I wonder if it is worth having a sacrificial snapshot on your system 
just in case the pool does fill up and there are no real snapshots to 
destroy?
-- 
Chris

David Kirkby wrote:
> I've got a problem on a Sun Blade 2000 running Solaris 10 update 7
> with ZFS file systems. One of the file systems is full, but when I try
> to remove a file, I get:
> 
> # rm 3.ini
> rm: 3.ini not removed: no space left on device
> 
> This is happening in single user mode too.
> 
> I've tried deleting the smallest file I could, in the hope that any
> disk space the deletion uses would be minimal. But I still can't
> remove the file. I even found a file of 0 bytes in length on the disk,
> but that too could not be deleted.
> 
> Short of drilling a hole in one of the platters of the disk, how can I
> remove files?
> 
> Dave

Format?  ;-)

Chris Ridd wrote:
> On 2009-10-06 10:30:41 +0100, David Kirkby <drkirkby@gmail.com> said:
> 
>> On 6 Oct, 10:11, Mark  Round <mark.ro...@gmail.com> wrote:
>>>> # rm 3.ini
>>>> rm: 3.ininotremoved:nospaceleftondevice
>>>
>>> Do you have any snapshots in theZFSpool ?
>>
>> Thank you. That did solve it.
>>
>> I'd previously tried
>>
>> cat /dev/null > some-big-existing-file
>>
>> but despite the size of the file then being only 0 bytes, still I
>> could not remove it.
> 
> Copy-on-write thwarts this cunning scheme :-)

It was one try I thought of, after trying to remove very small files. I 
thought I could get rid of a few hundred MB that way. The size changed 
to 0 bytes, with no error message, but still I could not remove any files.

> I wonder if it is worth having a sacrificial snapshot on your system 
> just in case the pool does fill up and there are no real snapshots to 
> destroy?

Yes, it probably is.

It would also be worth it if Sun had some tool that could remove a file, 
without the copy on write process, for use only by root. It seems crazy 
for a file system to get so full you can't empty it.

Had I not had a snapshot, what would I have needed to do? Drill a hole 
in the platter, or the *slightly* less severe option of formatting the 
disk? Either would have got rid of the files - perhaps the former a bit 
more securely.



-- 
I respectfully request that this message is not archived by companies as
unscrupulous as 'Experts Exchange' . In case you are unaware,
'Experts Exchange'  take questions posted on the web and try to find
idiots stupid enough to pay for the answers, which were posted freely
by others. They are leeches.

On 2009-10-06 19:45:51 +0100, Dave <foo@coo.com> said:

> It was one try I thought of, after trying to remove very small files. I 
> thought I could get rid of a few hundred MB that way. The size changed 
> to 0 bytes, with no error message, but still I could not remove any 
> files.

I'm not sure (I don't understand ZFS well enough), but I would assume 
that if you manage to get it to say the file is now 0 bytes long (so it 
has clearly committed a change) then the space the file used now should 
be free, *unless* the file belongs to a snapshot, in which case it is 
not free.  So, did the file you tried this on exist in any snapshot?

Tim Bradshaw <tfb@tfeb.org> writes:

>On 2009-10-06 19:45:51 +0100, Dave <foo@coo.com> said:

>> It was one try I thought of, after trying to remove very small files. I 
>> thought I could get rid of a few hundred MB that way. The size changed 
>> to 0 bytes, with no error message, but still I could not remove any 
>> files.

>I'm not sure (I don't understand ZFS well enough), but I would assume 
>that if you manage to get it to say the file is now 0 bytes long (so it 
>has clearly committed a change) then the space the file used now should 
>be free, *unless* the file belongs to a snapshot, in which case it is 
>not free.  So, did the file you tried this on exist in any snapshot?


Also, the "you can't rm file is the filesystem is full" is also fixed but
probably not in the presence of snapshots which contains the files to
be removed.

Casper
-- 
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

Tim Bradshaw wrote:
> On 2009-10-06 19:45:51 +0100, Dave <foo@coo.com> said:
> 
>> It was one try I thought of, after trying to remove very small files. 
>> I thought I could get rid of a few hundred MB that way. The size 
>> changed to 0 bytes, with no error message, but still I could not 
>> remove any files.
> 
> I'm not sure (I don't understand ZFS well enough), but I would assume 
> that if you manage to get it to say the file is now 0 bytes long (so it 
> has clearly committed a change) then the space the file used now should 
> be free, *unless* the file belongs to a snapshot, in which case it is 
> not free.  So, did the file you tried this on exist in any snapshot?
> 

The truth is, I can't say for sure. They probably were mainly older 
files, as I started first at clearing files which were very old (some 
had not been touched for around 7-8 years, so I don't think I really 
needed them any more. Hence they were probably in the snapshots. But I 
can't be sure.



-- 
I respectfully request that this message is not archived by companies as
unscrupulous as 'Experts Exchange' . In case you are unaware,
'Experts Exchange'  take questions posted on the web and try to find
idiots stupid enough to pay for the answers, which were posted freely
by others. They are leeches.