diff mbox

No space left on device (28)

Message ID 20130326152553.GN1955@localhost.localdomain (mailing list archive)
State New, archived
Headers show

Commit Message

Josef Bacik March 26, 2013, 3:25 p.m. UTC
On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote:
> Hi,
> Am 26.03.2013 15:44, schrieb Josef Bacik:
> >>>> Am 26.03.2013 13:53, schrieb Josef Bacik:
> >>>> no - it's just mounted with mount -o noatime
> >>>>
> >>>> :~# cat /proc/mounts | grep btrfs
> >>>> /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0
> >>>>
> >>>
> >>> Ok I think I see what's going on.  Can you try this patch and see if it fixes
> >>> it?  Thanks,
> >>
> >> It still does not fix the problem.
> >>
> >> The rsync output looks like this so it does not work for file a but then
> >> continues on c d e, ...
> >>
> >> sync -av --progress /backup/ /mnt/
> >> sending incremental file list
> >> .etc_openvpn/ipp.txt
> >>          229 100%    3.99kB/s    0:00:00 (xfer#2, to-check=1009/1196)
> >> .etc_openvpn/openvpn-status.log
> >>          360 100%    6.28kB/s    0:00:00 (xfer#3, to-check=1007/1196)
> >> rsync: rename "/mnt/.etc_openvpn/.ipp.txt.t9lucX" ->
> >> ".etc_openvpn/ipp.txt": No space left on device (28)
> >> .log/
> >> .log/UcliEvt.log
> >>       104188 100%  147.67kB/s    0:00:00 (xfer#4, to-check=1131/2700)
> >> .log/auth.log
> >>     15211522 100%    2.97MB/s    0:00:04 (xfer#5, to-check=1105/2700)
> >> .log/auth.log.1
> >>     19431424  61%    7.35MB/s    0:00:01
> >>
> >> the dmesg output looks like this:
> >> [  551.321576] returning enospc, space_info 3, size 0 reserved 0, flush
> >> 2, flush_state 7  dumping space info
> >> [  551.323694] space_info 4 has 6439526400 free, is full
> >> [  551.323696] space_info total=25748307968, used=19308666880, pinned=0,
> >> reserved=49152, may_use=6438453248, readonly=65536
> >>
> > 
> > Ok so then this is probably it, let me know if it helps.  Thanks,
> 
> OK it now has copied a lot of files (170) without an error all were very
> small.
>

Welp progress is good.  Throw this into the mix and go again, it's just adding
some more debugging so I can make sure I'm going down the right rabbit hole.
Thanks,

Josef

Comments

Stefan Priebe - Profihost AG March 26, 2013, 4:19 p.m. UTC | #1
HI,


Am 26.03.2013 16:25, schrieb Josef Bacik:
> On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote:
>> Hi,
>> Am 26.03.2013 15:44, schrieb Josef Bacik:
>>>>>> Am 26.03.2013 13:53, schrieb Josef Bacik:
>>>>>> no - it's just mounted with mount -o noatime
>>>>>>
>>>>>> :~# cat /proc/mounts | grep btrfs
>>>>>> /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0
>>>>>>
>>>>>
>>>>> Ok I think I see what's going on.  Can you try this patch and see if it fixes
>>>>> it?  Thanks,
>>>>
>>>> It still does not fix the problem.
>>>>
>>>> The rsync output looks like this so it does not work for file a but then
>>>> continues on c d e, ...
>>>>
>>>> sync -av --progress /backup/ /mnt/
>>>> sending incremental file list
>>>> .etc_openvpn/ipp.txt
>>>>           229 100%    3.99kB/s    0:00:00 (xfer#2, to-check=1009/1196)
>>>> .etc_openvpn/openvpn-status.log
>>>>           360 100%    6.28kB/s    0:00:00 (xfer#3, to-check=1007/1196)
>>>> rsync: rename "/mnt/.etc_openvpn/.ipp.txt.t9lucX" ->
>>>> ".etc_openvpn/ipp.txt": No space left on device (28)
>>>> .log/
>>>> .log/UcliEvt.log
>>>>        104188 100%  147.67kB/s    0:00:00 (xfer#4, to-check=1131/2700)
>>>> .log/auth.log
>>>>      15211522 100%    2.97MB/s    0:00:04 (xfer#5, to-check=1105/2700)
>>>> .log/auth.log.1
>>>>      19431424  61%    7.35MB/s    0:00:01
>>>>
>>>> the dmesg output looks like this:
>>>> [  551.321576] returning enospc, space_info 3, size 0 reserved 0, flush
>>>> 2, flush_state 7  dumping space info
>>>> [  551.323694] space_info 4 has 6439526400 free, is full
>>>> [  551.323696] space_info total=25748307968, used=19308666880, pinned=0,
>>>> reserved=49152, may_use=6438453248, readonly=65536
>>>>
>>>
>>> Ok so then this is probably it, let me know if it helps.  Thanks,
>>
>> OK it now has copied a lot of files (170) without an error all were very
>> small.
>>
>
> Welp progress is good.  Throw this into the mix and go again, it's just adding
> some more debugging so I can make sure I'm going down the right rabbit hole.
> Thanks,

Output is now:
[ 9587.445642] returning enospc, space_info 3, size 0 reserved 0, flush 
2, flush_state 7  dumping space info
[ 9587.527392] dumping block rsv 2, size 0 reserved 0
[ 9587.567871] dumping block rsv 5, size 196608 reserved 196608
[ 9587.607661] dumping block rsv 1, size 6438256640 reserved 6438256640
[ 9587.646958] space_info 4 has 6439428096 free, is full
[ 9587.646963] space_info total=25748307968, used=19308769280, pinned=0, 
reserved=45056, may_use=6438453248, readonly=65536
[ 9587.649410] returning enospc, space_info 3, size 0 reserved 0, flush 
2, flush_state 7  dumping space info
[ 9587.727000] dumping block rsv 2, size 0 reserved 0
[ 9587.765284] dumping block rsv 5, size 98304 reserved 98304
[ 9587.802849] dumping block rsv 1, size 6438256640 reserved 6438256640
[ 9587.839935] space_info 4 has 6439428096 free, is full
[ 9587.839936] space_info total=25748307968, used=19308769280, pinned=0, 
reserved=45056, may_use=6438354944, readonly=65536

Stefan
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 84e8909..1cf810a 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -4026,6 +4026,15 @@  static int flush_space(struct btrfs_root *root,
 
 	return ret;
 }
+
+static void dump_block_rsv(struct btrfs_block_rsv *block_rsv)
+{
+	spin_lock(&block_rsv->lock);
+	printk(KERN_ERR "dumping block rsv %u, size %Lu reserved %Lu\n",
+	       block_rsv->type, block_rsv->size, block_rsv->reserved);
+	spin_unlock(&block_rsv->lock);
+}
+
 /**
  * reserve_metadata_bytes - try to reserve bytes from the block_rsv's space
  * @root - the root we're allocating for
@@ -4179,6 +4188,9 @@  out:
 			       "flush %d, flush_state %d  dumping space info\n", block_rsv->type,
 			       block_rsv->size, block_rsv->reserved, flush, flush_state);
 			spin_unlock(&block_rsv->lock);
+			dump_block_rsv(&root->fs_info->delalloc_block_rsv);
+			dump_block_rsv(&root->fs_info->delayed_block_rsv);
+			dump_block_rsv(&root->fs_info->global_block_rsv);
 			dump_space_info(space_info, 0, 0);
 		}