From patchwork Tue Mar 26 15:25:53 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 2338261 Return-Path: X-Original-To: patchwork-linux-btrfs@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork1.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork1.kernel.org (Postfix) with ESMTP id A7F77400E6 for ; Tue, 26 Mar 2013 15:26:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965413Ab3CZPZ6 (ORCPT ); Tue, 26 Mar 2013 11:25:58 -0400 Received: from dkim2.fusionio.com ([66.114.96.54]:34764 "EHLO dkim2.fusionio.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934781Ab3CZPZ4 (ORCPT ); Tue, 26 Mar 2013 11:25:56 -0400 Received: from mx1.fusionio.com (unknown [10.101.1.160]) by dkim2.fusionio.com (Postfix) with ESMTP id 5090E9A0680 for ; Tue, 26 Mar 2013 09:25:56 -0600 (MDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fusionio.com; s=default; t=1364311556; bh=W62N8vd4GgbDMyEkYLXBj74CjwbrCUWZe9/iB8FMCYU=; h=Date:From:To:CC:Subject:References:In-Reply-To; b=dAa5bPJ2pYCYb0jrtUAVoSkDcy6k12yV5I7m/bOV+A/SxXi6GjXGDqrEoeMu0iRJN p5OBwWjSiTORlzWudq3PPDncLkzz4a9aZpWhq/GPCyasGFgTgl3zY3z/OOLDNYCom/ ZY8yrBc+kblUbAWJnrtpilphpfuSZKMsRrqmcd+w= X-ASG-Debug-ID: 1364311555-03d6a52ac0f1070001-6jHSXT Received: from mail1.int.fusionio.com (mail1.int.fusionio.com [10.101.1.21]) by mx1.fusionio.com with ESMTP id YLGvYAMyOlQJ01YO (version=TLSv1 cipher=AES128-SHA bits=128 verify=NO); Tue, 26 Mar 2013 09:25:55 -0600 (MDT) X-Barracuda-Envelope-From: JBacik@fusionio.com Received: from localhost (98.26.82.158) by mail.fusionio.com (10.101.1.19) with Microsoft SMTP Server (TLS) id 8.3.83.0; Tue, 26 Mar 2013 09:25:55 -0600 Date: Tue, 26 Mar 2013 11:25:53 -0400 From: Josef Bacik To: Stefan Priebe - Profihost AG CC: Josef Bacik , Chris Mason , "linux-btrfs@vger.kernel.org" Subject: Re: No space left on device (28) Message-ID: <20130326152553.GN1955@localhost.localdomain> X-ASG-Orig-Subj: Re: No space left on device (28) References: <20130322204932.GE1955@localhost.localdomain> <514CC52B.1000201@profihost.ag> <20130325201426.GG1955@localhost.localdomain> <51515226.9010509@profihost.ag> <20130326125319.GH1955@localhost.localdomain> <51519ABB.4030207@profihost.ag> <20130326133053.GK1955@localhost.localdomain> <5151A75C.6080703@profihost.ag> <20130326144426.GL1955@localhost.localdomain> <5151B8AF.60900@profihost.ag> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <5151B8AF.60900@profihost.ag> User-Agent: Mutt/1.5.21 (2011-07-01) X-Barracuda-Connect: mail1.int.fusionio.com[10.101.1.21] X-Barracuda-Start-Time: 1364311555 X-Barracuda-Encrypted: AES128-SHA X-Barracuda-URL: http://10.101.1.180:8000/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at fusionio.com X-Barracuda-Bayes: INNOCENT GLOBAL 0.0000 1.0000 -2.0210 X-Barracuda-Spam-Score: -2.02 X-Barracuda-Spam-Status: No, SCORE=-2.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=9.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.2.126301 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote: > Hi, > Am 26.03.2013 15:44, schrieb Josef Bacik: > >>>> Am 26.03.2013 13:53, schrieb Josef Bacik: > >>>> no - it's just mounted with mount -o noatime > >>>> > >>>> :~# cat /proc/mounts | grep btrfs > >>>> /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 > >>>> > >>> > >>> Ok I think I see what's going on. Can you try this patch and see if it fixes > >>> it? Thanks, > >> > >> It still does not fix the problem. > >> > >> The rsync output looks like this so it does not work for file a but then > >> continues on c d e, ... > >> > >> sync -av --progress /backup/ /mnt/ > >> sending incremental file list > >> .etc_openvpn/ipp.txt > >> 229 100% 3.99kB/s 0:00:00 (xfer#2, to-check=1009/1196) > >> .etc_openvpn/openvpn-status.log > >> 360 100% 6.28kB/s 0:00:00 (xfer#3, to-check=1007/1196) > >> rsync: rename "/mnt/.etc_openvpn/.ipp.txt.t9lucX" -> > >> ".etc_openvpn/ipp.txt": No space left on device (28) > >> .log/ > >> .log/UcliEvt.log > >> 104188 100% 147.67kB/s 0:00:00 (xfer#4, to-check=1131/2700) > >> .log/auth.log > >> 15211522 100% 2.97MB/s 0:00:04 (xfer#5, to-check=1105/2700) > >> .log/auth.log.1 > >> 19431424 61% 7.35MB/s 0:00:01 > >> > >> the dmesg output looks like this: > >> [ 551.321576] returning enospc, space_info 3, size 0 reserved 0, flush > >> 2, flush_state 7 dumping space info > >> [ 551.323694] space_info 4 has 6439526400 free, is full > >> [ 551.323696] space_info total=25748307968, used=19308666880, pinned=0, > >> reserved=49152, may_use=6438453248, readonly=65536 > >> > > > > Ok so then this is probably it, let me know if it helps. Thanks, > > OK it now has copied a lot of files (170) without an error all were very > small. > Welp progress is good. Throw this into the mix and go again, it's just adding some more debugging so I can make sure I'm going down the right rabbit hole. Thanks, Josef diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 84e8909..1cf810a 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4026,6 +4026,15 @@ static int flush_space(struct btrfs_root *root, return ret; } + +static void dump_block_rsv(struct btrfs_block_rsv *block_rsv) +{ + spin_lock(&block_rsv->lock); + printk(KERN_ERR "dumping block rsv %u, size %Lu reserved %Lu\n", + block_rsv->type, block_rsv->size, block_rsv->reserved); + spin_unlock(&block_rsv->lock); +} + /** * reserve_metadata_bytes - try to reserve bytes from the block_rsv's space * @root - the root we're allocating for @@ -4179,6 +4188,9 @@ out: "flush %d, flush_state %d dumping space info\n", block_rsv->type, block_rsv->size, block_rsv->reserved, flush, flush_state); spin_unlock(&block_rsv->lock); + dump_block_rsv(&root->fs_info->delalloc_block_rsv); + dump_block_rsv(&root->fs_info->delayed_block_rsv); + dump_block_rsv(&root->fs_info->global_block_rsv); dump_space_info(space_info, 0, 0); }