From patchwork Tue Jun 21 10:24:08 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Anand Jain X-Patchwork-Id: 9190273 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0EB646075A for ; Tue, 21 Jun 2016 10:24:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F25AB28156 for ; Tue, 21 Jun 2016 10:24:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E6A3A2815E; Tue, 21 Jun 2016 10:24:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1D59828161 for ; Tue, 21 Jun 2016 10:24:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751870AbcFUKYf (ORCPT ); Tue, 21 Jun 2016 06:24:35 -0400 Received: from aserp1040.oracle.com ([141.146.126.69]:27703 "EHLO aserp1040.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751408AbcFUKXq (ORCPT ); Tue, 21 Jun 2016 06:23:46 -0400 Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u5LANMFB005575 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 21 Jun 2016 10:23:22 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0022.oracle.com (8.14.4/8.13.8) with ESMTP id u5LANL1u029403 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 21 Jun 2016 10:23:22 GMT Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by aserv0121.oracle.com (8.13.8/8.13.8) with ESMTP id u5LANK2T008429; Tue, 21 Jun 2016 10:23:21 GMT Received: from arch2.sg.oracle.com (/10.186.101.143) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 21 Jun 2016 03:23:20 -0700 From: Anand Jain To: linux-btrfs@vger.kernel.org Cc: holger@applied-asynchrony.com, clm@fb.com, dsterba@suse.com, xiaolong.ye@intel.com Subject: [PATCH v2 2/2] btrfs: wait for bdev put Date: Tue, 21 Jun 2016 18:24:08 +0800 Message-Id: <1466504648-2937-1-git-send-email-anand.jain@oracle.com> X-Mailer: git-send-email 2.7.0 In-Reply-To: <1465901726-15490-2-git-send-email-anand.jain@oracle.com> References: <1465901726-15490-2-git-send-email-anand.jain@oracle.com> X-Source-IP: userv0022.oracle.com [156.151.31.74] Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Anand Jain Further to the commit bc178622d40d87e75abc131007342429c9b03351 btrfs: use rcu_barrier() to wait for bdev puts at unmount This patch implements a method to time wait on the __free_device() which actually does the bdev put. This is needed as the user space running 'btrfs fi show -d' immediately after the replace and unmount, is still reading older information from the device. mail-archive.com/linux-btrfs@vger.kernel.org/msg54188.html Signed-off-by: Anand Jain [updates: bc178622d40d87e75abc131007342429c9b03351] Tested-by: Holger Hoffstätte --- v2: Also to make sure bdev_closing is set it needs rcu_barrier(), restored rcu_barrier(). fs/btrfs/volumes.c | 45 +++++++++++++++++++++++++++++++++++++++++++-- fs/btrfs/volumes.h | 1 + 2 files changed, 44 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 604daf315669..ef61c34cafbf 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -27,6 +27,7 @@ #include #include #include +#include #include #include "ctree.h" #include "extent_map.h" @@ -254,6 +255,17 @@ static struct btrfs_device *__alloc_device(void) return dev; } +static int is_device_closing(struct list_head *head) +{ + struct btrfs_device *dev; + + list_for_each_entry(dev, head, dev_list) { + if (dev->bdev_closing) + return 1; + } + return 0; +} + static noinline struct btrfs_device *__find_device(struct list_head *head, u64 devid, u8 *uuid) { @@ -832,12 +844,22 @@ again: static void __free_device(struct work_struct *work) { struct btrfs_device *device; + struct btrfs_device *new_device_addr; device = container_of(work, struct btrfs_device, rcu_work); if (device->bdev) blkdev_put(device->bdev, device->mode); + /* + * If we are coming here from btrfs_close_one_device() + * then it allocates a new device structure for the same + * devid, so find device again with the devid + */ + new_device_addr = __find_device(&device->fs_devices->devices, + device->devid, NULL); + + new_device_addr->bdev_closing = 0; rcu_string_free(device->name); kfree(device); } @@ -884,6 +906,12 @@ static void btrfs_close_one_device(struct btrfs_device *device) list_replace_rcu(&device->dev_list, &new_device->dev_list); new_device->fs_devices = device->fs_devices; + /* + * So to wait for kworkers to finish all blkdev_puts, + * so device is really free when umount is done. + */ + new_device->bdev_closing = 1; + call_rcu(&device->rcu, free_device); } @@ -912,6 +940,7 @@ int btrfs_close_devices(struct btrfs_fs_devices *fs_devices) { struct btrfs_fs_devices *seed_devices = NULL; int ret; + int retry_cnt = 5; mutex_lock(&uuid_mutex); ret = __btrfs_close_devices(fs_devices); @@ -929,10 +958,22 @@ int btrfs_close_devices(struct btrfs_fs_devices *fs_devices) } /* * Wait for rcu kworkers under __btrfs_close_devices - * to finish all blkdev_puts so device is really - * free when umount is done. + * to finish all free_device() */ rcu_barrier(); + + /* + * Wait for a grace period so that __free_device() + * will actaully do the device close. + */ + while (is_device_closing(&fs_devices->devices) && + --retry_cnt) { + mdelay(1000); //1 sec + } + + if (!(retry_cnt > 0)) + printk(KERN_WARNING "BTRFS: %pU bdev_put didn't complete, giving up\n", + fs_devices->fsid); return ret; } diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h index 0ac90f8d85bd..945e49f5e17d 100644 --- a/fs/btrfs/volumes.h +++ b/fs/btrfs/volumes.h @@ -150,6 +150,7 @@ struct btrfs_device { /* Counter to record the change of device stats */ atomic_t dev_stats_ccnt; atomic_t dev_stat_values[BTRFS_DEV_STAT_VALUES_MAX]; + int bdev_closing; }; /*