From patchwork Thu Aug 29 10:31:05 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Lyakas X-Patchwork-Id: 2851260 Return-Path: X-Original-To: patchwork-linux-btrfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id CD3959F485 for ; Thu, 29 Aug 2013 10:31:15 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 90A7920444 for ; Thu, 29 Aug 2013 10:31:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 3572420443 for ; Thu, 29 Aug 2013 10:31:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756172Ab3H2KbJ (ORCPT ); Thu, 29 Aug 2013 06:31:09 -0400 Received: from mail-wg0-f45.google.com ([74.125.82.45]:39007 "EHLO mail-wg0-f45.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753134Ab3H2KbH (ORCPT ); Thu, 29 Aug 2013 06:31:07 -0400 Received: by mail-wg0-f45.google.com with SMTP id j13so229405wgh.0 for ; Thu, 29 Aug 2013 03:31:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=bI+dPoCvdgbfXH+GI3//UON7zQwli4mCiSSrPFfq+co=; b=aF02G6n2MgD618GZPcGp8t8tb1vOXt8YE7dbPh2FXL6ZT6q0FaCrnKdh5cfUG910q2 l1G0FCRuGwd93XqD9SNDKsxKH5Mj5VRGyfTBjOaG4zIrq6AVIZmTlIj+dOdKKeoG7fSq uQErYlvGr+N/jO/OZ6BQ6OubgS1cYSuiKD4g6s8s5t3V98KxSsYYQEtqjRTh5GMFCokm 59oRFfRsgLAffiA+Sbh2MZ81xaW3WC4ET+j87IRX5phG94qw4a7Y6PdntMkGS7LmEGQk ai7RZLTDY/WZVGywFijbKAxagMETFUoRSYb/xKGgVYG2dCYlTSaBEwQTz+J+ovHoyD59 M6dQ== X-Gm-Message-State: ALoCoQkImgtr+hq5vAn0OSEMX76UITsVsf4WsnyUpS6nOuNLO6BsfxfTRMqWguGLCKxoVT3Ok24/ MIME-Version: 1.0 X-Received: by 10.194.201.131 with SMTP id ka3mr4845520wjc.22.1377772265987; Thu, 29 Aug 2013 03:31:05 -0700 (PDT) Received: by 10.194.17.131 with HTTP; Thu, 29 Aug 2013 03:31:05 -0700 (PDT) Date: Thu, 29 Aug 2013 13:31:05 +0300 Message-ID: Subject: [PATCH] Notify caching_thread()s to give up on extent_commit_sem when needed. From: Alex Lyakas To: linux-btrfs Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-Spam-Status: No, score=-9.4 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP caching_thread()s do all their work under read access to extent_commit_sem. They give up on this read access only when need_resched() tells them, or when they exit. As a result, somebody that wants a WRITE access to this sem, might wait for a long time. Especially this is problematic in cache_block_group(), which can be called on critical paths like find_free_extent() and in commit path via commit_cowonly_roots(). This patch is an RFC, that attempts to fix this problem, by notifying the caching threads to give up on extent_commit_sem. On a system with a lot of metadata (~20Gb total metadata, ~10Gb extent tree), with increased number of caching_threads, commits were very slow, stuck in commit_cowonly_roots, due to this issue. With this patch, commits no longer get stuck in commit_cowonly_roots. This patch is not indented to be applied, just a request to comment on whether you agree this problem happens, and whether the fix goes in the right direction. Signed-off-by: Alex Lyakas --- fs/btrfs/ctree.h | 7 +++++++ fs/btrfs/disk-io.c | 1 + fs/btrfs/extent-tree.c | 9 +++++---- fs/btrfs/transaction.c | 2 +- 4 files changed, 14 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index c90be01..b602611 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -1427,6 +1427,13 @@ struct btrfs_fs_info { struct mutex ordered_extent_flush_mutex; struct rw_semaphore extent_commit_sem; + /* notifies the readers to give up on the sem ASAP */ + atomic_t extent_commit_sem_give_up_read; +#define BTRFS_DOWN_WRITE_EXTENT_COMMIT_SEM(fs_info) \ + do { atomic_inc(&(fs_info)->extent_commit_sem_give_up_read); \ + down_write(&(fs_info)->extent_commit_sem); \ + atomic_dec(&(fs_info)->extent_commit_sem_give_up_read); \ + } while (0) struct rw_semaphore cleanup_work_sem; diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 69e9afb..b88e688 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -2291,6 +2291,7 @@ int open_ctree(struct super_block *sb, mutex_init(&fs_info->cleaner_mutex); mutex_init(&fs_info->volume_mutex); init_rwsem(&fs_info->extent_commit_sem); + atomic_set(&fs_info->extent_commit_sem_give_up_read, 0); init_rwsem(&fs_info->cleanup_work_sem); init_rwsem(&fs_info->subvol_sem); sema_init(&fs_info->uuid_tree_rescan_sem, 1); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 95c6539..28fee78 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -442,7 +442,8 @@ next: if (ret) break; - if (need_resched()) { + if (need_resched() || + atomic_read(&fs_info->extent_commit_sem_give_up_read) > 0) { caching_ctl->progress = last; btrfs_release_path(path); up_read(&fs_info->extent_commit_sem); @@ -632,7 +633,7 @@ static int cache_block_group(struct btrfs_block_group_cache *cache, return 0; } - down_write(&fs_info->extent_commit_sem); + BTRFS_DOWN_WRITE_EXTENT_COMMIT_SEM(fs_info); atomic_inc(&caching_ctl->count); list_add_tail(&caching_ctl->list, &fs_info->caching_block_groups); up_write(&fs_info->extent_commit_sem); @@ -5462,7 +5463,7 @@ void btrfs_prepare_extent_commit(struct btrfs_trans_handle *trans, struct btrfs_block_group_cache *cache; struct btrfs_space_info *space_info; - down_write(&fs_info->extent_commit_sem); + BTRFS_DOWN_WRITE_EXTENT_COMMIT_SEM(fs_info); list_for_each_entry_safe(caching_ctl, next, &fs_info->caching_block_groups, list) { @@ -8219,7 +8220,7 @@ int btrfs_free_block_groups(struct btrfs_fs_info *info) struct btrfs_caching_control *caching_ctl; struct rb_node *n; - down_write(&info->extent_commit_sem); + BTRFS_DOWN_WRITE_EXTENT_COMMIT_SEM(fs_info); while (!list_empty(&info->caching_block_groups)) { caching_ctl = list_entry(info->caching_block_groups.next, struct btrfs_caching_control, list); diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index cac4a3f..976d20a 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -969,7 +969,7 @@ static noinline int commit_cowonly_roots(struct btrfs_trans_handle *trans, return ret; } - down_write(&fs_info->extent_commit_sem); + BTRFS_DOWN_WRITE_EXTENT_COMMIT_SEM(fs_info); switch_commit_root(fs_info->extent_root); up_write(&fs_info->extent_commit_sem);