From patchwork Wed Jun 27 12:45:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 10491373 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 19F21602B3 for ; Wed, 27 Jun 2018 12:47:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 07D2A28D7A for ; Wed, 27 Jun 2018 12:47:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F014B28D80; Wed, 27 Jun 2018 12:47:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5AEED28D7A for ; Wed, 27 Jun 2018 12:47:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 68C506B0275; Wed, 27 Jun 2018 08:47:23 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 63CE36B0276; Wed, 27 Jun 2018 08:47:23 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4DE3E6B0277; Wed, 27 Jun 2018 08:47:23 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f198.google.com (mail-qt0-f198.google.com [209.85.216.198]) by kanga.kvack.org (Postfix) with ESMTP id 1C0BC6B0275 for ; Wed, 27 Jun 2018 08:47:23 -0400 (EDT) Received: by mail-qt0-f198.google.com with SMTP id g8-v6so1855413qtp.19 for ; Wed, 27 Jun 2018 05:47:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=POtMcwteeTbv3vophBJXGy4EfG1rstscUIQXIDSlbzA=; b=SpArfptvKg4WjUcQN/sTes1njuoORx8USVDAbrhhDJQvz14yZeSe04imbkcaqeb5hV 3ACfkUuIQHYQDKTZHSe1Dl66QilNhhCYFgzIAeQ48yZD5tfZO+wGG1E8/qQHxi7nH2RH oHdFsMLZFkrnd21BQGMPeFG6jPTopi+dea329ectwGmk+HUlFZILZe1VKd6C4m7IUJ+0 07yR0NDwr33+Tw4zvDELnv+J6FdEEOSNajuRqHcwCPjW4iOl01eWUMJNjN01FsPwbNUq XaPfmSB1iCDjJJL8YpvrqfWsS33QO3niEUhxahtRlr+5tXqpXQInyPU6mlsVQWGLDjCW Rq4Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APt69E0mD4yiJGVsgzbPzUTEbS9InXvfpo+WzPXf8aCEbZXD4aU/1akC eiAsBJq449uEPNdFPkj6rzWqzrB6zyy1lDuZWKF67luyifLLW1Qf5cekltVRzhqSJkVmH12KLrP eXKItCx498yCzofpWAZ8Tjnk5FP7xKcFHj5UbQ3TfQ8k114nhkXGxImQA38clmwHNkw== X-Received: by 2002:a0c:8203:: with SMTP id h3-v6mr5161487qva.117.1530103642904; Wed, 27 Jun 2018 05:47:22 -0700 (PDT) X-Google-Smtp-Source: AAOMgpcbD3ODv9G++W52vxGaYu/yHbPsraQZVFz6avumtBXgW+LCCZnVxVT2ZZYTbznCzW9mOZBZ X-Received: by 2002:a0c:8203:: with SMTP id h3-v6mr5161453qva.117.1530103642209; Wed, 27 Jun 2018 05:47:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1530103642; cv=none; d=google.com; s=arc-20160816; b=V/m1hVBA6dcOiADoZZXF/Tk8pW5rkQrOg2g7JNMIgT2VHZFfRMPeVp2zNLyPq817gn UpKZLdNXufu2MMgVkn6uCImHWXHJ54cVvnZfqfCwoLm0JQ6IynepeSIAsLAKuTr4zpUh 22iZmqcuP2dqs3i/H0BAF5NxMBIkOEI8AMZpSw9s4FGzkaMMURHEGSoxXHdEjWh6QRvF uHyy5vi/pXEK/zsVnH8FH8kF5p7s1Xdd2jnSZtWN9g8Iu3BdpdA2LNfjNTLrQI7pebgS 6TMy1wrsnAjkYQcntjih7yLTN0WX9zVz+uCFH/0VgNT/rnBpEZqnkd6c4rZUsNliBOMd 82OQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=POtMcwteeTbv3vophBJXGy4EfG1rstscUIQXIDSlbzA=; b=CJ+l4SeFdL2YPOSg/+SeCwrM3sg4uLnrG0p1mIJG9LzbLgiL+bRjbfavqH1FEcgA42 JQVUU1aLyVgnfPqIWpvzCLVYS+rFEoLfXOSWa3DoZN0XyUsW+1klljDonL6EV1MWi9nG InIttR5/ccZUCY9DLK7FKMtiOlnRYMOdmC1iqU6VQ7Sr2pbuW//AB6HD6W3nfG9DPyOR N/TqUqcPAmmV+5ESU+kOO23TnPJ5JO1OGR+nlD0KpWB+d/jECRgp+NwRkXdOdH5g1s6j vm5MY4DgrNY2/pyNxJxT010z+lsa4RinTONQ5kiTmyYImWnmX+YUK3ZZghUeoFNeWREN 5VNg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id p184-v6si3893824qkb.380.2018.06.27.05.47.22 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Jun 2018 05:47:22 -0700 (PDT) Received-SPF: pass (google.com: domain of ming.lei@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id CA1FB70206; Wed, 27 Jun 2018 12:47:21 +0000 (UTC) Received: from localhost (ovpn-12-44.pek2.redhat.com [10.72.12.44]) by smtp.corp.redhat.com (Postfix) with ESMTP id B140B2026D5B; Wed, 27 Jun 2018 12:47:13 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , Kent Overstreet Cc: David Sterba , Huang Ying , Mike Snitzer , linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Theodore Ts'o , "Darrick J . Wong" , Coly Li , Filipe Manana , Randy Dunlap , Christoph Hellwig Subject: [PATCH V7 07/24] block: simplify bio_check_pages_dirty Date: Wed, 27 Jun 2018 20:45:31 +0800 Message-Id: <20180627124548.3456-8-ming.lei@redhat.com> In-Reply-To: <20180627124548.3456-1-ming.lei@redhat.com> References: <20180627124548.3456-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 27 Jun 2018 12:47:21 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.2]); Wed, 27 Jun 2018 12:47:21 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'ming.lei@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Christoph Hellwig bio_check_pages_dirty currently inviolates the invariant that bv_page of a bio_vec inside bi_vcnt shouldn't be zero, and that is going to become really annoying with multpath biovecs. Fortunately there isn't any all that good reason for it - once we decide to defer freeing the bio to a workqueue holding onto a few additional pages isn't really an issue anymore. So just check if there is a clean page that needs dirtying in the first path, and do a second pass to free them if there was none, while the cache is still hot. Also use the chance to micro-optimize bio_dirty_fn a bit by not saving irq state - we know we are called from a workqueue. Reviewed-by: Ming Lei Signed-off-by: Christoph Hellwig --- block/bio.c | 56 +++++++++++++++++++++----------------------------------- 1 file changed, 21 insertions(+), 35 deletions(-) diff --git a/block/bio.c b/block/bio.c index 43698bcff737..77f991688810 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1570,19 +1570,15 @@ static void bio_release_pages(struct bio *bio) struct bio_vec *bvec; int i; - bio_for_each_segment_all(bvec, bio, i) { - struct page *page = bvec->bv_page; - - if (page) - put_page(page); - } + bio_for_each_segment_all(bvec, bio, i) + put_page(bvec->bv_page); } /* * bio_check_pages_dirty() will check that all the BIO's pages are still dirty. * If they are, then fine. If, however, some pages are clean then they must * have been written out during the direct-IO read. So we take another ref on - * the BIO and the offending pages and re-dirty the pages in process context. + * the BIO and re-dirty the pages in process context. * * It is expected that bio_check_pages_dirty() will wholly own the BIO from * here on. It will run one put_page() against each page and will run one @@ -1600,52 +1596,42 @@ static struct bio *bio_dirty_list; */ static void bio_dirty_fn(struct work_struct *work) { - unsigned long flags; - struct bio *bio; + struct bio *bio, *next; - spin_lock_irqsave(&bio_dirty_lock, flags); - bio = bio_dirty_list; + spin_lock_irq(&bio_dirty_lock); + next = bio_dirty_list; bio_dirty_list = NULL; - spin_unlock_irqrestore(&bio_dirty_lock, flags); + spin_unlock_irq(&bio_dirty_lock); - while (bio) { - struct bio *next = bio->bi_private; + while ((bio = next) != NULL) { + next = bio->bi_private; bio_set_pages_dirty(bio); bio_release_pages(bio); bio_put(bio); - bio = next; } } void bio_check_pages_dirty(struct bio *bio) { struct bio_vec *bvec; - int nr_clean_pages = 0; + unsigned long flags; int i; bio_for_each_segment_all(bvec, bio, i) { - struct page *page = bvec->bv_page; - - if (PageDirty(page) || PageCompound(page)) { - put_page(page); - bvec->bv_page = NULL; - } else { - nr_clean_pages++; - } + if (!PageDirty(bvec->bv_page) && !PageCompound(bvec->bv_page)) + goto defer; } - if (nr_clean_pages) { - unsigned long flags; - - spin_lock_irqsave(&bio_dirty_lock, flags); - bio->bi_private = bio_dirty_list; - bio_dirty_list = bio; - spin_unlock_irqrestore(&bio_dirty_lock, flags); - schedule_work(&bio_dirty_work); - } else { - bio_put(bio); - } + bio_release_pages(bio); + bio_put(bio); + return; +defer: + spin_lock_irqsave(&bio_dirty_lock, flags); + bio->bi_private = bio_dirty_list; + bio_dirty_list = bio; + spin_unlock_irqrestore(&bio_dirty_lock, flags); + schedule_work(&bio_dirty_work); } EXPORT_SYMBOL_GPL(bio_check_pages_dirty);