From patchwork Sat Jun 9 12:29:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 10455511 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4E6F3601F7 for ; Sat, 9 Jun 2018 12:30:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3F8BB223A4 for ; Sat, 9 Jun 2018 12:30:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 34616223A6; Sat, 9 Jun 2018 12:30:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD072223A4 for ; Sat, 9 Jun 2018 12:30:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CA78A6B0007; Sat, 9 Jun 2018 08:30:55 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C2F236B0008; Sat, 9 Jun 2018 08:30:55 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACFF46B000A; Sat, 9 Jun 2018 08:30:55 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f200.google.com (mail-qt0-f200.google.com [209.85.216.200]) by kanga.kvack.org (Postfix) with ESMTP id 81DBE6B0007 for ; Sat, 9 Jun 2018 08:30:55 -0400 (EDT) Received: by mail-qt0-f200.google.com with SMTP id f8-v6so4518975qth.9 for ; Sat, 09 Jun 2018 05:30:55 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=c/hPWK14ZqMkq/ho+wBGoaj8Kbv9DZSx6avdvAUsmm4=; b=gqkO1P4r2XJ76VUOKvbhbMUiHhjzCTUy0In4xteGgtEvWv4/bDwgmC0Y/CYeSjry1c JtnAVcmlamHQAFKUNB0va1j1UzmO0eIQE6GlIwy3IBrBEn6JJymjxzKCUrF9o8wNLMsS lbwT8sUcSG7XC4zIgmovoAmqJ0lg1CDiwpWppAR0BeCHzsFEnAX7sN0QKMvZUkEFVQPb 5CyXyv5vNGAGqzOU3iNbhrXPgVR/Z/qbTj3e8Q1gM0s1l3zo99nQPpDfutsoPrAAGuQJ YEq14D1CPjuUsUBTnO0DNyZzq6DTZWxQSAXLSV+/FR1el+PEJRwN/WfW9DCdlnAAcSNC nSFA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: APt69E3xwpJHr8EotHQdHh4NC86Zw3kpQzeMSQASULm9chgoBIvx3OUN IjHpdx+rW0gfZXpwuz1ww1IAXypvOhc6ksLcAxUaWvKCpHFsDkmorjB/I418XkHqCQHQTsQnGip tahaWmJnWxEEfas/ISmMSfa75XC4YIdsoCPiRTKE508ai3HxEQvb9gTP7wm+ic/rNmg== X-Received: by 2002:aed:2387:: with SMTP id j7-v6mr9776443qtc.121.1528547455197; Sat, 09 Jun 2018 05:30:55 -0700 (PDT) X-Google-Smtp-Source: ADUXVKJFsI52dVJBplLsOkqaf2CCzxqsegBUE+uEGBrMKYAe4CN/E1lwpKpYBoHoUSZ/gd7J6nDL X-Received: by 2002:aed:2387:: with SMTP id j7-v6mr9776405qtc.121.1528547454466; Sat, 09 Jun 2018 05:30:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1528547454; cv=none; d=google.com; s=arc-20160816; b=DNx5SeK2B+in5+I+LZmf2S7fSjXw/4GRhMuY/D7iY/1mnqPC/mB98tozX1Ah9F7BuQ BaLr4/3wTodoNuhgknxKBuJVBcS8fmo8KkUmFZd3Ztba/VYCZEB76DGzmEPqf+EgfblH w7NdtczjG+HvbEtXXWcXMMtnP6IRwq/FMuzhN+D6t5jXsoNRkQQ7+SfLbJ8Harm6VVxG iTScqxraZwJiNPcFmThs53hbcpd7o8wcwjOe7Z6tE7hWLDWtJPKRMKngCt9DiaRr6YlC c9f0tPers+eoD1Hc7k55Uih8ggI7R8vjDIcZR9KZ4wfNhH7LitagoA6ldJq8TgbLB6T7 VCDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=c/hPWK14ZqMkq/ho+wBGoaj8Kbv9DZSx6avdvAUsmm4=; b=wvsVg3EwzNChealVJ4i/VGST08SgQ5vYOKKFKqBbUNbOpxDONLBF+gaZ9gKAOHUc+u M9Mghc//5BOMTW6Jf1v9wd7/zgq7/olZeDW3AP6mCkAfKxAwHSr8JTVib3oeKxph4oX4 siwstuID4lW3fxT1zwzHMoh/+SmQj7/Ncpj8EjfCFJwEMfRWBIyngVJ6vIfLOpjGR3Ll p4f+Kr2UGG9nl7DY0sBGVBaV4BlqJbbCNutwin6Lg+IiIwkO0MYn433847dx/riCo5+0 fe2l2K36IpXATxrX2rDlJxkuxBBBmlvwa7kvwYl854uvf2F50FmwEb5YLbkrp2AMVSSL Ku4g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx3-rdu2.redhat.com. [66.187.233.73]) by mx.google.com with ESMTPS id r8-v6si7578455qvm.272.2018.06.09.05.30.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 09 Jun 2018 05:30:54 -0700 (PDT) Received-SPF: pass (google.com: domain of ming.lei@redhat.com designates 66.187.233.73 as permitted sender) client-ip=66.187.233.73; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 66.187.233.73 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E9B844022407; Sat, 9 Jun 2018 12:30:53 +0000 (UTC) Received: from localhost (ovpn-12-40.pek2.redhat.com [10.72.12.40]) by smtp.corp.redhat.com (Postfix) with ESMTP id 34B4C200BCEB; Sat, 9 Jun 2018 12:30:44 +0000 (UTC) From: Ming Lei To: Jens Axboe , Christoph Hellwig , Alexander Viro , Kent Overstreet Cc: David Sterba , Huang Ying , linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Theodore Ts'o , "Darrick J . Wong" , Coly Li , Filipe Manana , Randy Dunlap , Christoph Hellwig Subject: [PATCH V6 01/30] block: simplify bio_check_pages_dirty Date: Sat, 9 Jun 2018 20:29:45 +0800 Message-Id: <20180609123014.8861-2-ming.lei@redhat.com> In-Reply-To: <20180609123014.8861-1-ming.lei@redhat.com> References: <20180609123014.8861-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.4 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Sat, 09 Jun 2018 12:30:54 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.6]); Sat, 09 Jun 2018 12:30:54 +0000 (UTC) for IP:'10.11.54.4' DOMAIN:'int-mx04.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'ming.lei@redhat.com' RCPT:'' X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Christoph Hellwig bio_check_pages_dirty currently inviolates the invariant that bv_page of a bio_vec inside bi_vcnt shouldn't be zero, and that is going to become really annoying with multpath biovecs. Fortunately there isn't any all that good reason for it - once we decide to defer freeing the bio to a workqueue holding onto a few additional pages isn't really an issue anymore. So just check if there is a clean page that needs dirtying in the first path, and do a second pass to free them if there was none, while the cache is still hot. Also use the chance to micro-optimize bio_dirty_fn a bit by not saving irq state - we know we are called from a workqueue. Reviewed-by: Ming Lei Signed-off-by: Christoph Hellwig --- block/bio.c | 56 +++++++++++++++++++++----------------------------------- 1 file changed, 21 insertions(+), 35 deletions(-) diff --git a/block/bio.c b/block/bio.c index 5f7563598b1c..3e7d117c3346 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1647,19 +1647,15 @@ static void bio_release_pages(struct bio *bio) struct bio_vec *bvec; int i; - bio_for_each_segment_all(bvec, bio, i) { - struct page *page = bvec->bv_page; - - if (page) - put_page(page); - } + bio_for_each_segment_all(bvec, bio, i) + put_page(bvec->bv_page); } /* * bio_check_pages_dirty() will check that all the BIO's pages are still dirty. * If they are, then fine. If, however, some pages are clean then they must * have been written out during the direct-IO read. So we take another ref on - * the BIO and the offending pages and re-dirty the pages in process context. + * the BIO and re-dirty the pages in process context. * * It is expected that bio_check_pages_dirty() will wholly own the BIO from * here on. It will run one put_page() against each page and will run one @@ -1677,52 +1673,42 @@ static struct bio *bio_dirty_list; */ static void bio_dirty_fn(struct work_struct *work) { - unsigned long flags; - struct bio *bio; + struct bio *bio, *next; - spin_lock_irqsave(&bio_dirty_lock, flags); - bio = bio_dirty_list; + spin_lock_irq(&bio_dirty_lock); + next = bio_dirty_list; bio_dirty_list = NULL; - spin_unlock_irqrestore(&bio_dirty_lock, flags); + spin_unlock_irq(&bio_dirty_lock); - while (bio) { - struct bio *next = bio->bi_private; + while ((bio = next) != NULL) { + next = bio->bi_private; bio_set_pages_dirty(bio); bio_release_pages(bio); bio_put(bio); - bio = next; } } void bio_check_pages_dirty(struct bio *bio) { struct bio_vec *bvec; - int nr_clean_pages = 0; + unsigned long flags; int i; bio_for_each_segment_all(bvec, bio, i) { - struct page *page = bvec->bv_page; - - if (PageDirty(page) || PageCompound(page)) { - put_page(page); - bvec->bv_page = NULL; - } else { - nr_clean_pages++; - } + if (!PageDirty(bvec->bv_page) && !PageCompound(bvec->bv_page)) + goto defer; } - if (nr_clean_pages) { - unsigned long flags; - - spin_lock_irqsave(&bio_dirty_lock, flags); - bio->bi_private = bio_dirty_list; - bio_dirty_list = bio; - spin_unlock_irqrestore(&bio_dirty_lock, flags); - schedule_work(&bio_dirty_work); - } else { - bio_put(bio); - } + bio_release_pages(bio); + bio_put(bio); + return; +defer: + spin_lock_irqsave(&bio_dirty_lock, flags); + bio->bi_private = bio_dirty_list; + bio_dirty_list = bio; + spin_unlock_irqrestore(&bio_dirty_lock, flags); + schedule_work(&bio_dirty_work); } EXPORT_SYMBOL_GPL(bio_check_pages_dirty);