From patchwork Mon Jun 11 14:06:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 10458035 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id DB396601A0 for ; Mon, 11 Jun 2018 14:12:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C779828505 for ; Mon, 11 Jun 2018 14:12:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B219928832; Mon, 11 Jun 2018 14:12:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE, T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A834B28712 for ; Mon, 11 Jun 2018 14:11:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A826A6B02A5; Mon, 11 Jun 2018 10:07:16 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A07026B02A6; Mon, 11 Jun 2018 10:07:16 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 882626B02A7; Mon, 11 Jun 2018 10:07:16 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl0-f69.google.com (mail-pl0-f69.google.com [209.85.160.69]) by kanga.kvack.org (Postfix) with ESMTP id 43A496B02A5 for ; Mon, 11 Jun 2018 10:07:16 -0400 (EDT) Received: by mail-pl0-f69.google.com with SMTP id y7-v6so12149029plt.17 for ; Mon, 11 Jun 2018 07:07:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=9kvEcFed1e0NqjRoLrLQEEJ3vX9NUobzEC8Ag4KiSOc=; b=c/PO6IVgIY3ui92qHF4jSft0KYzLugKmSx4WtIqrqPQmdgrt+39DDy8goLJwS471Js ks8Y6koh+sX33UTsvf1esKDrXo21W9NA6PTXDy9ubBBd3dbvdQrjVXHSuVR3XX1zGznB X4kLgk/RhGyb7ph/R+JwL/TtTZyGJl+HemrDsUiU3f8LofGLqowfWPdEK4X+1LVsXAfx 3myIxMU0dLiYFVL1UEHLTf6QQQg/VDOgGPlh+7kgBJgm8GUhSRCXvdsTEu4J1O7+U116 vliWdTH94mziHcuVNiuSS/N3861kqMVJik9TfouBh8Hn21f9dERgRHvrnZ0vTLvJAcN/ nayg== X-Gm-Message-State: APt69E0aMIO3YwygEX9V3pT7F47ukZ3Cc354mhhIQVVq7nWrPhEpTcE/ vabf9LzEoRWHSMVDVFcOvRHlimTUTBPs+cNNEd72jOihpK71zvDrMMljVd9R6OG3QOGH3sTAnHQ o6rNtl8u5X77T0TeSs9LjXhuAgfOr9cZU/PjPyqyk7J1bCYoZYUEjqAEqgklrekFQqQ== X-Received: by 2002:a62:d6d2:: with SMTP id a79-v6mr17923537pfl.87.1528726035898; Mon, 11 Jun 2018 07:07:15 -0700 (PDT) X-Google-Smtp-Source: ADUXVKIUjGwM3gF/DmdydkmjtaEQd8xjITRn5o6eP1MbDy3xRtNXmL29iL6ViqcaotZGR9m0lH1g X-Received: by 2002:a62:d6d2:: with SMTP id a79-v6mr17923462pfl.87.1528726034857; Mon, 11 Jun 2018 07:07:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1528726034; cv=none; d=google.com; s=arc-20160816; b=ye3vq1rOT64Ywpg822MZyqLomnuQpngWw6OqOwsUb1Bm1gDIwNUBl4JZgyRo9zuUjw BpmZfmaXvk93WQhiAKIp+TLXwraT1O4IKbN39ap7RW4nEd5E42z7/sm8DwYqVE9zeSQv EX5P3tZ6LuVibERT9IHsGkzMh9UeDFwYjaF4sREhwsbfpg6vhqwLDm8EDSM0B30+5a29 VE9T+JdybczjjAFqYIS0BkQ2hf97zayHJVnZovgeaSeoisvAL8LKO6fZJ6KO23OQaAlw Xn0brBtzQhvA1IF4+rd8he+cYuwlirsTZImwEQTl8ow5gch7YHbBb6x/ImIUnNLM2vYG O98Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=9kvEcFed1e0NqjRoLrLQEEJ3vX9NUobzEC8Ag4KiSOc=; b=t484HHJOwReLZxGXph5aD7hRU9J9PvSiyl1vZFcsb9yqSro9mpVCofwpSXP8S1KYAN M1QC56kUpG6nxEh3fYUWV/DC/gF/vXkpgf/H1brNmLbGHXM2g3g6JgdYlZ36W/B3qTJc wColUdWUqtF4PNwjFaCmODpp3/T5feGznaliry7OdhlPfRv+yvi6fOYNk5EWpUuHOj8L 5iAD1qs9CqXjQUrOu1qY4yAYRjVtjZrLqpD0eCZZ/v6wYOFmRPhssNVBobwPoHiyVo+k Dv5LwVf24TAYQLHStRaqgsnRb4YA72aIIzyG9QfR9bqIhsEEBA+Tp0GIueUA40rpeibe X9iA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=bombadil.20170209 header.b=OW50G6uv; spf=pass (google.com: best guess record for domain of willy@infradead.org designates 2607:7c80:54:e::133 as permitted sender) smtp.mailfrom=willy@infradead.org Received: from bombadil.infradead.org (bombadil.infradead.org. [2607:7c80:54:e::133]) by mx.google.com with ESMTPS id b7-v6si4120783pgr.471.2018.06.11.07.07.14 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 11 Jun 2018 07:07:14 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of willy@infradead.org designates 2607:7c80:54:e::133 as permitted sender) client-ip=2607:7c80:54:e::133; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=bombadil.20170209 header.b=OW50G6uv; spf=pass (google.com: best guess record for domain of willy@infradead.org designates 2607:7c80:54:e::133 as permitted sender) smtp.mailfrom=willy@infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=9kvEcFed1e0NqjRoLrLQEEJ3vX9NUobzEC8Ag4KiSOc=; b=OW50G6uvHRsxuqJTjyI8nNYin UHqOFwcsToQPa2GiU1bI6lXroVAtNuttUkYwp7wZca32mHiFfToDFujy4T0847Eca36va9QBKmyWo nXSV82qoydhw5KkMtmS+5io/qfI7sP2HtcP5Q4cVp7vL5OoQQYEevMDrxfJr8khf+BzynuIDnLYkT B6dU4WdZypnIRw6NaTBJsQN+Jbil0GcljNuYqgQ7CnwC6IfN47MLlsVStTzG5uhVbTnrP30MQVNf9 1iizoexkH/WPHpUinEmE44f20HUMsV+lehv7JdEhUKGp6KUSp//Xg2evu0iODUy/OF53U33YKOCba eJ1e5aGQw==; Received: from willy by bombadil.infradead.org with local (Exim 4.90_1 #2 (Red Hat Linux)) id 1fSNTV-00053T-96; Mon, 11 Jun 2018 14:07:13 +0000 From: Matthew Wilcox To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Matthew Wilcox , Jan Kara , Jeff Layton , Lukas Czerner , Ross Zwisler , Christoph Hellwig , Goldwyn Rodrigues , Nicholas Piggin , Ryusuke Konishi , linux-nilfs@vger.kernel.org, Jaegeuk Kim , Chao Yu , linux-f2fs-devel@lists.sourceforge.net Subject: [PATCH v13 66/72] dax: Convert dax writeback to XArray Date: Mon, 11 Jun 2018 07:06:33 -0700 Message-Id: <20180611140639.17215-67-willy@infradead.org> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180611140639.17215-1-willy@infradead.org> References: <20180611140639.17215-1-willy@infradead.org> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Matthew Wilcox Use XArray iteration instead of a pagevec. Signed-off-by: Matthew Wilcox --- fs/dax.c | 129 ++++++++++++++++++++++++++----------------------------- 1 file changed, 61 insertions(+), 68 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index 0566a150c458..f35913b8c5bb 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -897,11 +897,9 @@ static void dax_entry_mkclean(struct address_space *mapping, pgoff_t index, i_mmap_unlock_read(mapping); } -static int dax_writeback_one(struct dax_device *dax_dev, - struct address_space *mapping, pgoff_t index, void *entry) +static int dax_writeback_one(struct xa_state *xas, struct dax_device *dax_dev, + struct address_space *mapping, void *entry) { - struct radix_tree_root *pages = &mapping->i_pages; - void *entry2, **slot; unsigned long pfn; long ret = 0; size_t size; @@ -913,29 +911,35 @@ static int dax_writeback_one(struct dax_device *dax_dev, if (WARN_ON(!xa_is_value(entry))) return -EIO; - xa_lock_irq(pages); - entry2 = get_unlocked_mapping_entry(mapping, index, &slot); - /* Entry got punched out / reallocated? */ - if (!entry2 || WARN_ON_ONCE(!xa_is_value(entry2))) - goto put_unlocked; - /* - * Entry got reallocated elsewhere? No need to writeback. We have to - * compare pfns as we must not bail out due to difference in lockbit - * or entry type. - */ - if (dax_to_pfn(entry2) != dax_to_pfn(entry)) - goto put_unlocked; - if (WARN_ON_ONCE(dax_is_empty_entry(entry) || - dax_is_zero_entry(entry))) { - ret = -EIO; - goto put_unlocked; + if (unlikely(dax_is_locked(entry))) { + void *old_entry = entry; + + entry = get_unlocked_entry(xas); + + /* Entry got punched out / reallocated? */ + if (!entry || WARN_ON_ONCE(!xa_is_value(entry))) + goto put_unlocked; + /* + * Entry got reallocated elsewhere? No need to writeback. + * We have to compare pfns as we must not bail out due to + * difference in lockbit or entry type. + */ + if (dax_to_pfn(old_entry) != dax_to_pfn(entry)) + goto put_unlocked; + if (WARN_ON_ONCE(dax_is_empty_entry(entry) || + dax_is_zero_entry(entry))) { + ret = -EIO; + goto put_unlocked; + } + + /* Another fsync thread may have already done this entry */ + if (!xas_get_tag(xas, PAGECACHE_TAG_TOWRITE)) + goto put_unlocked; } - /* Another fsync thread may have already written back this entry */ - if (!radix_tree_tag_get(pages, index, PAGECACHE_TAG_TOWRITE)) - goto put_unlocked; /* Lock the entry to serialize with page faults */ - entry = lock_slot(mapping, slot); + dax_lock_entry(xas, entry); + /* * We can clear the tag now but we have to be careful so that concurrent * dax_writeback_one() calls for the same index cannot finish before we @@ -943,8 +947,8 @@ static int dax_writeback_one(struct dax_device *dax_dev, * at the entry only under the i_pages lock and once they do that * they will see the entry locked and wait for it to unlock. */ - radix_tree_tag_clear(pages, index, PAGECACHE_TAG_TOWRITE); - xa_unlock_irq(pages); + xas_clear_tag(xas, PAGECACHE_TAG_TOWRITE); + xas_unlock_irq(xas); /* * Even if dax_writeback_mapping_range() was given a wbc->range_start @@ -956,7 +960,7 @@ static int dax_writeback_one(struct dax_device *dax_dev, pfn = dax_to_pfn(entry); size = PAGE_SIZE << dax_entry_order(entry); - dax_entry_mkclean(mapping, index, pfn); + dax_entry_mkclean(mapping, xas->xa_index, pfn); dax_flush(dax_dev, page_address(pfn_to_page(pfn)), size); /* * After we have flushed the cache, we can clear the dirty tag. There @@ -964,16 +968,17 @@ static int dax_writeback_one(struct dax_device *dax_dev, * the pfn mappings are writeprotected and fault waits for mapping * entry lock. */ - xa_lock_irq(pages); - radix_tree_tag_clear(pages, index, PAGECACHE_TAG_DIRTY); - xa_unlock_irq(pages); - trace_dax_writeback_one(mapping->host, index, size >> PAGE_SHIFT); - put_locked_mapping_entry(mapping, index); + xas_reset(xas); + xas_lock_irq(xas); + xas_store(xas, entry); + xas_clear_tag(xas, PAGECACHE_TAG_DIRTY); + + trace_dax_writeback_one(mapping->host, xas->xa_index, + size >> PAGE_SHIFT); return ret; put_unlocked: - put_unlocked_mapping_entry(mapping, index, entry2); - xa_unlock_irq(pages); + put_unlocked_entry(xas, entry); return ret; } @@ -985,13 +990,13 @@ static int dax_writeback_one(struct dax_device *dax_dev, int dax_writeback_mapping_range(struct address_space *mapping, struct block_device *bdev, struct writeback_control *wbc) { + XA_STATE(xas, &mapping->i_pages, wbc->range_start >> PAGE_SHIFT); struct inode *inode = mapping->host; - pgoff_t start_index, end_index; - pgoff_t indices[PAGEVEC_SIZE]; + pgoff_t end_index = wbc->range_end >> PAGE_SHIFT; struct dax_device *dax_dev; - struct pagevec pvec; - bool done = false; - int i, ret = 0; + void *entry; + int ret = 0; + unsigned int scanned = 0; if (WARN_ON_ONCE(inode->i_blkbits != PAGE_SHIFT)) return -EIO; @@ -1003,41 +1008,29 @@ int dax_writeback_mapping_range(struct address_space *mapping, if (!dax_dev) return -EIO; - start_index = wbc->range_start >> PAGE_SHIFT; - end_index = wbc->range_end >> PAGE_SHIFT; - - trace_dax_writeback_range(inode, start_index, end_index); - - tag_pages_for_writeback(mapping, start_index, end_index); + trace_dax_writeback_range(inode, xas.xa_index, end_index); - pagevec_init(&pvec); - while (!done) { - pvec.nr = find_get_entries_tag(mapping, start_index, - PAGECACHE_TAG_TOWRITE, PAGEVEC_SIZE, - pvec.pages, indices); + tag_pages_for_writeback(mapping, xas.xa_index, end_index); - if (pvec.nr == 0) + xas_lock_irq(&xas); + xas_for_each_tagged(&xas, entry, end_index, PAGECACHE_TAG_TOWRITE) { + ret = dax_writeback_one(&xas, dax_dev, mapping, entry); + if (ret < 0) { + mapping_set_error(mapping, ret); break; - - for (i = 0; i < pvec.nr; i++) { - if (indices[i] > end_index) { - done = true; - break; - } - - ret = dax_writeback_one(dax_dev, mapping, indices[i], - pvec.pages[i]); - if (ret < 0) { - mapping_set_error(mapping, ret); - goto out; - } } - start_index = indices[pvec.nr - 1] + 1; + if (++scanned % XA_CHECK_SCHED) + continue; + + xas_pause(&xas); + xas_unlock_irq(&xas); + cond_resched(); + xas_lock_irq(&xas); } -out: + xas_unlock_irq(&xas); put_dax(dax_dev); - trace_dax_writeback_range_done(inode, start_index, end_index); - return (ret < 0 ? ret : 0); + trace_dax_writeback_range_done(inode, xas.xa_index, end_index); + return ret; } EXPORT_SYMBOL_GPL(dax_writeback_mapping_range);