From patchwork Sun Jan 16 12:18:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12714552 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87B20C433FE for ; Sun, 16 Jan 2022 12:38:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7DE306B0074; Sun, 16 Jan 2022 07:38:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 764B46B0075; Sun, 16 Jan 2022 07:38:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6544C6B0078; Sun, 16 Jan 2022 07:38:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0153.hostedemail.com [216.40.44.153]) by kanga.kvack.org (Postfix) with ESMTP id 560F96B0074 for ; Sun, 16 Jan 2022 07:38:08 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 10DF6944FD for ; Sun, 16 Jan 2022 12:38:08 +0000 (UTC) X-FDA: 79036102656.21.C07B0FD Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf30.hostedemail.com (Postfix) with ESMTP id 94E3080004 for ; Sun, 16 Jan 2022 12:38:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=xWU6598e4aJ6j8LBEVFg/+CSnDbanOj2pq/9yym6seM=; b=rbCpPuUDrtplvD2Re5I4f0vZpA U64QaNgvTkA4w7eHgfz8wtBAHipbF1+rombZMLPZ0A0OpcrMIhbjhJi+/wqXOjtE4HCQXb0Tp1PAt s0cihpaRBiw3iAZ5b/GDNR2di8Wo8QfKwG9AkbJKJJyiSJiMXUp0OIs5VuZhEhaz6KyIDcknXkq2X Md/yv9peIyIAKv4n4m9E5wGIjb6KtQiQuqYWD7FoW92q6OLgQGm9bF2XQTi22bSyjQFIatQbOwK4/ yvtahBz8MDpJjdQMLu/6U409X+RBe4eEwsc+5gy/UVgoJE1WxLOIhiV+TnmZeEKOhnj1+4oxpQUNM 0/FFx7JQ==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1n94UM-007FUF-Af; Sun, 16 Jan 2022 12:18:26 +0000 From: "Matthew Wilcox (Oracle)" To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" Subject: [PATCH 03/12] filemap: Allow large folios to be added to the page cache Date: Sun, 16 Jan 2022 12:18:13 +0000 Message-Id: <20220116121822.1727633-4-willy@infradead.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220116121822.1727633-1-willy@infradead.org> References: <20220116121822.1727633-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 94E3080004 X-Stat-Signature: m1s4dne1ndu57f3nocfqr19na3uqywfd Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=rbCpPuUD; dmarc=none; spf=none (imf30.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Rspamd-Server: rspam08 X-HE-Tag: 1642336687-174636 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We return -EEXIST if there are any non-shadow entries in the page cache in the range covered by the folio. If there are multiple shadow entries in the range, we set *shadowp to one of them (currently the one at the highest index). If that turns out to be the wrong answer, we can implement something more complex. This is mostly modelled after the equivalent function in the shmem code. Signed-off-by: Matthew Wilcox (Oracle) --- mm/filemap.c | 39 ++++++++++++++++++++++----------------- 1 file changed, 22 insertions(+), 17 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index afc8f5ca85ac..fe079b676ab7 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -851,26 +851,27 @@ noinline int __filemap_add_folio(struct address_space *mapping, { XA_STATE(xas, &mapping->i_pages, index); int huge = folio_test_hugetlb(folio); - int error; bool charged = false; + long nr = 1; VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); VM_BUG_ON_FOLIO(folio_test_swapbacked(folio), folio); mapping_set_update(&xas, mapping); - folio_get(folio); - folio->mapping = mapping; - folio->index = index; - if (!huge) { - error = mem_cgroup_charge(folio, NULL, gfp); + int error = mem_cgroup_charge(folio, NULL, gfp); VM_BUG_ON_FOLIO(index & (folio_nr_pages(folio) - 1), folio); if (error) - goto error; + return error; charged = true; + xas_set_order(&xas, index, folio_order(folio)); + nr = folio_nr_pages(folio); } gfp &= GFP_RECLAIM_MASK; + folio_ref_add(folio, nr); + folio->mapping = mapping; + folio->index = xas.xa_index; do { unsigned int order = xa_get_order(xas.xa, xas.xa_index); @@ -894,6 +895,8 @@ noinline int __filemap_add_folio(struct address_space *mapping, /* entry may have been split before we acquired lock */ order = xa_get_order(xas.xa, xas.xa_index); if (order > folio_order(folio)) { + /* How to handle large swap entries? */ + BUG_ON(shmem_mapping(mapping)); xas_split(&xas, old, order); xas_reset(&xas); } @@ -903,29 +906,31 @@ noinline int __filemap_add_folio(struct address_space *mapping, if (xas_error(&xas)) goto unlock; - mapping->nrpages++; + mapping->nrpages += nr; /* hugetlb pages do not participate in page cache accounting */ - if (!huge) - __lruvec_stat_add_folio(folio, NR_FILE_PAGES); + if (!huge) { + __lruvec_stat_mod_folio(folio, NR_FILE_PAGES, nr); + if (folio_test_pmd_mappable(folio)) + __lruvec_stat_mod_folio(folio, + NR_FILE_THPS, nr); + } unlock: xas_unlock_irq(&xas); } while (xas_nomem(&xas, gfp)); - if (xas_error(&xas)) { - error = xas_error(&xas); - if (charged) - mem_cgroup_uncharge(folio); + if (xas_error(&xas)) goto error; - } trace_mm_filemap_add_to_page_cache(folio); return 0; error: + if (charged) + mem_cgroup_uncharge(folio); folio->mapping = NULL; /* Leave page->index set: truncation relies upon it */ - folio_put(folio); - return error; + folio_put_refs(folio, nr); + return xas_error(&xas); } ALLOW_ERROR_INJECTION(__filemap_add_folio, ERRNO);