From patchwork Wed Apr 24 19:19:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13642432 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB2E4C4345F for ; Wed, 24 Apr 2024 19:19:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CD5276B02E4; Wed, 24 Apr 2024 15:19:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C06066B02E6; Wed, 24 Apr 2024 15:19:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 98E7A6B02E7; Wed, 24 Apr 2024 15:19:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 6C62F6B02E4 for ; Wed, 24 Apr 2024 15:19:28 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 2BDE71A0392 for ; Wed, 24 Apr 2024 19:19:28 +0000 (UTC) X-FDA: 82045389216.07.8DAE281 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf28.hostedemail.com (Postfix) with ESMTP id 84795C0027 for ; Wed, 24 Apr 2024 19:19:26 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=kElKxAay; spf=none (imf28.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713986366; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tgI9zM0JkPeDD4nYChnEAx4Cne/BbcH1RdD5pnSOOA4=; b=LCjXK678ihHBjUytXfcpaEncRTQ6+J7um0p6ZmESvT3Pe8nTYsEdWRHn/QU5Ol/P/ds/yM HcdYBP86l56Qo8JeHd1fCsx2VbbtHvYg+xhmunrWdlU07j8csKOOCmnqryh6jKom5BMwOF YQH4MBC8LiwmmodbP4a0BRPQvbOkqx8= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=kElKxAay; spf=none (imf28.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713986366; a=rsa-sha256; cv=none; b=g6c7yY/8rw2Sbs9F+Us+uaygsxzJUK1xijqM9tdHijAulcqV3LVduHktGF7aekAvvhlNy1 OhE34U8tJzhyu19w4WjUMlLWDrr2XaW9nVnWpu/aMNw509A4Qa5p4SJDWZwXsajsIQJQ7Y 6kuVQh8mbM1PWB1lSw6BGrudtF/o32M= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=tgI9zM0JkPeDD4nYChnEAx4Cne/BbcH1RdD5pnSOOA4=; b=kElKxAayScXBOhZkoDtgupSZUi gytK2a9XgJphA+2faEzI72JuXm1zdf4z4pku+xvVeECnh6yRolhM+b8BSvSkh+KgUQTsHgGx2G01u pOq8h97Y4oBvBscNGaDzGr9EuSMmBM2ba5Td6KnXiAWdm0n8OcGBu3ILbZr2gUIhB45jVoKZs4kNI D4Ml/LVZL7qH59+tEKaoi0R5DiSaTPEDoZdwZQPcqNDAX23m24u3ZUY7Kxid6egekZzqvvsNgN2AY 5bchRr0BpuDbRujWeDs1Q375C5BVRe73scIZOOQOpAuzek27nyrNsiI52fYqiwdwtfFJDiqnZZigm 1HZKmapw==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rzi9E-00000001W3v-1Ixi; Wed, 24 Apr 2024 19:19:16 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org Subject: [PATCH 5/7] gup: Use folios for gup_devmap Date: Wed, 24 Apr 2024 20:19:10 +0100 Message-ID: <20240424191914.361554-6-willy@infradead.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240424191914.361554-1-willy@infradead.org> References: <20240424191914.361554-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 84795C0027 X-Stat-Signature: ssnk6it81hmrebjtdw7t3f1ni5pmzqnf X-Rspam-User: X-Rspamd-Server: rspam07 X-HE-Tag: 1713986366-701432 X-HE-Meta: U2FsdGVkX1/SbbYouo2q/XCnb9yhopPQBlVexNCy6vGW1vy4+XEIzYtTQJH6Ac+ckYG2YGOE2z/usTPqV0sQrYF45sJszd+1ozRINhBPeeugvWzOpB+X4qHJbFseoQxKhWMROWqtgOwcmb6cgFJ9cPUbbADSHjIQyxPTS859oK4kp5oUty4I1uvWZhD345R0aiAYZOCM7gEGzvLPueOEkOe1MxK82THLZa18pKWMfD2E3yoWLOv6GZg9jVMkEjUYdN362l8ve07YSl319o0h78XeXLc4mEDY3LzXMKGJ2eQiCIPXA5l+gF17hcLyGU3a6fTdYTlUqtovu0T+tg+P6hKVPVin/vQEMBXSswNMtaOJxXUtOPdpeCVpDozjUQza1iNFiA9pFNqBCpyYLBU+2u9AugY6ZTlNS18SYOJC+xJCUn2ZiUiOmiGNN2knDUqqdGYmymuew93DAmb4H/WOjEtkX4kLsC+fQTwqGsxgqNyFVbX9k2B/uOMd9E62Ex/+uBS7PZz93x/pKTkSxVv3W4dDbgwqoffTI35VNCaS++gfC+f4LiNdlr+uOjAlP2u8kLuIaFaQ35L03y5uHMFHzlX25OFvKNVA66qq8fat0d2UtKulBEvtSJ9LhdksPx1HncHPqO8dk1SzDE+kWlNFW4OQgHha+sXYZv0ah+3jHEQIjlGQguEagKQt8niibP9nBv9DqDTdumT1WFRQfOiy+c3uiCZeyusO6JBJOg5RuylpWKe2FoepuXopZvti9xHtC9qyks15hNizg9TR/ltCb6gySnaaGidXJwJX9+BWCzu7gDrWfTvK1tEX6bLVx5BEEG6v+fzHvTNrQfvPjdCXGNCIM4Jona73Nyxn2M5M9hZ2KyQZQD9zemHAryaZw60L2nF++gZz9ZLIzkWDvbGvUDJXp3C5DzRR9aNARVMdRt2Uy1qdeB9pHzmmjssj8X03evTufX3xDb8jt2+wx/Z 00eIy43h SgKl4Xc4x444iJQ4MP1wdfwf2QZnSjrHVO+FuiOjULCagtuSndJ1ubpp5GBlXd05cYWondI6KEuxsYUOd4uEzrMj7KHdD0RRve80yvRUcxdQOAvCe36+RKPnPGS9kroxbQ1LjcIGtNcw/b4VH5EpXy8mtg4Xu9eANVL+HjijqmplUeprrCUyWQoO4YqKeKDcCcIkp24YwNTiQdrhu4Bz+J1oqE7KUZuxukKyn X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use try_grab_folio() instead of try_grab_page() so we get the folio back that we calculated, and then use folio_set_referenced() instead of SetPageReferenced(). Correspondingly, use gup_put_folio() to put any unneeded references. Signed-off-by: Matthew Wilcox (Oracle) --- mm/gup.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 49376f756936..e4cc12b8e985 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2877,13 +2877,10 @@ static void __maybe_unused gup_fast_undo_dev_pagemap(int *nr, int nr_start, unsigned int flags, struct page **pages) { while ((*nr) - nr_start) { - struct page *page = pages[--(*nr)]; + struct folio *folio = page_folio(pages[--(*nr)]); - ClearPageReferenced(page); - if (flags & FOLL_PIN) - unpin_user_page(page); - else - put_page(page); + folio_clear_referenced(folio); + gup_put_folio(folio, 1, flags); } } @@ -3024,6 +3021,7 @@ static int gup_fast_devmap_leaf(unsigned long pfn, unsigned long addr, struct dev_pagemap *pgmap = NULL; do { + struct folio *folio; struct page *page = pfn_to_page(pfn); pgmap = get_dev_pagemap(pfn, pgmap); @@ -3037,12 +3035,13 @@ static int gup_fast_devmap_leaf(unsigned long pfn, unsigned long addr, break; } - SetPageReferenced(page); - pages[*nr] = page; - if (unlikely(try_grab_page(page, flags))) { + folio = try_grab_folio(page, 1, flags); + if (!folio) { gup_fast_undo_dev_pagemap(nr, nr_start, flags, pages); break; } + folio_set_referenced(folio); + pages[*nr] = page; (*nr)++; pfn++; } while (addr += PAGE_SIZE, addr != end);