From patchwork Sat Jan 13 06:52:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kasireddy, Vivek" X-Patchwork-Id: 13518878 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1331CC4707C for ; Sat, 13 Jan 2024 07:16:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 43FCA6B0087; Sat, 13 Jan 2024 02:16:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3CA056B0085; Sat, 13 Jan 2024 02:16:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 243926B008C; Sat, 13 Jan 2024 02:16:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 0E14F6B0085 for ; Sat, 13 Jan 2024 02:16:43 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id D516C1201B5 for ; Sat, 13 Jan 2024 07:16:42 +0000 (UTC) X-FDA: 81673430244.25.08BF76D Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.11]) by imf10.hostedemail.com (Postfix) with ESMTP id C67AAC0005 for ; Sat, 13 Jan 2024 07:16:40 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="Ujq/3sQV"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf10.hostedemail.com: domain of vivek.kasireddy@intel.com designates 192.198.163.11 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705130201; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=51RyVHcz8ejXQz+KmM5laDmGCKyn7tUIikYMRLagD4I=; b=6/+ugbpfvlWqPIk9nZrs4Dt0si/lC2TUSdZiqmwbhKY8v+2OggnawP+hrfe6YNk95DTXyC 0MEF85z6VucUCfIMUTWrc9L8Od4BuCNxvaRrsnhpUtmlwPSUfvwRM8U+QO5f/PFiP9TLHA 3nld87fg8zDnxqI6RGiJSwWgNEy43fA= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="Ujq/3sQV"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf10.hostedemail.com: domain of vivek.kasireddy@intel.com designates 192.198.163.11 as permitted sender) smtp.mailfrom=vivek.kasireddy@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705130201; a=rsa-sha256; cv=none; b=3ud2lm9F9u2B7haQnuukZ7cmbje8POlTPcySgDnjfPSUc4G9EtSsoocCX8rL+S0ne4mnYh KeYBzqZNjrIR9TriXhAfHDCggfPIdsFBop3/iIcfjo1Dv5scEmmzki9SKQrJ/Q0UKUokt2 TrJjMg8MCEh5VTSCQIk74HUbzN3ixGY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1705130201; x=1736666201; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Mnov74aVUnZ/JsntlOzaB+N14p4KDWATuD5/BakBI78=; b=Ujq/3sQVdkIolRfh+0+8tUTt0FoyuGDGVNAj5Y3taPSPthppBteG35hl P1dfenSeX9WkaG4uV7S6nHjXn3eZxLYAIXWGtG2fPs42GynNEUTfM3Xri WhGrPu+ZVfr/tDOmL0nkJD0syGtWbrVhvj5bWTzcc20Ed/+8FtiiQ3OdH f/bjSf9FyRk9TUGFKLEeZxB+mMRwCWHc3GuZQrVhKMK8AXv2tV25CYwBc u4JfjlBIAppa7ZZrzU0NbvKhmG43RUr3sJQ5jwr+3/Cua9nbd2z5Jfn3q geV9tpwD5NKdkJijBfvv+1iWW4MM/Vvmn7HL6NVteE1yzr69+v03Y1lwJ Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10951"; a="6078114" X-IronPort-AV: E=Sophos;i="6.04,191,1695711600"; d="scan'208";a="6078114" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmvoesa105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jan 2024 23:16:38 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10951"; a="783269487" X-IronPort-AV: E=Sophos;i="6.04,191,1695711600"; d="scan'208";a="783269487" Received: from vkasired-desk2.fm.intel.com ([10.105.128.132]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jan 2024 23:16:37 -0800 From: Vivek Kasireddy To: dri-devel@lists.freedesktop.org, linux-mm@kvack.org Cc: Vivek Kasireddy , David Hildenbrand , Matthew Wilcox , Christoph Hellwig , Jason Gunthorpe , Peter Xu Subject: [PATCH v11 1/8] mm/gup: Introduce unpin_folio/unpin_folios helpers Date: Fri, 12 Jan 2024 22:52:16 -0800 Message-Id: <20240113065223.1532987-2-vivek.kasireddy@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240113065223.1532987-1-vivek.kasireddy@intel.com> References: <20240113065223.1532987-1-vivek.kasireddy@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: C67AAC0005 X-Stat-Signature: b58cakcbajjqtnencmhw8k5mtca48yp6 X-Rspam-User: X-HE-Tag: 1705130200-715081 X-HE-Meta: U2FsdGVkX1/UvfKahsZmM1/HaFoFDdiV2WRdvDM/enPbNa2mPXTynrqzxrEi7zKgrj93fl30sexCEVTZeKleTf3o2tsvxqDFPIMXandDgCq1nZCrePMvoKq0FYOVSXsSrYCUQQ3Cp+RgII6NS44nBRdiLPE00gr7MghJfhZfbaucwS/YqIMiZfiJQGCXy4eo+zyxrlvuku8sTSOW2eRfHF2wzcRbwUQ/BMqmszR0MgOxaaYsY5fxSoXDxMjmbg4uX1v2bHSNT83Su9hgJDTlicsT5DLPSihTf/aGsZ7RPAgH+QG7RVr59HZmXkhNHDA2H9YedeBc9mxZ6t8qctFshLxST1HysqCFvq0YaQisY9h4Y27xTNT40XcCXXrKiYe+V/0Ip0ugEaNjjVMfXz3Cxeu9pMknzv4SMih9sFu9bQOodxQGo4ZrgYDGjeZgFJKo+iRzZ3DrXCHfbUvI3sXjFz9zSckrhlOdgRPYIdA0Gc1tUyUwwJnpS7SjKLIsbVezKjIlBjnwlTTiTDozfztReNYKKYUVmKh+OXfgsaoL7E+Wv9Et+PxpFGI/tJMpTBYzPhqyZ1+pZW4IYdnjP9sJGjwqZ6vSdEgnV93u+ADVdDCh7Pq4lAq+nKinUJzpQHyeB9NOpXnNaOuYEIw5UnWNhhhf3g/bMIiDwYywXtq1hSF6kT9D2jX6fIfluGuNp8+tboupYEcwiD/ib34K+pk4P/kSOg5hXOs3XxouRrgOhMhDSXEhEpJTc8BU1UQ5cXsuWHzFeFxfZqZYX44XRYeR72F8FtxWUedSjeJLi1VyyldFOK/P4RKedPL+pHcOMZBf4O83kpB4sdgxcyV3cT8ZyXl5BUznGK9mWtKs8X5tj8+toaKiXkVbNCCfq+J53cZKq+babGtDmhr4v52E+62a/q46mUhSZ9N7mWeNuSQcyVLqyiSuPJR82xpH70neApUcX2mMUfOMP7r4QeALomw f7YjhAuD JTxf3ho07HaP8CQeqHjZR4375d6Wle0+/ppnJGklvsa8GJjPuEAQVAAQ3o6ufJkQvVA7wPA2Crq7e5YToXFTz/S8zrJfmx7M6yJ45p9esxrZcCLZA+xeG4rXzubCj0ZyzhRbzLT1Ja8TFsfhGu7t7PV/ft+msoVoM3m4OohrAbvetl5lyFvk0sgjeeNUyyL/tQlFXJV8RUrBG4Ia/ZacPwoidknoKFxf7SM6qJ+NlOp33T+MPVyZr9Thq5yRjwDocuV2gwn0kyUrc/KKdnseWVLTUjMGmACmwZZ487EE0xOZguZl6mmVBB8hfVw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: These helpers are the folio versions of unpin_user_page/unpin_user_pages. They are currently only useful for unpinning folios pinned by memfd_pin_folios() or other associated routines. However, they could find new uses in the future, when more and more folio-only helpers are added to GUP. Cc: David Hildenbrand Cc: Matthew Wilcox Cc: Christoph Hellwig Cc: Jason Gunthorpe Cc: Peter Xu Suggested-by: David Hildenbrand Signed-off-by: Vivek Kasireddy --- include/linux/mm.h | 2 ++ mm/gup.c | 81 ++++++++++++++++++++++++++++++++++++++++------ 2 files changed, 74 insertions(+), 9 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index da5219b48d52..a0321adaefad 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1585,11 +1585,13 @@ static inline void put_page(struct page *page) #define GUP_PIN_COUNTING_BIAS (1U << 10) void unpin_user_page(struct page *page); +void unpin_folio(struct folio *folio); void unpin_user_pages_dirty_lock(struct page **pages, unsigned long npages, bool make_dirty); void unpin_user_page_range_dirty_lock(struct page *page, unsigned long npages, bool make_dirty); void unpin_user_pages(struct page **pages, unsigned long npages); +void unpin_folios(struct folio **folios, unsigned long nfolios); static inline bool is_cow_mapping(vm_flags_t flags) { diff --git a/mm/gup.c b/mm/gup.c index 231711efa390..4d7bc4453819 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -30,6 +30,23 @@ struct follow_page_context { unsigned int page_mask; }; +static inline void sanity_check_pinned_folios(struct folio **folios, + unsigned long nfolios) +{ + if (!IS_ENABLED(CONFIG_DEBUG_VM)) + return; + + for (; nfolios; nfolios--, folios++) { + struct folio *folio = *folios; + + if (is_zero_folio(folio) || + !folio_test_anon(folio)) + continue; + + VM_BUG_ON_FOLIO(!PageAnonExclusive(&folio->page), folio); + } +} + static inline void sanity_check_pinned_pages(struct page **pages, unsigned long npages) { @@ -52,15 +69,11 @@ static inline void sanity_check_pinned_pages(struct page **pages, struct page *page = *pages; struct folio *folio = page_folio(page); - if (is_zero_page(page) || - !folio_test_anon(folio)) - continue; - if (!folio_test_large(folio) || folio_test_hugetlb(folio)) - VM_BUG_ON_PAGE(!PageAnonExclusive(&folio->page), page); - else - /* Either a PTE-mapped or a PMD-mapped THP. */ - VM_BUG_ON_PAGE(!PageAnonExclusive(&folio->page) && - !PageAnonExclusive(page), page); + sanity_check_pinned_folios(&folio, 1); + + /* Either a PTE-mapped or a PMD-mapped THP. */ + if (folio_test_large(folio) && !folio_test_hugetlb(folio)) + VM_BUG_ON_PAGE(!PageAnonExclusive(page), page); } } @@ -276,6 +289,21 @@ void unpin_user_page(struct page *page) } EXPORT_SYMBOL(unpin_user_page); +/** + * unpin_folio() - release a dma-pinned folio + * @folio: pointer to folio to be released + * + * Folios that were pinned via memfd_pin_folios() or other similar routines + * must be released either using unpin_folio() or unpin_folios(). This is so + * that such folios can be separately tracked and uniquely handled. + */ +void unpin_folio(struct folio *folio) +{ + sanity_check_pinned_folios(&folio, 1); + gup_put_folio(folio, 1, FOLL_PIN); +} +EXPORT_SYMBOL(unpin_folio); + /** * folio_add_pin - Try to get an additional pin on a pinned folio * @folio: The folio to be pinned @@ -488,6 +516,41 @@ void unpin_user_pages(struct page **pages, unsigned long npages) } EXPORT_SYMBOL(unpin_user_pages); +/** + * unpin_folios() - release an array of gup-pinned folios. + * @folios: array of folios to be marked dirty and released. + * @nfolios: number of folios in the @folios array. + * + * For each folio in the @folios array, release the folio using unpin_folio(). + * + * Please see the unpin_folio() documentation for details. + */ +void unpin_folios(struct folio **folios, unsigned long nfolios) +{ + unsigned long i = 0, j; + + /* + * If this WARN_ON() fires, then the system *might* be leaking folios + * (by leaving them pinned), but probably not. More likely, gup/pup + * returned a hard -ERRNO error to the caller, who erroneously passed + * it here. + */ + if (WARN_ON(IS_ERR_VALUE(nfolios))) + return; + + sanity_check_pinned_folios(folios, nfolios); + while (i < nfolios) { + for (j = i + 1; j < nfolios; j++) + if (folios[i] != folios[j]) + break; + + if (folios[i]) + gup_put_folio(folios[i], j - i, FOLL_PIN); + i = j; + } +} +EXPORT_SYMBOL(unpin_folios); + /* * Set the MMF_HAS_PINNED if not set yet; after set it'll be there for the mm's * lifecycle. Avoid setting the bit unless necessary, or it might cause write