From patchwork Tue Feb 18 22:22:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Williamson X-Patchwork-Id: 13980971 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBF8BC021AA for ; Tue, 18 Feb 2025 22:22:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 380BE2801A2; Tue, 18 Feb 2025 17:22:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3315328019B; Tue, 18 Feb 2025 17:22:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F9502801A2; Tue, 18 Feb 2025 17:22:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id F3BF028019B for ; Tue, 18 Feb 2025 17:22:41 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 83BDD1406B4 for ; Tue, 18 Feb 2025 22:22:41 +0000 (UTC) X-FDA: 83134490922.16.5E8E589 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf30.hostedemail.com (Postfix) with ESMTP id C41928000D for ; Tue, 18 Feb 2025 22:22:39 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=jAEag9Jt; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf30.hostedemail.com: domain of alex.williamson@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=alex.williamson@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739917359; a=rsa-sha256; cv=none; b=p+hnlaB9iFiiIxcebD/h6BL30aHz6lCbPNaXw+TXU/+A+GFjZBmJYF7NfQvw56hDu/oytK BRGQqZXfo6eELJeUCDFZp6RnQn/8BjqWDq5NLB4XbTNw35dt7Rc4yVWeKgrSZHqXj2N/Ul VWB6KAaTjbN1JW9Nj3EkF7OkjUa2FZQ= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=jAEag9Jt; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf30.hostedemail.com: domain of alex.williamson@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=alex.williamson@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739917359; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UlUtITEYQJv1hpgGQ8C/UoW2nF0g3ARQ77wXP5+0OfU=; b=bXUUox6GJfQWE1l4EJCFmQTM9YvXajFmBv3SbFm/Mn245KgM2pKvMIferDLpqT5e8qb6VW uaFbhaBjki0oPYWS9A/c5FimtXruJYiLmjkxEVjFYZ87p3SOpmrM+U+72i5R36UTjZRWzz IIXj/rwmKvfkuDwjErkkshQ+n4WWOEk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1739917359; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UlUtITEYQJv1hpgGQ8C/UoW2nF0g3ARQ77wXP5+0OfU=; b=jAEag9Jt70dBVsGk4FKlPQHjqi0x3igUFkdru3mDCcWz2us45EriGMwqOFW/47ohu0Yrvi 3xWvjGh9Yu9GHacyxJU74xfpqwJ/HrEGTaWPb/CK8aW08NaYXtoLsbD2Q+ivtwO9LwOH1U E4iyDtMO3Z6txkUSXH9eWKCG3HtmMCc= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-637-bV8OMXniNsCt5H9n6wIQ_A-1; Tue, 18 Feb 2025 17:22:34 -0500 X-MC-Unique: bV8OMXniNsCt5H9n6wIQ_A-1 X-Mimecast-MFC-AGG-ID: bV8OMXniNsCt5H9n6wIQ_A_1739917353 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8C000180087B; Tue, 18 Feb 2025 22:22:33 +0000 (UTC) Received: from omen.home.shazbot.org (unknown [10.22.88.77]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CD75E300019F; Tue, 18 Feb 2025 22:22:30 +0000 (UTC) From: Alex Williamson To: alex.williamson@redhat.com Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, peterx@redhat.com, mitchell.augustin@canonical.com, clg@redhat.com, jgg@nvidia.com, akpm@linux-foundation.org, linux-mm@kvack.org, david@redhat.com Subject: [PATCH v2 5/6] mm: Provide address mask in struct follow_pfnmap_args Date: Tue, 18 Feb 2025 15:22:05 -0700 Message-ID: <20250218222209.1382449-6-alex.williamson@redhat.com> In-Reply-To: <20250218222209.1382449-1-alex.williamson@redhat.com> References: <20250218222209.1382449-1-alex.williamson@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Rspam-User: X-Rspamd-Queue-Id: C41928000D X-Rspamd-Server: rspam12 X-Stat-Signature: y6yipbqw3zeqmjw3xam5drg3141p38xg X-HE-Tag: 1739917359-778245 X-HE-Meta: U2FsdGVkX1/zjoAR8ws+z7MsjYSkSCKILivI6XcHFpi4SW1AvxxvkJjRJ9lW8g3pUFqMJT6JB/xLNP3EGLOnqwckBtVGVRmAY77DM226+kDA9YmTyN/JTggW/rZPefSLT6XaHMspJo1x2Wz9SgMnfWG5CEI5UfkPlW/d3XcE0+422J/kdYoBdyrV8Lw/biQvoeS1RFo83OkgJlmMp2yGV6hrmNiJ0Ol/uWimFe5jDtdbtmL/aZptYAH4VjxMbia6Rbf/xWG2ju2xeEKItwBBiMrMiT5hlvsdq0HKeHZI/vWXlocivTByEsxdQDmxGXsyCl8U0DR7/3nwuqtKP8i/v53vJnUxH/wpHdZHJsBtF/AcLaNkB6X30Of+A18jLDSu2f1x4VuC6yfLGOSFHJhl2f7bgR7dLl/BlLdL7MjS4+il3C+33u6Si4+QbbBaqoUWsse+OhOL3CSBKm3btjzaVQICOZqVw1Bw7vpxjHjWj7HyfLuwwz2X7JNUSlETb2WLj3h+nVwN9D4ns+dYbtlM0I3r5QpvuuE/28T/Vn2vNbADO/fHmp9wwm63h8CpLwrjMQsXeVbEEjUqFqCmQW1A3yKNsdsTU1hUbhydLMFlRchPokDCOvWt77uVYiKyVPEXM5jnXb2OJDRtLmTm/j1e9nA6zCliBUUU5RwbyBFf3HjqNAx1ETilKKJ9viKea5mjAA2IycWDIjNhRJjt/Gnd2T+pjHwtWli9ZgXlQiyzxyNUIFdxsgct+HpRnG8gom4Kyrwnaj9WcH69CKpW4mpHyW9gZeZRvyELtXk/V4JHnPH/uFwS3F8AsMhd0zD4p8hY0ba7f85+yVj4P/wCje99e8odJy05/ITZUoHewg/tbG/gYt0lOqQUsfImJwNQ6Qvc7DGttRAeQXBpblqtwUBcoKw3IzdS5NRY514vWxgyeK4L3GsG9V+G55cabAMXRevaGH3O8cjyRD6VaDHq7ag RkkGsjxG q7QD60HE+vOhxfV8KOwJv/H1gd4kCuwLrdXiPTNLGn86vWw3E1xWPU7Wbh38FKr6bwcr3SfzJvtJjGz5cViu5cRgqh7SbMhmsw2ouxeK9+n8KDewa00cjsT1Sm9ZkzYs7qyQF6/6kRKmAt1o/wveP+2fyLLaMNqbFFvFCCBaHBXx/O7Hx25ysMg7d2lSuyiN2FIjM/icB1zUxKaGKcSkGyKnT8ACQPSIgZ8DA2CIUrq0q5qCSCmhiarTrcpmcaGhzHdOceBOtxwdGLvqtZXpEjfGoteop5+eQg2i6Dji+tVgiktAE/Am4ynpWE0j1cIceqsaxB32EVHQSEKT04ft6B9AjmSlcQMPw7RZX14u2+P6pPh1MH/XuYOX/HDLeyKNg52zYZ2UPEbrucDlJ4CPcchomWcfif+ufRZCzQaO9p7Zc+CQR2Fhe2gO5XYNRsHjxSFQuyG0zPU55HroAWJXYPBD5nAcud3bXKhSatF2olnrYVFijFKwqT3IFE19PgKURA2yM37TNyhkc4D5suHmjAduHSs17vWzjdRn9DF6AUolIifpP9Be8fWgcDr5pLqrT6jor X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: follow_pfnmap_start() walks the page table for a given address and fills out the struct follow_pfnmap_args in pfnmap_args_setup(). The address mask of the page table level is already provided to this latter function for calculating the pfn. This address mask can also be useful for the caller to determine the extent of the contiguous mapping. For example, vfio-pci now supports huge_fault for pfnmaps and is able to insert pud and pmd mappings. When we DMA map these pfnmaps, ex. PCI MMIO BARs, we iterate follow_pfnmap_start() to get each pfn to test for a contiguous pfn range. Providing the mapping address mask allows us to skip the extent of the mapping level. Assuming a 1GB pud level and 4KB page size, iterations are reduced by a factor of 256K. In wall clock time, mapping a 32GB PCI BAR is reduced from ~1s to <1ms. Cc: Andrew Morton Cc: David Hildenbrand Cc: linux-mm@kvack.org Reviewed-by: Peter Xu Reviewed-by: "Mitchell Augustin" Tested-by: "Mitchell Augustin" Reviewed-by: Jason Gunthorpe Signed-off-by: Alex Williamson Acked-by: David Hildenbrand --- include/linux/mm.h | 2 ++ mm/memory.c | 1 + 2 files changed, 3 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 7b1068ddcbb7..92b30dba7e38 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2417,11 +2417,13 @@ struct follow_pfnmap_args { * Outputs: * * @pfn: the PFN of the address + * @addr_mask: address mask covering pfn * @pgprot: the pgprot_t of the mapping * @writable: whether the mapping is writable * @special: whether the mapping is a special mapping (real PFN maps) */ unsigned long pfn; + unsigned long addr_mask; pgprot_t pgprot; bool writable; bool special; diff --git a/mm/memory.c b/mm/memory.c index 539c0f7c6d54..8f0969f132fe 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -6477,6 +6477,7 @@ static inline void pfnmap_args_setup(struct follow_pfnmap_args *args, args->lock = lock; args->ptep = ptep; args->pfn = pfn_base + ((args->address & ~addr_mask) >> PAGE_SHIFT); + args->addr_mask = addr_mask; args->pgprot = pgprot; args->writable = writable; args->special = special;