From patchwork Tue Dec 27 03:08:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kirill A. Shutemov" X-Patchwork-Id: 13082041 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC045C46467 for ; Tue, 27 Dec 2022 03:08:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3063C8E0003; Mon, 26 Dec 2022 22:08:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2901F8E0001; Mon, 26 Dec 2022 22:08:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E1698E0003; Mon, 26 Dec 2022 22:08:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id EE0E38E0001 for ; Mon, 26 Dec 2022 22:08:49 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C7BEC160388 for ; Tue, 27 Dec 2022 03:08:49 +0000 (UTC) X-FDA: 80286603978.09.C51EE2F Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by imf19.hostedemail.com (Postfix) with ESMTP id EE79A1A0004 for ; Tue, 27 Dec 2022 03:08:47 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="IonMKDx/"; spf=none (imf19.hostedemail.com: domain of kirill.shutemov@linux.intel.com has no SPF policy when checking 192.55.52.151) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1672110528; a=rsa-sha256; cv=none; b=mY8Z7gw+cGxYViNc7sR7H6fA9h160zXDcSbmgSxDtHzg66k24DEHTQu44nwGGlmXSVN+et 89/R7pRwj+PcItCaLtjnCfeR1DmJjgVW/YmzgC27Rb5BJw1vyZvFbfkJ09MEqCtyUdmM8w fDMD2gxlhQRT1sp08xO1VVlAQd3HT6w= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="IonMKDx/"; spf=none (imf19.hostedemail.com: domain of kirill.shutemov@linux.intel.com has no SPF policy when checking 192.55.52.151) smtp.mailfrom=kirill.shutemov@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1672110528; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C7HJvEkFakmMiC+4clhqvBp0TSytYpgLWXa6owkCvnA=; b=jGR6jXx+gfs7+VLCQEPQU9/mk+pNWtLu1CKerq9ueQK4QMHR9BO8qgmM5pY5lztcPSOBeI wXM8F+Zkobl/2LWk9VEjYSLAUN/qBcy2OgBT4WjAoEfUrXhjqF9BCuNl4UBxLiiNNTHwF2 p9D5LxNsXo7Z02FvGwwaap6A4Eg1mHE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1672110528; x=1703646528; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=73Lfy6YiNOngUHGIq+SvJHDmwJJPCjL8Zm/vzQL6EVI=; b=IonMKDx/Mc36bNlhQQBWDlTElDRWxT3HvGJamGMphC+iEWEM+jqt/L6y 9jn5rXDwkTiOwU/agvlvbqWDgIpjIokYYEZ0S4pQACEhsahf8wSVhOvx8 ymJ0iKSN64YjiVMrtZULRiqEJ37oTV6ohCiVJ4ArOnSZI+MfBrPABsl73 B/OIjnvziGCsnJEhvO/9sg1hrVLqRJYft41JBH5wy+s6pnBLr4y1exu9X 1gtzr7PPBFk4XL8ALUZP6RjrRDdQ4CWm0BC4pVX+PWGOTGnNz+3n5HF18 XdYgzXQMe1TxTTmoMDZuLaFASHFzDKEgxrHGxR8DLZA6RUPL491+Pw9vF g==; X-IronPort-AV: E=McAfee;i="6500,9779,10572"; a="300994660" X-IronPort-AV: E=Sophos;i="5.96,277,1665471600"; d="scan'208";a="300994660" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Dec 2022 19:08:43 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10572"; a="741605194" X-IronPort-AV: E=Sophos;i="5.96,277,1665471600"; d="scan'208";a="741605194" Received: from ppogotov-mobl.ger.corp.intel.com (HELO box.shutemov.name) ([10.252.62.152]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Dec 2022 19:08:39 -0800 Received: by box.shutemov.name (Postfix, from userid 1000) id A1E7910BBB1; Tue, 27 Dec 2022 06:08:36 +0300 (+03) From: "Kirill A. Shutemov" To: Dave Hansen , Andy Lutomirski , Peter Zijlstra Cc: x86@kernel.org, Kostya Serebryany , Andrey Ryabinin , Andrey Konovalov , Alexander Potapenko , Taras Madan , Dmitry Vyukov , "H . J . Lu" , Andi Kleen , Rick Edgecombe , Bharata B Rao , Jacob Pan , Ashok Raj , Linus Torvalds , linux-mm@kvack.org, linux-kernel@vger.kernel.org, "Kirill A. Shutemov" Subject: [PATCHv13 04/16] mm: Introduce untagged_addr_remote() Date: Tue, 27 Dec 2022 06:08:17 +0300 Message-Id: <20221227030829.12508-5-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.38.2 In-Reply-To: <20221227030829.12508-1-kirill.shutemov@linux.intel.com> References: <20221227030829.12508-1-kirill.shutemov@linux.intel.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: EE79A1A0004 X-Rspamd-Server: rspam01 X-Stat-Signature: fbfhr4gk4yha7fm11qm4nwbfrn4azuj8 X-HE-Tag: 1672110527-827226 X-HE-Meta: U2FsdGVkX1/+odKZu59MS1u0+4lqL3OxweB+RtqqDzbg+rDl4XcvaqfIvH+f5B0ulEI6UmQtKLC0j46bIB/04DpUGc8CAJLZACUnmLS/khEnzKOdxPW2zLQ11ypk5BZ+crQ4WOGmSbdK9rZ8bfjsG7IWdIxsMj+ZDIeV9zSu0SJVxCY3LqG6xmkWTWGabsoXR9jNgXspk7e5QuCf2hzYOcN8kmonk3Z+9xkb2qzIV+uAAFzYxkg1hVjIm5FEdQQfHS59nEp36xAm7EZi8HZ6KdgamBJ7FJ8RyTUyaSAefuIovKbv50j8NRv78rqYZYsqM6Vhn+bOa/ITOELZiEq1NbeePvgpi8/rAxyrzQ0YEj+Yoxmb7cF7iC9l3t8szF9yoSg7A5SOk+75RB8t9uYob+9qNKFYPxgL30TsVkAhJYmLaPxxZh07r3nMNvZyI+Hv63KgJ2FtS7/F5IoUKHgGBvcSLJoJXrUQMvyYZH5CWbzKvnCqz95JXM4uEgsPLFoZ3G/KQdahtrRj2fQuSxu3amZwk3nbx6sPSzmbvh9/bTUL6/uf7bbBGbrrQ/BaOhEhZadZUWrUxb/2RP1urvHJrTRmL6pvxGzZL/XcvuJN6tSeS1doWQ+n4940EaWvPA992qAF6B9RRUk4me3vNWMsf7pG5cXhXakUBznNy7pFy5q1xPI0Io7K8p9CyNSFyzkR5OLOkjJXJ2ZEXCSFWZ3f40m2Z414ywyeXhRAr0SnAuNbf06NSPtgP4EDoGuI/86VkeZ0LfWdkMMSlhizyj97YTzOcoo31B69b7Rn7F/xDfzDdhiCnROLGF2MPehAlQsAomhQCKCps8DoNOLXqVVokIdiQnqCkPG8pQYYhFPlxjHNnQOyXm4bIHSVpAzLhQJT9Y4e34uyoYSMj6JWrcBXivH+/rem+OGyemTfNtzCuxMXW2Kaeu/aIKEnoR0Oy+eT2smTW0x88bYgkFN/mUb uTHKkITd FuMG2tfRRGAQV7ThfSr0Tymblr1UGCwc7Q4OE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: untagged_addr() removes tags/metadata from the address and brings it to the canonical form. The helper is implemented on arm64 and sparc. Both of them do untagging based on global rules. However, Linear Address Masking (LAM) on x86 introduces per-process settings for untagging. As a result, untagged_addr() is now only suitable for untagging addresses for the current proccess. The new helper untagged_addr_remote() has to be used when the address targets remote process. It requires the mmap lock for target mm to be taken. Export dump_mm() as there's now the first user for it: VFIO can be compiled as module and untagged_addr_remote() triggers dump_mm() via mmap_assert_locked(). Signed-off-by: Kirill A. Shutemov --- arch/sparc/include/asm/uaccess_64.h | 2 ++ drivers/vfio/vfio_iommu_type1.c | 2 +- fs/proc/task_mmu.c | 2 +- include/linux/mm.h | 11 ----------- include/linux/uaccess.h | 22 ++++++++++++++++++++++ mm/debug.c | 1 + mm/gup.c | 4 ++-- mm/madvise.c | 5 +++-- mm/migrate.c | 2 +- 9 files changed, 33 insertions(+), 18 deletions(-) diff --git a/arch/sparc/include/asm/uaccess_64.h b/arch/sparc/include/asm/uaccess_64.h index 94266a5c5b04..b825a5dd0210 100644 --- a/arch/sparc/include/asm/uaccess_64.h +++ b/arch/sparc/include/asm/uaccess_64.h @@ -8,8 +8,10 @@ #include #include +#include #include #include +#include #include #include diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 23c24fe98c00..daf34f957b7b 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -573,7 +573,7 @@ static int vaddr_get_pfns(struct mm_struct *mm, unsigned long vaddr, goto done; } - vaddr = untagged_addr(vaddr); + vaddr = untagged_addr_remote(mm, vaddr); retry: vma = vma_lookup(mm, vaddr); diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index e35a0398db63..25786a808901 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1693,7 +1693,7 @@ static ssize_t pagemap_read(struct file *file, char __user *buf, /* watch out for wraparound */ start_vaddr = end_vaddr; if (svpfn <= (ULONG_MAX >> PAGE_SHIFT)) - start_vaddr = untagged_addr(svpfn << PAGE_SHIFT); + start_vaddr = untagged_addr_remote(mm, svpfn << PAGE_SHIFT); /* Ensure the address is inside the task */ if (start_vaddr > mm->task_size) diff --git a/include/linux/mm.h b/include/linux/mm.h index f3f196e4d66d..6b28eb9c6ea2 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -96,17 +96,6 @@ extern int mmap_rnd_compat_bits __read_mostly; #include #include -/* - * Architectures that support memory tagging (assigning tags to memory regions, - * embedding these tags into addresses that point to these memory regions, and - * checking that the memory and the pointer tags match on memory accesses) - * redefine this macro to strip tags from pointers. - * It's defined as noop for architectures that don't support memory tagging. - */ -#ifndef untagged_addr -#define untagged_addr(addr) (addr) -#endif - #ifndef __pa_symbol #define __pa_symbol(x) __pa(RELOC_HIDE((unsigned long)(x), 0)) #endif diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h index afb18f198843..bfdadf5f8bbb 100644 --- a/include/linux/uaccess.h +++ b/include/linux/uaccess.h @@ -10,6 +10,28 @@ #include +/* + * Architectures that support memory tagging (assigning tags to memory regions, + * embedding these tags into addresses that point to these memory regions, and + * checking that the memory and the pointer tags match on memory accesses) + * redefine this macro to strip tags from pointers. + * + * Passing down mm_struct allows to define untagging rules on per-process + * basis. + * + * It's defined as noop for architectures that don't support memory tagging. + */ +#ifndef untagged_addr +#define untagged_addr(addr) (addr) +#endif + +#ifndef untagged_addr_remote +#define untagged_addr_remote(mm, addr) ({ \ + mmap_assert_locked(mm); \ + untagged_addr(addr); \ +}) +#endif + /* * Architectures should provide two primitives (raw_copy_{to,from}_user()) * and get rid of their private instances of copy_{to,from}_user() and diff --git a/mm/debug.c b/mm/debug.c index 7f8e5f744e42..3c1b490c7e2b 100644 --- a/mm/debug.c +++ b/mm/debug.c @@ -215,6 +215,7 @@ void dump_mm(const struct mm_struct *mm) mm->def_flags, &mm->def_flags ); } +EXPORT_SYMBOL_GPL(dump_mm); static bool page_init_poisoning __read_mostly = true; diff --git a/mm/gup.c b/mm/gup.c index f45a3a5be53a..e28d787ba8f8 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1091,7 +1091,7 @@ static long __get_user_pages(struct mm_struct *mm, if (!nr_pages) return 0; - start = untagged_addr(start); + start = untagged_addr_remote(mm, start); VM_BUG_ON(!!pages != !!(gup_flags & (FOLL_GET | FOLL_PIN))); @@ -1265,7 +1265,7 @@ int fixup_user_fault(struct mm_struct *mm, struct vm_area_struct *vma; vm_fault_t ret; - address = untagged_addr(address); + address = untagged_addr_remote(mm, address); if (unlocked) fault_flags |= FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE; diff --git a/mm/madvise.c b/mm/madvise.c index a56a6d17e201..90cd4a442fd2 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1407,8 +1407,6 @@ int do_madvise(struct mm_struct *mm, unsigned long start, size_t len_in, int beh size_t len; struct blk_plug plug; - start = untagged_addr(start); - if (!madvise_behavior_valid(behavior)) return -EINVAL; @@ -1440,6 +1438,9 @@ int do_madvise(struct mm_struct *mm, unsigned long start, size_t len_in, int beh mmap_read_lock(mm); } + start = untagged_addr_remote(mm, start); + end = start + len; + blk_start_plug(&plug); error = madvise_walk_vmas(mm, start, end, behavior, madvise_vma_behavior); diff --git a/mm/migrate.c b/mm/migrate.c index a4d3fc65085f..2cdffeff9ffe 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1839,7 +1839,7 @@ static int do_pages_move(struct mm_struct *mm, nodemask_t task_nodes, goto out_flush; if (get_user(node, nodes + i)) goto out_flush; - addr = (unsigned long)untagged_addr(p); + addr = (unsigned long)untagged_addr_remote(mm, p); err = -ENODEV; if (node < 0 || node >= MAX_NUMNODES)