From patchwork Tue Mar 12 22:28:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 13590670 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21277C54E5D for ; Tue, 12 Mar 2024 22:29:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AA1358E0022; Tue, 12 Mar 2024 18:29:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A52218E0011; Tue, 12 Mar 2024 18:29:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 87C658E0022; Tue, 12 Mar 2024 18:29:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 722658E0011 for ; Tue, 12 Mar 2024 18:29:09 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 25F1C1609DF for ; Tue, 12 Mar 2024 22:29:09 +0000 (UTC) X-FDA: 81889828818.25.39C5DDD Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.15]) by imf24.hostedemail.com (Postfix) with ESMTP id 3B55718000E for ; Tue, 12 Mar 2024 22:29:05 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=TFB8VeGm; spf=pass (imf24.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.198.163.15 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710282546; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bFDsCgelve8reXSxgTmUguiyPwGkmOEZmQXzPltn6Qg=; b=qdVtGGCsyCUhyXnMdJPOPQ9TcaK5nRB6FEaq5cU3u0eyavlcAM900bXe84XvpkobPTdgbc G1E52I2Mo6oqeKjzstA/iUPxpJF7JkXqlmbI2fOigOOQowa1MkUvtaO4V+oDIgEy/Hcs72 iADI8ACH8g0K6WqkrvGqYVs/ZTYC6cI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710282546; a=rsa-sha256; cv=none; b=xhGRS8zIOVWkCP40kec3cyPxTJUAVoJBdFW1jUZli3sPEZ3BanDpRee0xgyvXI+VFEOweW pBKAdYgJN9hJlnhqHs87Ov1nZqus3py1IZd+OaC18IHUWZbFUNgz4XsFa+vtq7DyQ/BM/n kz9t8urEr9gUr2zxlCB1AGuFQ74EoBE= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=TFB8VeGm; spf=pass (imf24.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.198.163.15 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1710282546; x=1741818546; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dgRXKbvPqyuL8hOX1YSrzuU0fpl568sxa0M2x6sm+7M=; b=TFB8VeGmoVtM4LI8BsNN8T/jXdfDHfhMxEJhfEypwQUKq9yNUtkynoZg xOw12WER5Qufxai22EBT3Ks78X+zu6wkhaYhLcrAVTaHnvP6esq7fAOQR vKFEUSfl6sI7FYsJyqdgTXJE+9gNQRwN8daiJYWd79RdTVQo7t4THKXJV Ltu0FoqU6KiovaW33u8CQOFZo07W/UW9DVdy4eMTHGYjWK/zU9gI5kxgt l2+tohsOMVgjIYroWM3H7BEEwFNTwt7EDd+nrQtesakuGct5F01CG3yhZ q4LC7s7VXvaBdvHOxrVeZSEhFZHe9hNpLqNdCEHG8Wsu3WrBy46XNIUAB Q==; X-IronPort-AV: E=McAfee;i="6600,9927,11011"; a="5191929" X-IronPort-AV: E=Sophos;i="6.07,119,1708416000"; d="scan'208";a="5191929" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmvoesa109.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Mar 2024 15:29:02 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.07,119,1708416000"; d="scan'208";a="16356831" Received: from gargayus-mobl1.amr.corp.intel.com (HELO rpedgeco-desk4.intel.com) ([10.255.231.196]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Mar 2024 15:29:01 -0700 From: Rick Edgecombe To: Liam.Howlett@oracle.com, akpm@linux-foundation.org, bp@alien8.de, broonie@kernel.org, dave.hansen@linux.intel.com, debug@rivosinc.com, hpa@zytor.com, keescook@chromium.org, kirill.shutemov@linux.intel.com, luto@kernel.org, mingo@redhat.com, peterz@infradead.org, tglx@linutronix.de, x86@kernel.org, christophe.leroy@csgroup.eu Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, rick.p.edgecombe@intel.com Subject: [PATCH v3 02/12] mm: Introduce arch_get_unmapped_area_vmflags() Date: Tue, 12 Mar 2024 15:28:33 -0700 Message-Id: <20240312222843.2505560-3-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240312222843.2505560-1-rick.p.edgecombe@intel.com> References: <20240312222843.2505560-1-rick.p.edgecombe@intel.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 3B55718000E X-Rspam-User: X-Stat-Signature: jznsd3hqnr9i1pf38eundfgzao7qaq85 X-Rspamd-Server: rspam03 X-HE-Tag: 1710282545-72193 X-HE-Meta: U2FsdGVkX1+FA4n7m1m1lyRKcyu+RxpSEyZ7oTfPaSnOghlmIWlPIzsBuyEXqp4mtsSK7joApRrsw3wGamsXHIbBrffmd6q9PCcJ6Ag4q4foRVgE+78+n7hTfwrkGCLvEKCwcBV/HNo2+U+2+RBHwS1APCX7kbsIyBYdRSs+YWNZFpXQbGmvzKveELzLw4y8wEoR41syZQGBSk0GCAYLW3Slxh0xFiUdUdG8/PonIa4owoQKWq/rx4beeT9eCaFUIcjyqlvIlvhkPc9gWmINZWZP+ZgpFTgcn1Vt2LncwJfR2GLuLC0q1nYvEvkA0ghCGaWot/2byvW+qaOCgJpUt2gbD4l9MrLbQW+iZgDjcckGn4dZTyTk55nsEE9NWRzSjjG9AkWceS3Av8pxdoor2dMqxacb0V2LFKB6pBdvN5+/5zBgJE8MdIFqUJplLct/UzNJsol/8qHoy/M0fIjae94h8+kLgcMnILRGdhdok8uGIDSmaY+guixMzDQmdlhNUtQH/9d9RqKQCag+ZdXkDnQ+3nTRUAA0xZ00LeGYFQ5RAd41jeNU0Rg64wWwHv5RYiFyUROmEMpDUN/9WviOo+rfIW+bDxUrfM/JnLuuKiIubi7H3ExIBOk6Yoz5Cvpq7Ctm0i8sLgx4qc652s5bfkaMTRHGGPJ4xg+q5EzuIKLv/ygFNQ+R5o3Eomk9H6A63Jhk9T/O3m/XZGQK1+tti2lnxqdETpbB0NMrcyKTOmnqNg0pS5fI95BMMfsk4Mg6kBHbHv3JLr0NaFsg6IArtLzAbaWrwFeGMZN7qXv+egwo7KZm3sCGwUiF8l/USuCLLyvk8SY0mY/q6gX3Q+vD+gzSUPHRer+u2/Gb/B02bm80frrp6ghLZ9qvEH6wfzWyPpQn+TeSHqw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When memory is being placed, mmap() will take care to respect the guard gaps of certain types of memory (VM_SHADOWSTACK, VM_GROWSUP and VM_GROWSDOWN). In order to ensure guard gaps between mappings, mmap() needs to consider two things: 1. That the new mapping isn’t placed in an any existing mappings guard gaps. 2. That the new mapping isn’t placed such that any existing mappings are not in *its* guard gaps. The long standing behavior of mmap() is to ensure 1, but not take any care around 2. So for example, if there is a PAGE_SIZE free area, and a mmap() with a PAGE_SIZE size, and a type that has a guard gap is being placed, mmap() may place the shadow stack in the PAGE_SIZE free area. Then the mapping that is supposed to have a guard gap will not have a gap to the adjacent VMA. In order to take the start gap into account, the maple tree search needs to know the size of start gap the new mapping will need. The call chain from do_mmap() to the actual maple tree search looks like this: do_mmap(size, vm_flags, map_flags, ..) mm/mmap.c:get_unmapped_area(size, map_flags, ...) arch_get_unmapped_area(size, map_flags, ...) vm_unmapped_area(struct vm_unmapped_area_info) One option would be to add another MAP_ flag to mean a one page start gap (as is for shadow stack), but this consumes a flag unnecessarily. Another option could be to simply increase the size passed in do_mmap() by the start gap size, and adjust after the fact, but this will interfere with the alignment requirements passed in struct vm_unmapped_area_info, and unknown to mmap.c. Instead, introduce variants of arch_get_unmapped_area/_topdown() that take vm_flags. In future changes, these variants can be used in mmap.c:get_unmapped_area() to allow the vm_flags to be passed through to vm_unmapped_area(), while preserving the normal arch_get_unmapped_area/_topdown() for the existing callers. Signed-off-by: Rick Edgecombe --- include/linux/sched/mm.h | 17 +++++++++++++++++ mm/mmap.c | 28 ++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+) diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h index cde946e926d8..7b44441865c5 100644 --- a/include/linux/sched/mm.h +++ b/include/linux/sched/mm.h @@ -191,6 +191,23 @@ unsigned long mm_get_unmapped_area(struct mm_struct *mm, struct file *filp, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags); +extern unsigned long +arch_get_unmapped_area_vmflags(struct file *filp, unsigned long addr, + unsigned long len, unsigned long pgoff, + unsigned long flags, vm_flags_t vm_flags); +extern unsigned long +arch_get_unmapped_area_topdown_vmflags(struct file *filp, unsigned long addr, + unsigned long len, unsigned long pgoff, + unsigned long flags, vm_flags_t); + +unsigned long mm_get_unmapped_area_vmflags(struct mm_struct *mm, + struct file *filp, + unsigned long addr, + unsigned long len, + unsigned long pgoff, + unsigned long flags, + vm_flags_t vm_flags); + unsigned long generic_get_unmapped_area(struct file *filp, unsigned long addr, unsigned long len, unsigned long pgoff, diff --git a/mm/mmap.c b/mm/mmap.c index 39e9a3ae3ca5..e23ce8ca24c9 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1810,6 +1810,34 @@ arch_get_unmapped_area_topdown(struct file *filp, unsigned long addr, } #endif +#ifndef HAVE_ARCH_UNMAPPED_AREA_VMFLAGS +extern unsigned long +arch_get_unmapped_area_vmflags(struct file *filp, unsigned long addr, unsigned long len, + unsigned long pgoff, unsigned long flags, vm_flags_t vm_flags) +{ + return arch_get_unmapped_area(filp, addr, len, pgoff, flags); +} + +extern unsigned long +arch_get_unmapped_area_topdown_vmflags(struct file *filp, unsigned long addr, + unsigned long len, unsigned long pgoff, + unsigned long flags, vm_flags_t vm_flags) +{ + return arch_get_unmapped_area_topdown(filp, addr, len, pgoff, flags); +} +#endif + +unsigned long mm_get_unmapped_area_vmflags(struct mm_struct *mm, struct file *filp, + unsigned long addr, unsigned long len, + unsigned long pgoff, unsigned long flags, + vm_flags_t vm_flags) +{ + if (test_bit(MMF_TOPDOWN, &mm->flags)) + return arch_get_unmapped_area_topdown_vmflags(filp, addr, len, pgoff, + flags, vm_flags); + return arch_get_unmapped_area_vmflags(filp, addr, len, pgoff, flags, vm_flags); +} + unsigned long get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags)