From patchwork Sat Dec 3 00:35:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 13063362 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC211C47089 for ; Sat, 3 Dec 2022 00:37:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 368D16B007D; Fri, 2 Dec 2022 19:37:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2F1B56B007B; Fri, 2 Dec 2022 19:37:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0F5238E0001; Fri, 2 Dec 2022 19:37:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E5D096B007D for ; Fri, 2 Dec 2022 19:37:27 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id C6CCFC03D1 for ; Sat, 3 Dec 2022 00:37:27 +0000 (UTC) X-FDA: 80199131334.12.5292C45 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by imf30.hostedemail.com (Postfix) with ESMTP id 3A8AB80011 for ; Sat, 3 Dec 2022 00:37:26 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Laof0acB; spf=pass (imf30.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670027847; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references:dkim-signature; bh=RuVeAKBdy5nPRl76k0Iqev2IB3RHW+4gMvSht8VTB/E=; b=Aoya3ogtS7ii+VS/Bd8Gu+feSK3FqKPPHwTFe1U7G5kFdAe58MW8/1skOf31i9uOUKxVXy qP3HLNqYa3nnR1y4TYvtpKYi4D4jmhguCESc/WamWm0oUgN05PCR7WeAVhK1Bd+23aDEA8 hTU8qE2ZZ43yN+RfZ4nmaPx8Leg4ezA= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Laof0acB; spf=pass (imf30.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670027847; a=rsa-sha256; cv=none; b=CuDmHdmqXtuOXV9TyH7A8yaF0U/jUBNXD+c8dK0zrU86e++SZ78hN9+FetdN6GEvyxTL0e P9nGOxBhiDKVcxt8UaDLaBX4POujIjtCIfsgWaIEcJx5vVi6O8vTzpVQb3nkIpxNyoXPwu LyBH8Lb9DKT60FbHrMHoHtMlZjZ/6I0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1670027847; x=1701563847; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=Fk/Qne5mwiZTFgT2silFS8+68/JE0zNL0yZ7bHwxdtU=; b=Laof0acBd5eAeVzxxQnbLUMNj/AAgQJrlFy9KJ88txOGWrVDE6+j0DIN PAw4rqTfkAglYN9Rd5/XEX2DdDpDLYVJ34dpA4idthkm7FERd807KZKpN lTYtEvejeLnu9uoYzVyxPQptlaJq4kynbD1k9MxgCj7d8OnXnzIGhURJS L1gY3I2HV2cOVevEPEXJS8J0SQRv2h9Bp85HJ9TumqhVNdjwjxMQ7f/vf Z4HLzRXpkpCpLbAmGDFT4j2a483r4GO+Jqt58YnXpvdzu5fHwkTUEDZL3 n04bwCKddxwviCpnyOjyomcz3lvTW5gKIMvsNNDESu6VvpQgf1RGHeduR g==; X-IronPort-AV: E=McAfee;i="6500,9779,10549"; a="313711172" X-IronPort-AV: E=Sophos;i="5.96,213,1665471600"; d="scan'208";a="313711172" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Dec 2022 16:37:20 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10549"; a="787479924" X-IronPort-AV: E=Sophos;i="5.96,213,1665471600"; d="scan'208";a="787479924" Received: from bgordon1-mobl1.amr.corp.intel.com (HELO rpedgeco-desk.amr.corp.intel.com) ([10.212.211.211]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Dec 2022 16:37:18 -0800 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com Cc: rick.p.edgecombe@intel.com, Yu-cheng Yu Subject: [PATCH v4 21/39] mm/mprotect: Exclude shadow stack from preserve_write Date: Fri, 2 Dec 2022 16:35:48 -0800 Message-Id: <20221203003606.6838-22-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221203003606.6838-1-rick.p.edgecombe@intel.com> References: <20221203003606.6838-1-rick.p.edgecombe@intel.com> X-Rspamd-Queue-Id: 3A8AB80011 X-Stat-Signature: hphoq588ns6xjr5x79o3o6xnfzok8zsc X-Rspam-User: X-Spamd-Result: default: False [-0.30 / 9.00]; BAYES_HAM(-2.90)[91.31%]; SUSPICIOUS_RECIPS(1.50)[]; SUBJECT_HAS_UNDERSCORES(1.00)[]; MID_CONTAINS_FROM(1.00)[]; DMARC_POLICY_ALLOW(-0.50)[intel.com,none]; R_SPF_ALLOW(-0.20)[+ip4:192.55.52.93/32:c]; R_DKIM_ALLOW(-0.20)[intel.com:s=Intel]; MIME_GOOD(-0.10)[text/plain]; RCVD_NO_TLS_LAST(0.10)[]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWELVE(0.00)[40]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[intel.com:+]; TO_DN_SOME(0.00)[]; ARC_SIGNED(0.00)[hostedemail.com:s=arc-20220608:i=1]; TAGGED_RCPT(0.00)[]; ARC_NA(0.00)[] X-Rspamd-Server: rspam08 X-HE-Tag: 1670027846-197854 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu-cheng Yu The x86 Control-flow Enforcement Technology (CET) feature includes a new type of memory called shadow stack. This shadow stack memory has some unusual properties, which requires some core mm changes to function properly. In change_pte_range(), when a PTE is changed for prot_numa, _PAGE_RW is preserved to avoid the additional write fault after the NUMA hinting fault. However, pte_write() now includes both normal writable and shadow stack (Write=0, Dirty=1) PTEs, but the latter does not have _PAGE_RW and has no need to preserve it. Exclude shadow stack from preserve_write test, and apply the same change to change_huge_pmd(). Tested-by: Pengfei Xu Tested-by: John Allen Signed-off-by: Yu-cheng Yu Reviewed-by: Kirill A. Shutemov Signed-off-by: Rick Edgecombe Reviewed-by: Kees Cook --- v4: - Add "why" to comments in code (Peterz) Yu-cheng v25: - Move is_shadow_stack_mapping() to a separate line. Yu-cheng v24: - Change arch_shadow_stack_mapping() to is_shadow_stack_mapping(). mm/huge_memory.c | 8 ++++++++ mm/mprotect.c | 8 ++++++++ 2 files changed, 16 insertions(+) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 60451e588955..b6294c4ad471 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1803,6 +1803,14 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, return 0; preserve_write = prot_numa && pmd_write(*pmd); + + /* + * Preserve only normal writable huge PMD, but not shadow + * stack (RW=0, Dirty=1), so the restoring code doesn't + * make the shadow stack memory conventionally writable. + */ + if (vma->vm_flags & VM_SHADOW_STACK) + preserve_write = false; ret = 1; #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION diff --git a/mm/mprotect.c b/mm/mprotect.c index de7351631e21..026347f1f1ee 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -115,6 +115,14 @@ static unsigned long change_pte_range(struct mmu_gather *tlb, pte_t ptent; bool preserve_write = prot_numa && pte_write(oldpte); + /* + * Preserve only normal writable PTE, but not shadow + * stack (RW=0, Dirty=1), so the restoring code doesn't + * make the shadow stack memory conventionally writable. + */ + if (vma->vm_flags & VM_SHADOW_STACK) + preserve_write = false; + /* * Avoid trapping faults against the zero or KSM * pages. See similar comment in change_huge_pmd.