From patchwork Tue Aug 30 20:08:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: D Scott Phillips X-Patchwork-Id: 12959839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 99222ECAAA1 for ; Tue, 30 Aug 2022 20:09:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=PqeoeGfpBqLjBFtBvvVuYhKQlO/nDl4K1F0CeaQDK5I=; b=qFGvH3rgS2sbOI mMf/4qoVE4rMfZnwp6BPXRktEyfmAqRfhxOjdnWOI4d8O3BBFzSBQ2Ixenl61ao9LMm6Nh7Kc36MG gvdfkxgNYEkl3Mkz/IXgPUmJLPkPx/CXheNoMgTR71PNLY33UVFSzm63kCDrk0aG6Y9jWy2ZidlyY fgPoDHcH3H+MVMLZvEF4J48lp7nXzdXxQvR3XAVkoRu6EOpcxWPDkcg0A1lF9Tda4ibfGlxZa2yEs W/1hSMD7QxMeb3DZAfJbHmKnmbZQwax+oSt0ZK91uez/JkM44pGQC81tg8efjgpTfRq1it2GdX9GS TaCSxm9yhcYyJrRzfo1w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oT7XL-001hk3-5j; Tue, 30 Aug 2022 20:08:39 +0000 Received: from mail-bn8nam11on2100.outbound.protection.outlook.com ([40.107.236.100] helo=NAM11-BN8-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oT7XH-001hen-Og for linux-arm-kernel@lists.infradead.org; Tue, 30 Aug 2022 20:08:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IOvCqCg6AsUVfK0p2JUurfvoJVRM53DQacZcH+dOqH/T1xSSfPDzcRuRf2xRDUupLjnqSA0w0pbZOEKvcv5DwyjH8DEgu9kdn4sSttCsvwsspBLbr6IQfhKXrn7wvhY9DNPa/LCD05orVL5MUnufV380Ujn0yFub0BVqPgIkgmdh7YMtnG4x6k8CByR5W9zhKPBqSvF6dowpOBi1zWkBpXcQrdcmejvQcu7siH17ZULvENRfG5nPlroRcJrN+84l9kbCa128OynqS+m0QchyFYiWZy1FFu29HPJWZvvcYy28WirmuJF74dwuQ0DzzIKthdgXJX3vWsJkaXUqyBwoiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Z0OJBl757uGGxHxQZ9LJuFG6HSCYJiIMrTKMHpfjlHc=; b=d/fw1rCrce6mGNmpdc6dJPBmjw3WoPgn2WDTB/VGw1SmVXttmIfpsTl+Gqb3AYJ1v3WQ9jltjbRNtPrMtzk0c5/Je3BSRqKLo7J3Y43BS3ECNKlFp2NGPglvMPAo8dYMAW2dZA/dx7+VLns2Zn2oIJfw+w11G7mdVJRCc57ERR8x+X6NkZrrMO8gS1CL+vXKuwdbfMajBFCLLa/JFx5a7ml5jKCQsYrtxH9m/DQwjpoTrx7/E7gQdZOon911kF092c+SzwlDC1q1k1ekvGBORVMO8vAX3cW1IDXID9YqhnGkTdavETEP9QALKjmUqOTJJTAavK1kDWXCi5dZsQmrVg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z0OJBl757uGGxHxQZ9LJuFG6HSCYJiIMrTKMHpfjlHc=; b=R3IOOS09zfAPAVm9lWMZdFGUhMcbwT/055hXoE0sgQdZ4i2rUhabw2zBJqxXhMUXvFysg+gw5cY+ApNlniCLjGwWMgGHg7JRJ56cluSf1feu2Tky37t8xuGUdI3T9T1boO8uCgnv5qV0GIrR2v/9sXQJocuKKgNyFQEzVDZIV2M= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; Received: from MWHPR0101MB2893.prod.exchangelabs.com (2603:10b6:301:33::25) by BN7PR01MB3634.prod.exchangelabs.com (2603:10b6:406:83::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5566.19; Tue, 30 Aug 2022 20:08:31 +0000 Received: from MWHPR0101MB2893.prod.exchangelabs.com ([fe80::a82f:4278:a97f:2c29]) by MWHPR0101MB2893.prod.exchangelabs.com ([fe80::a82f:4278:a97f:2c29%4]) with mapi id 15.20.5566.016; Tue, 30 Aug 2022 20:08:31 +0000 From: D Scott Phillips To: Mark Brown , linux-arm-kernel@lists.infradead.org Cc: Catalin Marinas , Will Deacon , patches@amperecomputing.com Subject: [PATCH] arm64: Work around missing `bti c` in modules Date: Tue, 30 Aug 2022 13:08:26 -0700 Message-Id: <20220830200826.1432338-1-scott@os.amperecomputing.com> X-Mailer: git-send-email 2.37.2 X-ClientProxiedBy: CH2PR04CA0029.namprd04.prod.outlook.com (2603:10b6:610:52::39) To MWHPR0101MB2893.prod.exchangelabs.com (2603:10b6:301:33::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5af6ad64-11a1-4422-7397-08da8ac36870 X-MS-TrafficTypeDiagnostic: BN7PR01MB3634:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1K8vAdGCDkV6E7l8/Sbcn0sohoqNgiFCVUM4GxloutzpvPB89nGsNfM87VROum+YzMUzKHh/66iAKhtlduDyclKbcP2Aysb4hSslkENUUBdgwerLImPK/RhI9uqBINedGLvfn9fS4Snf8TtGJSRTHOOQueycCIbhFmi1Eo2DOQJPGbaAdn0RLiW/ad7Y5iTpWxB8U2b0bmYCs9M/rS/W8ekU2ttN60o67nGsOdYwDLCiNsH6/xVOwWz9y4M+zrpmiGmG9y71b5DFdZvrtElI7RwRtXjY8SmvqZJcNBI3NJYYOIwAasNsYDYhe4LiLn12vPPS4hfvR6nEBZT3Jyt8YSJYCFLxw6db+FDOS+UJK2ojxnVsrEpTWAyJjl2WVGuVSy8Uil3bRZtJYv3J5Fpmp9uWsDECSK8eL58Xq2NLtkP6IPrCLJZMJ0d7olJdr8XjMU6mcB6A7dYWXkkHhV1cLcHI+0o6UNRKDwYMLt2J9QtLXtMOguoB20HbZ8e/VtVJQmBtO+Q//3LMFA6xCPf31DRcl4LdEApYBXWnjN3C4Dl/DG1XZtA01IY3WD1sSP70l8ncK1XhTlH0YWWJX3wTlj05lw500Uraafel85whrWbpjBNbYsh0NQXzPQur+HGxKBdLbcEcVbj8oJs2M3KtBDnNiCYQKHKrZ648ZhHuZcHiS+B7MAtUm0KAdne78cXznASXMgqrSRn29kFSAYz7nfQy+FU8wBQR7xmi4pD2fuzdEy/en3hRbsv9KrMw6dhG/+SVe8cGogLD4lQS9R15dQPP9ThuRI3VXVb2LuHYJxOV2dzOV0tHi38ap0LEbkNE X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR0101MB2893.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(366004)(396003)(39850400004)(346002)(136003)(376002)(5660300002)(478600001)(6666004)(6486002)(107886003)(86362001)(41300700001)(26005)(6512007)(6506007)(83380400001)(52116002)(1076003)(186003)(2616005)(2906002)(8936002)(966005)(54906003)(66476007)(316002)(4326008)(66946007)(66556008)(8676002)(38350700002)(38100700002)(81973001);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: d8oedwSaujRUAuKbg9G6UYAEECg2LQ94WrOFw8MNFcW74cqUBd1y2aQ4MLqLWopwq2MZseQa5Cn2cXF9lgM8jJyX6i5miLfphwdS1NnN7wLbO8ut9k0bzXgseo/uOJJ1uXqcvtSJjXjp/lxbdzNC+10Hs4NZz0RUhB8vS+bbZS6vpXUFAyNVYZoADKteDVZgR/e/slFNmfswvyfHDwvrIzLbi7Wf/n/E5TWKsO1Y5bNMESyhxm+mnLt1MTs114J8RswhIF79OQ9MBteRaA5+Eb9BXLmpS73RwYq4yyX1uYMGinccHk5FxHfzV8XMPS+6NHtgWSm3V/A8ZHT7g+aCXBAwStd/W9ZReYJuDzEvKn8nMFSBOGpNBZahlQTw9fyXrsKPHVrvDGRKHONljf717s7BYW0k8ucc3fB0WKmroACquIWe8+JnIiIqKd6jKo07tjOcAFWiSnmvBFaexYt7j2xKvLDmARSvGPyzlpmaXS1aRIWa8mLybSHBTO39irJ7FdimrZVehu6KBGRk/2rnhlc21/zK5RfYdbi1cFORr/bsf42RvpRIFarbvR5y6fTe50wjZHxGcNxzivBC0LtEApbv8LptXVIE1WY99zvDiUgyaKtw3gMhtJGJdqs5VdLloRK28XBxX1k0dPEvGv+5zCzn4mtrrLI6qay1/BIn6OVDk38CFv1ZRqCUF1opdzT1ppwqNl7VuFVDum6qSYvETbQ2cWUzk84tf/kOJzQQEG7t8JRaZKZUGgpCnssh2VITK3J3zQDiiHEKjA1xiqL7ytaAP0KXVs2zRtflyAaR160n9Q327z+NG1xOshw+VxSBSiOJh5NqrTr+gUXF2AYzHV5te97OmbOmw9t4jUkpMRI1fJ0AgIcDsVFIQSAn4tSsROybz1q+6S7SNm259l7c1N6qTgpPMcTqLaGAv6VR52ZxvL9CiFJrmlUaja4D34hCRvbio8AgENWeUgcWKZBb1nbwBZ0lGbwcpoVYOgI/lylbUEoAFK5/r9NbR9bHtAJChM4bH08IMGbC2+P0N/187EIZ8b7uiBtLh9AwHIxqXPewowAdkF80vAzVj2ATpHH8xNlwlHQzkWy3OV3jRCnY5xFo4AcCJJNmM/oVnL1QBECU2fgjwQHVMmtkc/vSH0GHBNgvbUlIhXirt5TMCWdB5hhORbG1Br75Yz0h9RWzrZ0M474lkzvokeEgbeOPZKJZgJRDR9wP2G9p/bKVDLveTaN8ftW/xBmp/3R2p1qBuoDwwxpC2DV/QVP4y58nzow+ibm/LTGnwHVRrln7ihMm0Dl8YSVpJEUHSav1Ef2oGu6YGll38c6TZeY6Y5HDtKhDJAM2khjzD0302L1kUVDMF86H4hXgS90vbbOcGDr0/ggD5JL6aurLNrdkTbNLb+tm3NkL//ZqSCC6UI1NE5LAxhhfSXY/cTWUfBFq3gRhK+dWLaJsC222Tr/UB5Dxztxk2ON4Wh1i5nwQvHK1gXtUiZtMfH8bN1TL2YNmP9bGItB10CQ7FIrH5k2qWnCWIdYMd8Juxijh/bGYv5BH9vfKn1Hv5d3FERPCMAsjDevERoG3/DeLZHSGFVvZUlzmPKah2eaITpZDb5dF98FBjmB2cw== X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5af6ad64-11a1-4422-7397-08da8ac36870 X-MS-Exchange-CrossTenant-AuthSource: MWHPR0101MB2893.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2022 20:08:31.4301 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nlCc5XbvdqjdotGU6vpmhMgltNAdwGXqD5sPKjcrsTMPpa+ALofAtDvi/UtCR64cdWdz0V7GoUMC53ewt7gxmjN5kXQAe7ptoxDh9zVQWMYg7/JGLqdIROgTjxcYrg65 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR01MB3634 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220830_130836_084244_AE78666D X-CRM114-Status: GOOD ( 14.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org GCC does not insert a `bti c` instruction at the beginning of a function when all callers reach the function through a direct branch[1]. In the case of cross-section calls (like __init to non __init), a thunk may be inserted which uses an indirect branch. If that happens, the first instruction in the callee function will result in a Branch Target Exception due to the missing `bti c`. Handle Branch Target Exceptions which happen in the kernel due to module calls from __init to non-__init by clearing PSTATE.BTYPE and resuming. [1]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106671 Signed-off-by: D Scott Phillips --- arch/arm64/kernel/entry-common.c | 12 ++++++++++ arch/arm64/kernel/traps.c | 39 ++++++++++++++++++++++++++++++-- 2 files changed, 49 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index c75ca36b4a49..dad27e854d8c 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -388,6 +388,15 @@ static void noinstr el1_undef(struct pt_regs *regs) exit_to_kernel_mode(regs); } +static void noinstr el1_bti(struct pt_regs *regs) +{ + enter_from_kernel_mode(regs); + local_daif_inherit(regs); + do_bti(regs); + local_daif_mask(); + exit_to_kernel_mode(regs); +} + static void noinstr el1_dbg(struct pt_regs *regs, unsigned long esr) { unsigned long far = read_sysreg(far_el1); @@ -427,6 +436,9 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) case ESR_ELx_EC_UNKNOWN: el1_undef(regs); break; + case ESR_ELx_EC_BTI: + el1_bti(regs); + break; case ESR_ELx_EC_BREAKPT_CUR: case ESR_ELx_EC_SOFTSTP_CUR: case ESR_ELx_EC_WATCHPT_CUR: diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index b7fed33981f7..f4f1dfa64137 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -501,8 +501,43 @@ NOKPROBE_SYMBOL(do_undefinstr); void do_bti(struct pt_regs *regs) { - BUG_ON(!user_mode(regs)); - force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0); + struct module *mod; + + if (user_mode(regs)) { + force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0); + return; + } + + /* + * GCC does not insert a `bti c` instruction at the beginning + * of a function when all callers reach the function through a + * direct branch. In the case of cross-section calls (like + * __init to non __init), a thunk may be inserted which uses + * an indirect branch. If that happens, the first instruction + * in the callee function will result in a Branch Target + * Exception due to the missing `bti c`. + * + * If that's the case here, clear PSTATE.BTYPE and resume. + */ + if (IS_ENABLED(CONFIG_CC_IS_GCC)) { + preempt_disable(); + mod = __module_text_address(regs->pc); + preempt_enable(); + + if (mod && try_module_get(mod)) { + bool from_init; + + from_init = within_module_init(regs->regs[30], mod); + module_put(mod); + + if (from_init) { + regs->pstate &= ~PSR_BTYPE_MASK; + return; + } + } + } + + die("Oops - BTI", regs, 0); } NOKPROBE_SYMBOL(do_bti);