From patchwork Thu Sep 1 15:41:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: D Scott Phillips X-Patchwork-Id: 12962839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 27AF3ECAAD1 for ; Thu, 1 Sep 2022 15:43:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=o24P9zC+f8MU5xPP1WFzApxKs5ZQB3BmH7aqh7m03y4=; b=mdDrTraArMoot7 AnSiYcbU2TUPs5TiEdj/fosT42RPuAnxG98YO/bOz8sCDIxU1Go7LQ16ATVrNdrlz1/gDT3puhC7y f34+YfQtF9ykBOCysvfdZ9hjqMYMsjh0ovRrACYmJNFsJI9Gv92ho31oduA8eqDThkJ/QN5irB2GO j1fqqxZ5hLG9/M3Qfmo/Ebs/AZKyLRs8TqZW4B64ynOrV0vrMrU9OltBGPMNvD2gEaUvsvo/IuHVT hEY674VVzrGGmTMcOoSsFF/hWMIuBMBfnrvFcpRkM/6nk+tplmQ/Fv6dQsEcqpCCgxuPvUGWCCfiD JCZUyHCIYAAekif58VFg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oTmKW-00CwMo-HS; Thu, 01 Sep 2022 15:42:08 +0000 Received: from mail-bn8nam11on2100.outbound.protection.outlook.com ([40.107.236.100] helo=NAM11-BN8-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oTmKR-00CwJp-Dy for linux-arm-kernel@lists.infradead.org; Thu, 01 Sep 2022 15:42:06 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ma207jdmaR4iDZpi9hqtqBwfskQZvW5vk7yk4KnxU350dmL6udRajNi//D/ZRqpA99+8Cm2Q+1ZfbhyTXd5QwfxOXq8NSxy9tUR0Iv4rIvPkUS2j9XbtC9BKpsBwFOb7yuQadbWH6KV+gcIdcQ0EWg3KkN4eQeZW/bJMoziblutbSjomz7eAP4eqYLXKbWPP61PhZQggv5ABgFpycdEDq159RUp0pco9Qqa4ze91Mh83qHVK7bebPcumLt+SNPGsZ217N5Gjss+UJjl0xCIbOA/bMgjTjQ3IMzAvv59rwxK+ozAwjuPKbwjFPxfik4rg23QWFQBNRdl0MBNqkjEQLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tit2ktgL1ty94N5m25aHZCuKd64gB38uLZOPdForUQc=; b=LTrdbHQmzQ/ysTisySmSxdos881iBsLQ8NPY9Sm6r2LZqBFUY5WQ5qCuk7PqYJWzjwaqu5f74rc9yoCZY3XEyTs3kbpIb21RINCKA7+/KIVXNN8OT/R7IA/vWZt4e5MPYoWjsY6eObijag2I6b045+ZkZIP57Ka7nOKEg3x9ZcP9Wa2LwFiD6EFjRgbnr0duSebLKUc8GOcX0gcsdKd1AbivlUsGWpcuIPZ2zyznZdqLQYZTtIHCLwB6zEHiWrRxZ0KfDEcBXdnROUB4Enjtnt+NMZMvLjGUL+BuBAYmgGRwjGwTM3aZHX/b6AMUUNYchwJDs1tSnwf7eYHULBJCrw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tit2ktgL1ty94N5m25aHZCuKd64gB38uLZOPdForUQc=; b=k+uddiszncS9PXPZUcdHIFYRTxpKXGIFEaVi72++hRNvMHbjEck8XMg7YkCivPrbDIe9BjgVCB1Ev3YPA5uNtnrv9WdHxjhAiNbxn4Vfgu5BvGRkl56tSo5/WhJKlxkPuRlQqzQft8/04KJ48GWMuaa1eTWzVkc/sv4+K6VNm10= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; Received: from MWHPR0101MB2893.prod.exchangelabs.com (2603:10b6:301:33::25) by BN8PR01MB5619.prod.exchangelabs.com (2603:10b6:408:bc::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Thu, 1 Sep 2022 15:41:31 +0000 Received: from MWHPR0101MB2893.prod.exchangelabs.com ([fe80::a82f:4278:a97f:2c29]) by MWHPR0101MB2893.prod.exchangelabs.com ([fe80::a82f:4278:a97f:2c29%4]) with mapi id 15.20.5566.016; Thu, 1 Sep 2022 15:41:31 +0000 From: D Scott Phillips To: Mark Brown , linux-arm-kernel@lists.infradead.org Cc: Catalin Marinas , Will Deacon , patches@amperecomputing.com Subject: [PATCH v2] arm64: Work around missing `bti c` in modules Date: Thu, 1 Sep 2022 08:41:27 -0700 Message-Id: <20220901154127.2120577-1-scott@os.amperecomputing.com> X-Mailer: git-send-email 2.37.2 X-ClientProxiedBy: CH0PR03CA0054.namprd03.prod.outlook.com (2603:10b6:610:b3::29) To MWHPR0101MB2893.prod.exchangelabs.com (2603:10b6:301:33::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ac7b0074-d1f8-488c-e9ac-08da8c3070e9 X-MS-TrafficTypeDiagnostic: BN8PR01MB5619:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wt9iyjSK41l16NzwECh9jMQXXBVo4xWYByODnn4yDCsWqQyKoB3ErKlMohNzZnTYHH4JjjWiHEvqCfk4Vt7C2ZnChHuJaEpE6eBZM/ZCbvO0g5xbFDAAhE4Jdrq2uLfrCoQjZ4kab30l9kwtxxPGBq7rqgKGClvCpjo76AO0Qczgw8P5s6+lbco5+SYUsfN8vftG67OPgviwb3xgO4XmkIYPcuLVHiiqfaoSoyS3s6CaJ01KEl8Epv8OCzGaaZQG9qZckG/plLvvTQHeY2Ow0dYiVRiFkz1LPQiCu5lXNNtWtd971yYI7iN8O3jPQ26AxtBiDRFsWsoGT7nZHaUTWlJNAcVY3S1dtxKC8+7mvpz09fq2u41QMvr9pBt6usOze5h+m8+N1RlwAsM1xeEWs0sOqNqe6qXzw6Up48Hqz+mFde2SLSynOaF6rwCdiSgr7ssl8IQqsQJWfiVMAusNdvn5Paqk1vYeEbqwxB1S9VROejVymjfPYqjjuByyQd738/5whyM5JtIC8yApgL31RahRC0+1A6++Z4RDKTB+FcLBH3vRdVHsI0ebA6lQR0z+Mn6J1cN+lctiVCNP4dLO+Z+c4Xa9sJItL02oHOkHKAzk5cT56j9oNJuPyPCqftaD0I7xybXQhIJHElC/OKBOT9jo13tQEmfNTlHFYLs0363n4XYdyh15u0MPYSSmPuCDy8jbscv1tXsPkZY+2BuvDquWKaH0XRAq2CPnukDu0hWBNT6BHSvLFBOqP81FuvsJEP97ONksEK/qe/QdxkWT519vBkYFMsQ0QibXoYaSGFcEI6AffpcK+dyk+25fsK9y X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR0101MB2893.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(376002)(396003)(366004)(346002)(39850400004)(136003)(38350700002)(38100700002)(66476007)(66556008)(4326008)(8676002)(66946007)(316002)(86362001)(478600001)(6486002)(83380400001)(107886003)(41300700001)(966005)(26005)(2616005)(6506007)(52116002)(186003)(6512007)(1076003)(6666004)(54906003)(5660300002)(2906002)(8936002)(81973001);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Uf0YiqMPwyvPaQgre/AoTWda1CzEFxzHjuhDHuk9T7bpXQt8yQJcSO6EmGmOXc+rrcGxj8p+uLTLSebhdc6s3fU/UklQKXa5vRfJOz4qF5720L8+qD9QNOf95j/3v7xmBmG0t2mCUcrmeKby4hupYR3gV1lOpTmcA+yO/St08oQxtwzMcGcm1qH6zLK59Z40oggAfq03hIWohYNmt5qtXXomYttxCt0fITyFtYmgC6ZMlaxoIoBTxFtJKhxk483Di1X8yJT32nn2YnXMtlwq4ajzWki2PTvpuf2CiR1zvR8hR8Yw1pz4762L/md+PmiEz3Qb8eGo1DVF84sBhwex8s63HRolp7gH9qp9SGxvmS20zCL1GCMDrsVGOA06virQyZNVxYRp4z/3sT4eioAm+VJoGYI79r5nnXvQ4FkFmJiIZkbLPve2adMDCoD1UEuy0iq2cu8pNELFI2nGBU0QV/lTccbwNn1YICbIPmmu5wQwZoZPjIyKsWIw0meE24FXxn9saV23FpZqOWgW1J36YJpn+dKo9wQXLOwF1Pd662dwWahtnvALq6dmSNEU9qIlWqkJDjcvo/722ENQIVkytIqJsr7A8dJSOQsH+dhWJURytgPr50k6VN1ib+skIcSRx8BAfpXZhCTH/h/Yq9EwWDHv7C51KRlBUF+MwaWhpNE103rs6nEglETH7eQXCs9pa309W7V5GjOafg/EcUi2q4Z+Bt1nXR9EPZu0LgqXM64XuMO0m2v2lxAyVgVDxoEda8MearneJeUMP3iOGGe+2rekaEZj7/4+UZqNugfGf3z2QvBJaQ4lYDvj8lB4V4DsVha77AmMWTndy7otviiktjDw8Sb0xYjUBfCB5w4SJssAs63W2E7XFtS3S9f/1NElMA6ZIhPQc2iCTwdwrbe1hZfgsu3+xnsfIwvpfDFsp5jQz8rOn1ZL+LsyrqjWXYSqtfrPt0skRpWhmMhL1YmTECWv+ca7EgRXaZjk7gWRvZGDwi+ai4hQEq0fz+IxQbHf+BFbZuuoOD1ipoHCL8zw+XfIrSKRASrDCN1aCkY091w3GUQX6aONYbLwZcI3cxBndvg4vWh/oIOyjgcxeAurO0Z2wtBbHrnLdzkTSjcEe3EduK9SlRtZkdAfDCgVcDQihKkhvh+81zwm0iNpUy1oc2O1amJDIr0mKzSr0fLDbdGoV7J9pUOh92D/P19XKxBZVLmnSIqPaHe49mE2V7QXty81Fp90GamZJFG7PbMi+AsjfVaQVJ9TRXvKJ50WwbZv9F9D+hqIFcxTlMfML9xbKjAu8wPCW0Xig87TNtTFHefY9MtK3MO4fv3aUBKbb9WtXGnrLL8xq21i5n+Qi+N8l5/0FcqoC2q9ruiyvRDvpVkTSbl/tyPuSufftJaskzT1puvX/ROppzgDNtE1QTPlN37wWas8KJISf/vcJ8n7L1inpwPqDwwyAmTNPdA1mmH72HOFVzqB/fgLSf+bhv9n+AegKthC05CIbnj14xGFtBx5uLH5eZoCeObfKb3BJq89VgsdoemVeDpwMbWnuDN7/1NP1lj2faM0m7uNl7iaHG2INmRPdBGc/xNt3Ao7LOqFe4KYY5Pzh5xluaYR/X0q1g== X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: ac7b0074-d1f8-488c-e9ac-08da8c3070e9 X-MS-Exchange-CrossTenant-AuthSource: MWHPR0101MB2893.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2022 15:41:31.0324 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UKMF5/O1uM/YQGBZsX089AxCG6Fl2dEfPwiQgnsBY4dVYj1FcH1kElAlTCSzCMUb5eNeEYSE2YcJnI56cAe5IxhsiLDgkNkIufmmBVkg2rxp3Z0xBr8uEo73pdZuSHvM X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR01MB5619 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220901_084203_699635_1D684927 X-CRM114-Status: GOOD ( 16.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org GCC does not insert a `bti c` instruction at the beginning of a function when all callers reach the function through a direct branch[1]. In the case of cross-section calls (like __init to non __init), a thunk may be inserted which uses an indirect branch. If that happens, the first instruction in the callee function will result in a Branch Target Exception due to the missing `bti c`. Handle Branch Target Exceptions which happen in the kernel due to module calls from __init to non-__init by clearing PSTATE.BTYPE and resuming. [1]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106671 Signed-off-by: D Scott Phillips --- Changes since v1: - Add the gcc bug id into the traps.c comment - Cover the try_module_get with the preempt_disable - Add a CC_HAS_ config for the compiler bug that we'll eventually refine arch/arm64/Kconfig | 3 +++ arch/arm64/kernel/entry-common.c | 12 +++++++++ arch/arm64/kernel/traps.c | 43 ++++++++++++++++++++++++++++++-- 3 files changed, 56 insertions(+), 2 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index cd93c9041679..d5d4d2891657 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1860,6 +1860,9 @@ config ARM64_BTI_KERNEL is enabled and the system supports BTI all kernel code including modular code must have BTI enabled. +config CC_HAS_CROSS_SECTION_BTI_MISSING + def_bool CC_IS_GCC # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106671 + config CC_HAS_BRANCH_PROT_PAC_RET_BTI # GCC 9 or later, clang 8 or later def_bool $(cc-option,-mbranch-protection=pac-ret+leaf+bti) diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 56cefd33eb8e..696e3f3c90ea 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -388,6 +388,15 @@ static void noinstr el1_undef(struct pt_regs *regs) exit_to_kernel_mode(regs); } +static void noinstr el1_bti(struct pt_regs *regs) +{ + enter_from_kernel_mode(regs); + local_daif_inherit(regs); + do_bti(regs); + local_daif_mask(); + exit_to_kernel_mode(regs); +} + static void noinstr el1_dbg(struct pt_regs *regs, unsigned long esr) { unsigned long far = read_sysreg(far_el1); @@ -427,6 +436,9 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) case ESR_ELx_EC_UNKNOWN: el1_undef(regs); break; + case ESR_ELx_EC_BTI: + el1_bti(regs); + break; case ESR_ELx_EC_BREAKPT_CUR: case ESR_ELx_EC_SOFTSTP_CUR: case ESR_ELx_EC_WATCHPT_CUR: diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index 9ac7a81b79be..f1135166ecdb 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -501,8 +501,47 @@ NOKPROBE_SYMBOL(do_undefinstr); void do_bti(struct pt_regs *regs) { - BUG_ON(!user_mode(regs)); - force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0); + struct module *mod; + + if (user_mode(regs)) { + force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc, 0); + return; + } + + /* + * GCC does not insert a `bti c` instruction at the beginning + * of a function when all callers reach the function through a + * direct branch. In the case of cross-section calls (like + * __init to non __init), a thunk may be inserted which uses + * an indirect branch. If that happens, the first instruction + * in the callee function will result in a Branch Target + * Exception due to the missing `bti c`. + * + * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106671 + * + * If that's the case here, clear PSTATE.BTYPE and resume. + */ + if (IS_ENABLED(CONFIG_CC_HAS_CROSS_SECTION_BTI_MISSING)) { + preempt_disable(); + mod = __module_text_address(regs->pc); + + if (mod && try_module_get(mod)) { + bool from_init; + + from_init = within_module_init(regs->regs[30], mod); + module_put(mod); + + if (from_init) { + preempt_enable(); + regs->pstate &= ~PSR_BTYPE_MASK; + return; + } + } + + preempt_enable(); + } + + die("Oops - BTI", regs, 0); } NOKPROBE_SYMBOL(do_bti);