From patchwork Mon Mar 7 11:53:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Doebel, Bjoern" X-Patchwork-Id: 12771624 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 80327C433EF for ; Mon, 7 Mar 2022 11:54:34 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.285880.485175 (Exim 4.92) (envelope-from ) id 1nRBwV-0004sM-Fi; Mon, 07 Mar 2022 11:54:23 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 285880.485175; Mon, 07 Mar 2022 11:54:23 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nRBwV-0004s9-Cb; Mon, 07 Mar 2022 11:54:23 +0000 Received: by outflank-mailman (input) for mailman id 285880; Mon, 07 Mar 2022 11:54:21 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nRBwT-0004Lv-Mp for xen-devel@lists.xenproject.org; Mon, 07 Mar 2022 11:54:21 +0000 Received: from smtp-fw-9102.amazon.com (smtp-fw-9102.amazon.com [207.171.184.29]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 51be7f8e-9e0d-11ec-8eba-a37418f5ba1a; Mon, 07 Mar 2022 12:54:20 +0100 (CET) Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO email-inbound-relay-iad-1e-7dac3c4d.us-east-1.amazon.com) ([10.25.36.214]) by smtp-border-fw-9102.sea19.amazon.com with ESMTP; 07 Mar 2022 11:54:03 +0000 Received: from EX13D03EUA003.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan3.iad.amazon.com [10.40.163.38]) by email-inbound-relay-iad-1e-7dac3c4d.us-east-1.amazon.com (Postfix) with ESMTPS id 8653E99B13; Mon, 7 Mar 2022 11:54:01 +0000 (UTC) Received: from EX13MTAUEA001.ant.amazon.com (10.43.61.82) by EX13D03EUA003.ant.amazon.com (10.43.165.89) with Microsoft SMTP Server (TLS) id 15.0.1497.28; Mon, 7 Mar 2022 11:54:00 +0000 Received: from dev-dsk-doebel-1c-c6d5f274.eu-west-1.amazon.com (10.13.240.106) by mail-relay.amazon.com (10.43.61.243) with Microsoft SMTP Server id 15.0.1497.28 via Frontend Transport; Mon, 7 Mar 2022 11:53:59 +0000 Received: by dev-dsk-doebel-1c-c6d5f274.eu-west-1.amazon.com (Postfix, from userid 3160037) id 61D514F5E; Mon, 7 Mar 2022 11:53:59 +0000 (UTC) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Inumbo-ID: 51be7f8e-9e0d-11ec-8eba-a37418f5ba1a DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.de; i=@amazon.de; q=dns/txt; s=amazon201209; t=1646654061; x=1678190061; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7eEisiDKKSHw1++xLYtfPHKS3ymk9DBq2ioO+9jWrTY=; b=lBKIm+cqWi3PpCyDIQtJaz+qO0O1a9HOHrtyfifZwJFUryjFTwa1Qm2A 2SOjFDyOA6YHtfbxRfiTD/so6hXxfT1ZOkTWfLzXdUT5ZbKd9ASqzQExu SEWGbblXeL2cCZ3VQHKPEKGTaHJcMesca7zQfEahBLdjjcFfWuS3+PVrb U=; X-IronPort-AV: E=Sophos;i="5.90,162,1643673600"; d="scan'208";a="200034811" From: Bjoern Doebel To: CC: Michael Kurth , Martin Pohlack , Roger Pau Monne , Andrew Cooper , Bjoern Doebel , Konrad Rzeszutek Wilk , Ross Lagerwall Subject: [PATCH 1/2] Livepatch: resolve old address before function verification Date: Mon, 7 Mar 2022 11:53:52 +0000 Message-ID: <15b092bb3af6e32f72ee8fca45317687d23b8be4.1646653825.git.doebel@amazon.de> X-Mailer: git-send-email 2.32.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: Bulk When verifying that a livepatch can be applied, we may as well want to inspect the target function to be patched. To do so, we need to resolve this function's address before running the arch-specific livepatch_verify hook. Signed-off-by: Bjoern Doebel CC: Konrad Rzeszutek Wilk CC: Ross Lagerwall Reviewed-by: Ross Lagerwall --- xen/common/livepatch.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/xen/common/livepatch.c b/xen/common/livepatch.c index ec301a9f12..be2cf75c2d 100644 --- a/xen/common/livepatch.c +++ b/xen/common/livepatch.c @@ -684,11 +684,11 @@ static int prepare_payload(struct payload *payload, return -EINVAL; } - rc = arch_livepatch_verify_func(f); + rc = resolve_old_address(f, elf); if ( rc ) return rc; - rc = resolve_old_address(f, elf); + rc = arch_livepatch_verify_func(f); if ( rc ) return rc; From patchwork Mon Mar 7 11:53:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Doebel, Bjoern" X-Patchwork-Id: 12771626 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2D230C433EF for ; Mon, 7 Mar 2022 11:54:42 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.285881.485186 (Exim 4.92) (envelope-from ) id 1nRBwd-0005LF-U4; Mon, 07 Mar 2022 11:54:31 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 285881.485186; Mon, 07 Mar 2022 11:54:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nRBwd-0005Kx-Qp; Mon, 07 Mar 2022 11:54:31 +0000 Received: by outflank-mailman (input) for mailman id 285881; Mon, 07 Mar 2022 11:54:30 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nRBwc-0004Lv-8O for xen-devel@lists.xenproject.org; Mon, 07 Mar 2022 11:54:30 +0000 Received: from smtp-fw-80006.amazon.com (smtp-fw-80006.amazon.com [99.78.197.217]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 56adcddc-9e0d-11ec-8eba-a37418f5ba1a; Mon, 07 Mar 2022 12:54:28 +0100 (CET) Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO email-inbound-relay-iad-1a-2d7489a4.us-east-1.amazon.com) ([10.25.36.214]) by smtp-border-fw-80006.pdx80.corp.amazon.com with ESMTP; 07 Mar 2022 11:54:08 +0000 Received: from EX13D44EUB004.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan2.iad.amazon.com [10.40.163.34]) by email-inbound-relay-iad-1a-2d7489a4.us-east-1.amazon.com (Postfix) with ESMTPS id 8EB46A277F; Mon, 7 Mar 2022 11:54:05 +0000 (UTC) Received: from EX13MTAUEA001.ant.amazon.com (10.43.61.82) by EX13D44EUB004.ant.amazon.com (10.43.166.198) with Microsoft SMTP Server (TLS) id 15.0.1497.28; Mon, 7 Mar 2022 11:54:04 +0000 Received: from dev-dsk-doebel-1c-c6d5f274.eu-west-1.amazon.com (10.13.240.106) by mail-relay.amazon.com (10.43.61.243) with Microsoft SMTP Server id 15.0.1497.28 via Frontend Transport; Mon, 7 Mar 2022 11:54:03 +0000 Received: by dev-dsk-doebel-1c-c6d5f274.eu-west-1.amazon.com (Postfix, from userid 3160037) id 4E48D4E7F; Mon, 7 Mar 2022 11:54:03 +0000 (UTC) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-Inumbo-ID: 56adcddc-9e0d-11ec-8eba-a37418f5ba1a DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.de; i=@amazon.de; q=dns/txt; s=amazon201209; t=1646654069; x=1678190069; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=T4HQVwdD+Sp1df6ID2a/ASUAEQvXVBttAi63ggcy0Qg=; b=oqPK8vc+PN0FW5dmOzkrjNIO2hC8qvNtHVSV7ncHGGbsAHLeWuXzNre5 WLmZE+qVdmG9aQhv6UxebDJkZ7NeQVvl4MB/LNHtOIfD+mpzjAGQkuUZR B0SnZuV0F0g+d/5AbHpgto6QXF/P/LwLU9k8lkt28Yux0jzX0b944bpqr A=; X-IronPort-AV: E=Sophos;i="5.90,162,1643673600"; d="scan'208";a="68586367" From: Bjoern Doebel To: CC: Michael Kurth , Martin Pohlack , Roger Pau Monne , Andrew Cooper , Bjoern Doebel , Konrad Rzeszutek Wilk , Ross Lagerwall Subject: [PATCH 2/2] xen/x86: Livepatch: support patching CET-enhanced functions Date: Mon, 7 Mar 2022 11:53:54 +0000 Message-ID: X-Mailer: git-send-email 2.32.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: Bulk Xen enabled CET for supporting architectures. The control flow aspect of CET expects functions that can be called indirectly (i.e., via function pointers) to start with an ENDBR64 instruction. Otherwise a control flow exception is raised. This expectation breaks livepatching flows because we patch functions by overwriting their first 5 bytes with a JMP + , thus breaking the ENDBR64. We fix this by checking the start of a patched function for being ENDBR64. In the positive case we move the livepatch JMP to start behind the ENDBR64 instruction. To avoid having to guess the ENDBR64 offset again on patch reversal (which might race with other mechanisms adding/removing ENDBR dynamically), use the livepatch metadata to store the computed offset along with the saved bytes of the overwritten function. Signed-off-by: Bjoern Doebel CC: Konrad Rzeszutek Wilk CC: Ross Lagerwall ---- Note that on top of livepatching functions, Xen supports an additional mode where we can "remove" a function by overwriting it with NOPs. This is only supported for functions up to 31 bytes in size and this patch reduces this limit to 30 bytes. --- xen/arch/x86/livepatch.c | 63 +++++++++++++++++++++++++++++++++++----- 1 file changed, 55 insertions(+), 8 deletions(-) diff --git a/xen/arch/x86/livepatch.c b/xen/arch/x86/livepatch.c index 65530c1e57..da7611c01d 100644 --- a/xen/arch/x86/livepatch.c +++ b/xen/arch/x86/livepatch.c @@ -14,11 +14,29 @@ #include #include +#include #include #include #include #include +/* + * CET hotpatching support: We may have functions starting with an ENDBR64 + * instruction that MUST remain the first instruction of the function, hence + * we need to move any hotpatch trampoline further into the function. For that + * we need to keep track of the patching offset used for any loaded hotpatch + * (to avoid racing against other fixups adding/removing ENDBR64 or similar + * instructions). + * + * We do so by making use of the existing opaque metadata area. We use its + * first 4 bytes to track the offset into the function used for patching and + * the remainder of the data to store overwritten code bytes. + */ +struct x86_livepatch_meta { + uint8_t patch_offset; + uint8_t instruction[LIVEPATCH_OPAQUE_SIZE - sizeof(uint8_t)]; +}; + static bool has_active_waitqueue(const struct vm_event_domain *ved) { /* ved may be xzalloc()'d without INIT_LIST_HEAD() yet. */ @@ -104,18 +122,36 @@ void noinline arch_livepatch_revive(void) int arch_livepatch_verify_func(const struct livepatch_func *func) { + BUILD_BUG_ON(sizeof(struct x86_livepatch_meta) != LIVEPATCH_OPAQUE_SIZE); + /* If NOPing.. */ if ( !func->new_addr ) { + struct x86_livepatch_meta *lp; + + lp = (struct x86_livepatch_meta *)func->opaque; /* Only do up to maximum amount we can put in the ->opaque. */ - if ( func->new_size > sizeof(func->opaque) ) + if ( func->new_size > sizeof(lp->instruction) ) return -EOPNOTSUPP; if ( func->old_size < func->new_size ) return -EINVAL; } - else if ( func->old_size < ARCH_PATCH_INSN_SIZE ) - return -EINVAL; + else + { + /* + * Space needed now depends on whether the target function + * starts with an ENDBR64 instruction. + */ + uint8_t needed; + + needed = ARCH_PATCH_INSN_SIZE; + if ( is_endbr64(func->old_addr) ) + needed += ENDBR64_LEN; + + if ( func->old_size < needed ) + return -EINVAL; + } return 0; } @@ -127,15 +163,21 @@ int arch_livepatch_verify_func(const struct livepatch_func *func) void noinline arch_livepatch_apply(struct livepatch_func *func) { uint8_t *old_ptr; - uint8_t insn[sizeof(func->opaque)]; + struct x86_livepatch_meta *lp; + uint8_t insn[sizeof(lp->instruction)]; unsigned int len; + lp = (struct x86_livepatch_meta *)func->opaque; + lp->patch_offset = 0; old_ptr = func->old_addr; len = livepatch_insn_len(func); if ( !len ) return; - memcpy(func->opaque, old_ptr, len); + if ( is_endbr64(old_ptr) ) + lp->patch_offset += ENDBR64_LEN; + + memcpy(lp->instruction, old_ptr + lp->patch_offset, len); if ( func->new_addr ) { int32_t val; @@ -143,14 +185,15 @@ void noinline arch_livepatch_apply(struct livepatch_func *func) BUILD_BUG_ON(ARCH_PATCH_INSN_SIZE != (1 + sizeof(val))); insn[0] = 0xe9; /* Relative jump. */ - val = func->new_addr - func->old_addr - ARCH_PATCH_INSN_SIZE; + val = func->new_addr - (func->old_addr + lp->patch_offset + + ARCH_PATCH_INSN_SIZE); memcpy(&insn[1], &val, sizeof(val)); } else add_nops(insn, len); - memcpy(old_ptr, insn, len); + memcpy(old_ptr + lp->patch_offset, insn, len); } /* @@ -159,7 +202,11 @@ void noinline arch_livepatch_apply(struct livepatch_func *func) */ void noinline arch_livepatch_revert(const struct livepatch_func *func) { - memcpy(func->old_addr, func->opaque, livepatch_insn_len(func)); + struct x86_livepatch_meta *lp; + + lp = (struct x86_livepatch_meta *)func->opaque; + + memcpy(func->old_addr + lp->patch_offset, lp->instruction, livepatch_insn_len(func)); } /*