From patchwork Fri Sep 23 08:46:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liao, Chang" X-Patchwork-Id: 12986317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F0948ECAAD8 for ; Fri, 23 Sep 2022 08:51:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=e9YXfTX/xorzfTfY+QryN48o+LISglxgY9MtFwcvZbI=; b=D1euOaOfShbw7T AJKrXHRjz7zCIzBUmIj2XWPbWjFRfc3pWZGPGB3inusZYduNf9ZqOTTD7IM5jdp+RSj1RNIjbgCSH xAms9JG/LmnDvDDatfQYTWssdccPMZInX8XLToyTIUEhgF96tWA6KNfosTRwqCyiDTxMOKAjjWd6G 7Y/bd73JJSaZiIQBl87swverRDAjrcMFtq+aYZ52IbgT2BicGAj9Gxx3hGWXXH9s+y/zQ0YZy6eDA TAADriRJ39LZlm9VlU+WfnoVoyV9/POlO9eEYPfJ8GcOISY1GB3scxtaxHzNJFpFICaXnJmXpRNxO EqbnfEMSSA2DKSk4CZpA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1obeOm-0033YU-H5; Fri, 23 Sep 2022 08:51:04 +0000 Received: from szxga01-in.huawei.com ([45.249.212.187]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1obeOS-0033QH-EK; Fri, 23 Sep 2022 08:50:45 +0000 Received: from kwepemi500012.china.huawei.com (unknown [172.30.72.53]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4MYm4j0QwHzpT3m; Fri, 23 Sep 2022 16:47:49 +0800 (CST) Received: from huawei.com (10.67.174.53) by kwepemi500012.china.huawei.com (7.221.188.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 23 Sep 2022 16:50:39 +0800 From: Liao Chang To: , , , , , , , , , , , CC: , , , Subject: [PATCH 1/3] riscv/kprobe: Optimize the performance of patching single-step slot Date: Fri, 23 Sep 2022 16:46:56 +0800 Message-ID: <20220923084658.99304-2-liaochang1@huawei.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220923084658.99304-1-liaochang1@huawei.com> References: <20220923084658.99304-1-liaochang1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.53] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemi500012.china.huawei.com (7.221.188.12) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220923_015044_694102_8DC1BB13 X-CRM114-Status: UNSURE ( 8.84 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Single-step slot would not be used until kprobe is enabled, that means no race condition occurs on it under SMP, hence it is safe to pacth ss slot without stopping machine. Acked-by: Masami Hiramatsu (Google) Signed-off-by: Liao Chang --- arch/riscv/kernel/probes/kprobes.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/arch/riscv/kernel/probes/kprobes.c b/arch/riscv/kernel/probes/kprobes.c index e6e950b7cf32..bc1f39b96e41 100644 --- a/arch/riscv/kernel/probes/kprobes.c +++ b/arch/riscv/kernel/probes/kprobes.c @@ -24,12 +24,14 @@ post_kprobe_handler(struct kprobe *, struct kprobe_ctlblk *, struct pt_regs *); static void __kprobes arch_prepare_ss_slot(struct kprobe *p) { unsigned long offset = GET_INSN_LENGTH(p->opcode); + kprobe_opcode_t slot[MAX_INSN_SIZE]; p->ainsn.api.restore = (unsigned long)p->addr + offset; - patch_text(p->ainsn.api.insn, p->opcode); - patch_text((void *)((unsigned long)(p->ainsn.api.insn) + offset), - __BUG_INSN_32); + memcpy(slot, &p->opcode, offset); + *(kprobe_opcode_t *)((unsigned long)slot + offset) = __BUG_INSN_32; + patch_text_nosync(p->ainsn.api.insn, slot, + offset + GET_INSN_LENGTH(__BUG_INSN_32)); } static void __kprobes arch_prepare_simulate(struct kprobe *p) From patchwork Fri Sep 23 08:46:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liao, Chang" X-Patchwork-Id: 12986319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E7DD5C6FA90 for ; Fri, 23 Sep 2022 08:51:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=QhkN1RuuT66n3ihis04RhHPaj2yidvLmDjoNn0PgaB4=; b=kGyMG4C0YVrw/1 U13NKZJWEG9HhcJvzN+XHozfvnEgCNNrIlV1Tso5zgcjW1te7zbYsRCPrttPiyWPRQ8k6x7wl7Gsn ECSaru0Bnk9hgpfqkIlzdqJyJbhABAHOnBk/KCBZiaYAhitKBYH3du4tsFD57bpFAhAiqWvgGDY35 804bcUxdceEMatC30Lju/TI9dquIIU1r5vth2UYh1tnaa//p9kOmKyOfES3x3Ui+LklF41v0f/uC4 Xmv+qmm8rQN4oga1xB+NNqKPErdJZG169Pwp6+Hhf8IFEw3jb5EtL81KjoBr5hA5lrvGM//ygr71l evwXhpuJLsWMsJl+RcqQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1obeOp-0033Zz-IE; Fri, 23 Sep 2022 08:51:07 +0000 Received: from szxga02-in.huawei.com ([45.249.212.188]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1obeOS-0033R5-EI; Fri, 23 Sep 2022 08:50:47 +0000 Received: from kwepemi500012.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4MYm3Q04rLzWgpB; Fri, 23 Sep 2022 16:46:41 +0800 (CST) Received: from huawei.com (10.67.174.53) by kwepemi500012.china.huawei.com (7.221.188.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 23 Sep 2022 16:50:39 +0800 From: Liao Chang To: , , , , , , , , , , , CC: , , , Subject: [PATCH 2/3] csky/kprobe: Optimize the performance of patching single-step slot Date: Fri, 23 Sep 2022 16:46:57 +0800 Message-ID: <20220923084658.99304-3-liaochang1@huawei.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220923084658.99304-1-liaochang1@huawei.com> References: <20220923084658.99304-1-liaochang1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.53] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemi500012.china.huawei.com (7.221.188.12) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220923_015044_694673_F4E42BCD X-CRM114-Status: UNSURE ( 7.91 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Single-step slot would not be used until kprobe is enabled, that means no race condition occurs on it under SMP, hence it is safe to pacth ss slot without stopping machine. Acked-by: Masami Hiramatsu (Google) Signed-off-by: Liao Chang --- arch/csky/kernel/probes/kprobes.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/csky/kernel/probes/kprobes.c b/arch/csky/kernel/probes/kprobes.c index 3c6e5c725d81..4feb5ce16264 100644 --- a/arch/csky/kernel/probes/kprobes.c +++ b/arch/csky/kernel/probes/kprobes.c @@ -57,7 +57,11 @@ static void __kprobes arch_prepare_ss_slot(struct kprobe *p) p->ainsn.api.restore = (unsigned long)p->addr + offset; - patch_text(p->ainsn.api.insn, p->opcode); + memcpy(p->ainsn.api.insn, &p->opcode, offset); + dcache_wb_range((unsigned long)p->ainsn.api.insn, + (unsigned long)p->ainsn.api.insn + offset); + icache_inv_range((unsigned long)p->ainsn.api.insn, + (unsigned long)p->ainsn.api.insn + offset); } static void __kprobes arch_prepare_simulate(struct kprobe *p) From patchwork Fri Sep 23 08:46:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Liao, Chang" X-Patchwork-Id: 12986318 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4DA62C6FA82 for ; Fri, 23 Sep 2022 08:51:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tbJ+n30NM9NBR01oMAfSX++lK1rJJG+VZGsmxziD+Fk=; b=W9sLnGWt6RPMlR W5pU+72jDABxFuH92dIIQv/52XschXanesqUFvX4pMhxbREz7UOl67LaC6CE1R1mSZzOYoub1k/yx KlcRE+52e738Yx5wm2m8YkaDdnF56Z5Ez4v9Fh9jYEmAqdfSarZ07crxkID0afQmMffj5aJnnY4jc sxjb6yU3taFqCqdhujNQYGJaj4HOVWXCU6Fwt306SlKrCkVemtabuLCnI6r0x5BUPljJ6O/c3sS9U S1GOT5QzgV5yQmUPW9R6jVvLcjxE7N0fPFYTRhFRMTAX2nB0GaE1oToLLaHF65pG2DFjvlwtzYhOG 8sCtGv7Dgl+uDkx2i1uw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1obeOn-0033Z6-TH; Fri, 23 Sep 2022 08:51:05 +0000 Received: from szxga02-in.huawei.com ([45.249.212.188]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1obeOS-0033R6-EI; Fri, 23 Sep 2022 08:50:45 +0000 Received: from kwepemi500012.china.huawei.com (unknown [172.30.72.53]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4MYm3Q4S2lzWgy5; Fri, 23 Sep 2022 16:46:42 +0800 (CST) Received: from huawei.com (10.67.174.53) by kwepemi500012.china.huawei.com (7.221.188.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 23 Sep 2022 16:50:40 +0800 From: Liao Chang To: , , , , , , , , , , , CC: , , , Subject: [PATCH 3/3] arm64/kprobe: Optimize the performance of patching single-step slot Date: Fri, 23 Sep 2022 16:46:58 +0800 Message-ID: <20220923084658.99304-4-liaochang1@huawei.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220923084658.99304-1-liaochang1@huawei.com> References: <20220923084658.99304-1-liaochang1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.67.174.53] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemi500012.china.huawei.com (7.221.188.12) X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220923_015044_694985_B17AF750 X-CRM114-Status: UNSURE ( 9.23 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Single-step slot would not be used until kprobe is enabled, that means no race condition occurs on it under SMP, hence it is safe to pacth ss slot without stopping machine. Since I and D caches are coherent within single-step slot from aarch64_insn_patch_text_nosync(), hence no need to do it again via flush_icache_range(). Acked-by: Masami Hiramatsu (Google) Signed-off-by: Liao Chang Acked-by: Will Deacon --- arch/arm64/kernel/probes/kprobes.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c index d1d182320245..29b98bc12833 100644 --- a/arch/arm64/kernel/probes/kprobes.c +++ b/arch/arm64/kernel/probes/kprobes.c @@ -44,13 +44,10 @@ post_kprobe_handler(struct kprobe *, struct kprobe_ctlblk *, struct pt_regs *); static void __kprobes arch_prepare_ss_slot(struct kprobe *p) { kprobe_opcode_t *addr = p->ainsn.api.insn; - void *addrs[] = {addr, addr + 1}; - u32 insns[] = {p->opcode, BRK64_OPCODE_KPROBES_SS}; /* prepare insn slot */ - aarch64_insn_patch_text(addrs, insns, 2); - - flush_icache_range((uintptr_t)addr, (uintptr_t)(addr + MAX_INSN_SIZE)); + aarch64_insn_patch_text_nosync(addr, p->opcode); + aarch64_insn_patch_text_nosync(addr + 1, BRK64_OPCODE_KPROBES_SS); /* * Needs restoring of return address after stepping xol.