From patchwork Mon Mar 2 14:24:52 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wang Nan X-Patchwork-Id: 5914121 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 338F3BF440 for ; Mon, 2 Mar 2015 14:55:09 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 61583201FA for ; Mon, 2 Mar 2015 14:55:08 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7FBD620035 for ; Mon, 2 Mar 2015 14:55:07 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1YSRhS-0002rf-5R; Mon, 02 Mar 2015 14:52:02 +0000 Received: from casper.infradead.org ([2001:770:15f::2]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1YSRbq-0005rj-DZ for linux-arm-kernel@bombadil.infradead.org; Mon, 02 Mar 2015 14:46:14 +0000 Received: from szxga01-in.huawei.com ([119.145.14.64]) by casper.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1YSRJJ-0008Ai-5Y for linux-arm-kernel@lists.infradead.org; Mon, 02 Mar 2015 14:27:07 +0000 Received: from 172.24.2.119 (EHLO lggeml422-hub.china.huawei.com) ([172.24.2.119]) by szxrg01-dlp.huawei.com (MOS 4.3.7-GA FastPath queued) with ESMTP id CKE50597; Mon, 02 Mar 2015 22:25:54 +0800 (CST) Received: from kernel-host.huawei (10.107.197.247) by lggeml422-hub.china.huawei.com (10.72.61.32) with Microsoft SMTP Server id 14.3.158.1; Mon, 2 Mar 2015 22:25:43 +0800 From: Wang Nan To: , , , , Subject: [RFC PATCH v4 14/34] early kprobes: use stop_machine() based x86 optimizer. Date: Mon, 2 Mar 2015 22:24:52 +0800 Message-ID: <1425306312-3437-15-git-send-email-wangnan0@huawei.com> X-Mailer: git-send-email 1.8.4 In-Reply-To: <1425306312-3437-1-git-send-email-wangnan0@huawei.com> References: <1425306312-3437-1-git-send-email-wangnan0@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.107.197.247] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20150302_142706_234473_43777539 X-CRM114-Status: GOOD ( 11.12 ) X-Spam-Score: -4.2 (----) Cc: x86@kernel.org, lizefan@huawei.com, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Use stop_machine() to wrap code modification for x86 when optimizing early kprobes. Since early kprobes are registered before smp inited, text_poke_bp() is not ready at that time. This patch use stop_machine() based code modification for early kprobes. At very early stage, stop_machine() is simply irq operations. After kprobes fully initianized, we will use text_poke_bp(). Only kprobes registered after cpu_stop_init() before init_kprobes() will use real stop_machine(). Signed-off-by: Wang Nan --- arch/x86/kernel/kprobes/opt.c | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/arch/x86/kernel/kprobes/opt.c b/arch/x86/kernel/kprobes/opt.c index 7b3b9d1..ef3c0be 100644 --- a/arch/x86/kernel/kprobes/opt.c +++ b/arch/x86/kernel/kprobes/opt.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include @@ -377,6 +378,20 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, return 0; } +struct optimize_kprobe_early_param { + struct optimized_kprobe *op; + u8 *insn_buf; +}; + +static int optimize_kprobe_stop_machine(void *data) +{ + struct optimize_kprobe_early_param *param = data; + + text_poke_early(param->op->kp.addr, + param->insn_buf, RELATIVEJUMP_SIZE); + return 0; +} + /* * Replace breakpoints (int3) with relative jumps. * Caller must call with locking kprobe_mutex and text_mutex. @@ -399,8 +414,17 @@ void arch_optimize_kprobes(struct list_head *oplist) insn_buf[0] = RELATIVEJUMP_OPCODE; *(s32 *)(&insn_buf[1]) = rel; - text_poke_bp(op->kp.addr, insn_buf, RELATIVEJUMP_SIZE, - op->optinsn.insn); + if (unlikely(kprobes_is_early())) { + struct optimize_kprobe_early_param p = { + .op = op, + .insn_buf = insn_buf, + }; + + stop_machine(optimize_kprobe_stop_machine, &p, NULL); + } else { + text_poke_bp(op->kp.addr, insn_buf, RELATIVEJUMP_SIZE, + op->optinsn.insn); + } list_del_init(&op->list); }