From patchwork Mon Jun 27 03:06:52 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Long X-Patchwork-Id: 9199849 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2C45B60754 for ; Mon, 27 Jun 2016 03:11:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 188C628555 for ; Mon, 27 Jun 2016 03:11:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0AEDC28564; Mon, 27 Jun 2016 03:11:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED, T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 82D2528555 for ; Mon, 27 Jun 2016 03:11:52 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.85_2 #1 (Red Hat Linux)) id 1bHMwP-0004xo-Tc; Mon, 27 Jun 2016 03:10:29 +0000 Received: from mail-qt0-x22b.google.com ([2607:f8b0:400d:c0d::22b]) by bombadil.infradead.org with esmtps (Exim 4.85_2 #1 (Red Hat Linux)) id 1bHMtZ-0000oa-DA for linux-arm-kernel@lists.infradead.org; Mon, 27 Jun 2016 03:07:34 +0000 Received: by mail-qt0-x22b.google.com with SMTP id m2so16555610qtd.1 for ; Sun, 26 Jun 2016 20:07:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=XIczb/yFLeJez+OCQXeNvfoSyZvAq4tU7qSZ3D48FT8=; b=BvnMQ33CKa0eIhh3p0eTTWji/3X54QmoByhH2J21x3FELdy89QugNoxB5getuLGMQ7 ubAY7zdMk1qwd4UmQxEJdkd6mc69noj8um8xlwLWxsY6r8tm1rfRc8zI+y3LPu4bViJX 9O/qX/yUWJ9RDFyMnvc4yWgODNdCgXDXi4lSg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=XIczb/yFLeJez+OCQXeNvfoSyZvAq4tU7qSZ3D48FT8=; b=lZ76QAn9gS5zkaX+z9huKRTHhchlynGDrxCSBzjPrZS0dtGU3Emhb8wp38zmgvns0a IRNViQEq0/eUz1K9pNPBtJWz0RwWGgTPjp0RvCCV7fLoLVsMydQUrvnjs1EgERZgALvh tNwSyPyPzkK5D6dG/xzwEVLnlpLtttT6ZkkYD06LPRuFQxzEpT5lTqRGddw2gjQC+RRZ V5aLfyPle+/RRG8QkDeVbhvWQEIIRoorq+lrY+gX5WOMz5BmNDTO0l+3arw7LhGWcDiq RNL9Szt8DIWd3s/mLfy0GdOnNF7uZXt4S7SlH2aL7Gx1lp39/Xwlnw8LE36A3zqaosPa IfRw== X-Gm-Message-State: ALyK8tIpZciOmYjE5N9UB5AHzoNBAYVK6G+y+UMhpplkNI3RouZupU5/VTPKyan4luXCm8fS X-Received: by 10.237.37.142 with SMTP id x14mr20905788qtc.9.1466996832347; Sun, 26 Jun 2016 20:07:12 -0700 (PDT) Received: from localhost.localdomain (pool-72-71-243-181.cncdnh.fast00.myfairpoint.net. [72.71.243.181]) by smtp.googlemail.com with ESMTPSA id e187sm262769qkf.35.2016.06.26.20.07.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 26 Jun 2016 20:07:11 -0700 (PDT) From: David Long To: Catalin Marinas , Huang Shijie , James Morse , Marc Zyngier , Pratyush Anand , Sandeepa Prabhu , Will Deacon , William Cohen , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Steve Capper , Masami Hiramatsu , Li Bin Subject: [PATCH v14 09/10] arm64: Add kernel return probes support (kretprobes) Date: Sun, 26 Jun 2016 23:06:52 -0400 Message-Id: <1466996813-23295-10-git-send-email-dave.long@linaro.org> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1466996813-23295-1-git-send-email-dave.long@linaro.org> References: <1466996813-23295-1-git-send-email-dave.long@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160626_200733_598211_3DA67E05 X-CRM114-Status: GOOD ( 19.77 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jisheng Zhang , Mark Rutland , Daniel Thompson , Vladimir Murzin , Petr Mladek , Ard Biesheuvel , Jens Wiklander , Robin Murphy , Mark Brown , Suzuki K Poulose , Dave P Martin , Andrey Ryabinin , yalin wang , Yang Shi , Zi Shen Lim , John Blackwood , Andrew Morton , =?UTF-8?q?Alex=20Benn=C3=A9e?= , Adam Buchbinder , Christoffer Dall MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Sandeepa Prabhu The pre-handler of this special 'trampoline' kprobe executes the return probe handler functions and restores original return address in ELR_EL1. This way the saved pt_regs still hold the original register context to be carried back to the probed kernel function. Signed-off-by: Sandeepa Prabhu Signed-off-by: David A. Long --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/kprobes/kprobes.c | 90 ++++++++++++++++++++++++++++++++++++- 2 files changed, 90 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 1f7d644..6af0e2e 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -89,6 +89,7 @@ config ARM64 select HAVE_RCU_TABLE_FREE select HAVE_SYSCALL_TRACEPOINTS select HAVE_KPROBES + select HAVE_KRETPROBES if HAVE_KPROBES select IOMMU_DMA if IOMMU_SUPPORT select IRQ_DOMAIN select IRQ_FORCED_THREADING diff --git a/arch/arm64/kernel/kprobes/kprobes.c b/arch/arm64/kernel/kprobes/kprobes.c index 89936d2..b5d9b9c 100644 --- a/arch/arm64/kernel/kprobes/kprobes.c +++ b/arch/arm64/kernel/kprobes/kprobes.c @@ -578,7 +578,95 @@ bool arch_within_kprobe_blacklist(unsigned long addr) void __kprobes __used *trampoline_probe_handler(struct pt_regs *regs) { - return NULL; + struct kretprobe_instance *ri = NULL; + struct hlist_head *head, empty_rp; + struct hlist_node *tmp; + unsigned long flags, orig_ret_address = 0; + unsigned long trampoline_address = + (unsigned long)&kretprobe_trampoline; + kprobe_opcode_t *correct_ret_addr = NULL; + + INIT_HLIST_HEAD(&empty_rp); + kretprobe_hash_lock(current, &head, &flags); + + /* + * It is possible to have multiple instances associated with a given + * task either because multiple functions in the call path have + * return probes installed on them, and/or more than one + * return probe was registered for a target function. + * + * We can handle this because: + * - instances are always pushed into the head of the list + * - when multiple return probes are registered for the same + * function, the (chronologically) first instance's ret_addr + * will be the real return address, and all the rest will + * point to kretprobe_trampoline. + */ + hlist_for_each_entry_safe(ri, tmp, head, hlist) { + if (ri->task != current) + /* another task is sharing our hash bucket */ + continue; + + orig_ret_address = (unsigned long)ri->ret_addr; + + if (orig_ret_address != trampoline_address) + /* + * This is the real return address. Any other + * instances associated with this task are for + * other calls deeper on the call stack + */ + break; + } + + kretprobe_assert(ri, orig_ret_address, trampoline_address); + + correct_ret_addr = ri->ret_addr; + hlist_for_each_entry_safe(ri, tmp, head, hlist) { + if (ri->task != current) + /* another task is sharing our hash bucket */ + continue; + + orig_ret_address = (unsigned long)ri->ret_addr; + if (ri->rp && ri->rp->handler) { + __this_cpu_write(current_kprobe, &ri->rp->kp); + get_kprobe_ctlblk()->kprobe_status = KPROBE_HIT_ACTIVE; + ri->ret_addr = correct_ret_addr; + ri->rp->handler(ri, regs); + __this_cpu_write(current_kprobe, NULL); + } + + recycle_rp_inst(ri, &empty_rp); + + if (orig_ret_address != trampoline_address) + /* + * This is the real return address. Any other + * instances associated with this task are for + * other calls deeper on the call stack + */ + break; + } + + kretprobe_hash_unlock(current, &flags); + + hlist_for_each_entry_safe(ri, tmp, &empty_rp, hlist) { + hlist_del(&ri->hlist); + kfree(ri); + } + return (void *)orig_ret_address; +} + +void __kprobes arch_prepare_kretprobe(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + ri->ret_addr = (kprobe_opcode_t *)regs->regs[30]; + + /* replace return addr (x30) with trampoline */ + regs->regs[30] = (long)&kretprobe_trampoline; +} + +int __kprobes arch_trampoline_kprobe(struct kprobe *p) +{ + return 0; } int __init arch_init_kprobes(void)