From patchwork Fri Jul 8 16:35:53 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Long X-Patchwork-Id: 9221437 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id ABAF960572 for ; Fri, 8 Jul 2016 16:39:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9AE24262F2 for ; Fri, 8 Jul 2016 16:39:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8F1E426538; Fri, 8 Jul 2016 16:39:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED, T_DKIM_INVALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3AF2F262F2 for ; Fri, 8 Jul 2016 16:39:50 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.85_2 #1 (Red Hat Linux)) id 1bLYnM-0006QA-Nh; Fri, 08 Jul 2016 16:38:28 +0000 Received: from mail-qk0-x235.google.com ([2607:f8b0:400d:c09::235]) by bombadil.infradead.org with esmtps (Exim 4.85_2 #1 (Red Hat Linux)) id 1bLYlZ-000499-H8 for linux-arm-kernel@lists.infradead.org; Fri, 08 Jul 2016 16:36:40 +0000 Received: by mail-qk0-x235.google.com with SMTP id s63so5863356qkb.2 for ; Fri, 08 Jul 2016 09:36:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=fWjNojju5UJqhKKRXfLQL5/k9noTBCqUSJXf4jeUL6s=; b=ZiNTiZd3ilZ6u7Hvn7iOsn5NMgEKg617Q1nfZxyxYPznX3h/OVtZp21Qs3wnldMxpy lx+0iXQkzIGSx/t26bSuiWG5fikOVYMLtMwB8zpyjC9FOQfkYskIud0hl4eSSPCrQgi2 kaztWGASo4KgTzbVOi2sL5HtNGCUXYN1Q2qlg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=fWjNojju5UJqhKKRXfLQL5/k9noTBCqUSJXf4jeUL6s=; b=ZPdQ4PHRMAxqnJNLEytZNcBbfmpv8rg6ZKKQTmrBiUUKkM0260K9+UhEjo2cx72m9J kJukrfvlqjI3u6ePSSCqyHM4GkpYxEPon8jFhpklv6hbtkBN2WKGX+DpRQr3sorR0Tw9 ISV9L/8Mhc2JG2MkOhZ6WHWZ/7BbVYt0eEZIW6AgNth8+xVW3SXRM4RlIWx+6Plykslr vw+oZqP5Acquvm3BCkl0jEXGzraqZI9ewZgKCJ5ep4jo5rwfhrcTjOGrepvvQwp5QIX5 +aXQ4MzlCpmAnhrqRAnCKmFYi4GvBKIlF7qsYDUhqqmhfYSlX94KzxLYQxdyJy+J6eXt 298w== X-Gm-Message-State: ALyK8tISA8dXNQEhaCRDGLdtPGv6XrfiFQzpxkXsMc4Yg042H2H9glRdXgZDwfismssfq2s7 X-Received: by 10.55.183.129 with SMTP id h123mr8303361qkf.105.1467995776275; Fri, 08 Jul 2016 09:36:16 -0700 (PDT) Received: from localhost.localdomain (pool-72-71-243-181.cncdnh.fast00.myfairpoint.net. [72.71.243.181]) by smtp.googlemail.com with ESMTPSA id l27sm2254627qtc.34.2016.07.08.09.36.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 08 Jul 2016 09:36:15 -0700 (PDT) From: David Long To: Catalin Marinas , Huang Shijie , James Morse , Marc Zyngier , Pratyush Anand , Sandeepa Prabhu , Will Deacon , William Cohen , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Steve Capper , Masami Hiramatsu , Li Bin Subject: [PATCH v15 09/10] arm64: Add kernel return probes support (kretprobes) Date: Fri, 8 Jul 2016 12:35:53 -0400 Message-Id: <1467995754-32508-10-git-send-email-dave.long@linaro.org> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1467995754-32508-1-git-send-email-dave.long@linaro.org> References: <1467995754-32508-1-git-send-email-dave.long@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160708_093637_912213_582C8C30 X-CRM114-Status: GOOD ( 19.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jisheng Zhang , Mark Rutland , Daniel Thompson , Vladimir Murzin , Petr Mladek , Ard Biesheuvel , Jens Wiklander , Robin Murphy , Mark Brown , Suzuki K Poulose , Dave P Martin , Andrey Ryabinin , yalin wang , Yang Shi , Zi Shen Lim , John Blackwood , Andrew Morton , =?UTF-8?q?Alex=20Benn=C3=A9e?= , Adam Buchbinder , Christoffer Dall MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Sandeepa Prabhu The pre-handler of this special 'trampoline' kprobe executes the return probe handler functions and restores original return address in ELR_EL1. This way the saved pt_regs still hold the original register context to be carried back to the probed kernel function. Signed-off-by: Sandeepa Prabhu Signed-off-by: David A. Long Acked-by: Masami Hiramatsu --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/probes/kprobes.c | 90 +++++++++++++++++++++++++++++++++++++- 2 files changed, 90 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 1f7d644..6af0e2e 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -89,6 +89,7 @@ config ARM64 select HAVE_RCU_TABLE_FREE select HAVE_SYSCALL_TRACEPOINTS select HAVE_KPROBES + select HAVE_KRETPROBES if HAVE_KPROBES select IOMMU_DMA if IOMMU_SUPPORT select IRQ_DOMAIN select IRQ_FORCED_THREADING diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c index be1f074..9c70e88 100644 --- a/arch/arm64/kernel/probes/kprobes.c +++ b/arch/arm64/kernel/probes/kprobes.c @@ -578,7 +578,95 @@ bool arch_within_kprobe_blacklist(unsigned long addr) void __kprobes __used *trampoline_probe_handler(struct pt_regs *regs) { - return NULL; + struct kretprobe_instance *ri = NULL; + struct hlist_head *head, empty_rp; + struct hlist_node *tmp; + unsigned long flags, orig_ret_address = 0; + unsigned long trampoline_address = + (unsigned long)&kretprobe_trampoline; + kprobe_opcode_t *correct_ret_addr = NULL; + + INIT_HLIST_HEAD(&empty_rp); + kretprobe_hash_lock(current, &head, &flags); + + /* + * It is possible to have multiple instances associated with a given + * task either because multiple functions in the call path have + * return probes installed on them, and/or more than one + * return probe was registered for a target function. + * + * We can handle this because: + * - instances are always pushed into the head of the list + * - when multiple return probes are registered for the same + * function, the (chronologically) first instance's ret_addr + * will be the real return address, and all the rest will + * point to kretprobe_trampoline. + */ + hlist_for_each_entry_safe(ri, tmp, head, hlist) { + if (ri->task != current) + /* another task is sharing our hash bucket */ + continue; + + orig_ret_address = (unsigned long)ri->ret_addr; + + if (orig_ret_address != trampoline_address) + /* + * This is the real return address. Any other + * instances associated with this task are for + * other calls deeper on the call stack + */ + break; + } + + kretprobe_assert(ri, orig_ret_address, trampoline_address); + + correct_ret_addr = ri->ret_addr; + hlist_for_each_entry_safe(ri, tmp, head, hlist) { + if (ri->task != current) + /* another task is sharing our hash bucket */ + continue; + + orig_ret_address = (unsigned long)ri->ret_addr; + if (ri->rp && ri->rp->handler) { + __this_cpu_write(current_kprobe, &ri->rp->kp); + get_kprobe_ctlblk()->kprobe_status = KPROBE_HIT_ACTIVE; + ri->ret_addr = correct_ret_addr; + ri->rp->handler(ri, regs); + __this_cpu_write(current_kprobe, NULL); + } + + recycle_rp_inst(ri, &empty_rp); + + if (orig_ret_address != trampoline_address) + /* + * This is the real return address. Any other + * instances associated with this task are for + * other calls deeper on the call stack + */ + break; + } + + kretprobe_hash_unlock(current, &flags); + + hlist_for_each_entry_safe(ri, tmp, &empty_rp, hlist) { + hlist_del(&ri->hlist); + kfree(ri); + } + return (void *)orig_ret_address; +} + +void __kprobes arch_prepare_kretprobe(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + ri->ret_addr = (kprobe_opcode_t *)regs->regs[30]; + + /* replace return addr (x30) with trampoline */ + regs->regs[30] = (long)&kretprobe_trampoline; +} + +int __kprobes arch_trampoline_kprobe(struct kprobe *p) +{ + return 0; } int __init arch_init_kprobes(void)