From patchwork Thu Feb 18 23:48:20 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Long X-Patchwork-Id: 8355121 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 0AF01C0553 for ; Thu, 18 Feb 2016 23:52:44 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 26F46203E6 for ; Thu, 18 Feb 2016 23:52:43 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.9]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3EB43203B6 for ; Thu, 18 Feb 2016 23:52:42 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1aWYLT-000751-Rx; Thu, 18 Feb 2016 23:50:51 +0000 Received: from mail-qg0-x22b.google.com ([2607:f8b0:400d:c04::22b]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1aWYJf-0004Gn-L9 for linux-arm-kernel@lists.infradead.org; Thu, 18 Feb 2016 23:49:04 +0000 Received: by mail-qg0-x22b.google.com with SMTP id y89so49900212qge.2 for ; Thu, 18 Feb 2016 15:48:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=9CfVyYedfGbIzJdJ2yNubEoRJscyGVZgxpJa78vDX2A=; b=ATiCFrZ8bbfpTY18rIUu63Y3avy5LoX3oVcGHmO166pXafGy9SjTNdtzVzq+YsDAAW tvo0Adh0l2AXVa3JseitHkD1rfMNb0AtT6EI+jFlvd/+a7KQCdT9l7cIlwZ1xVn2k2fB ostkPOKAG4em+DPOM/yw9xnIWL/tD08u6kEUk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9CfVyYedfGbIzJdJ2yNubEoRJscyGVZgxpJa78vDX2A=; b=KVGWdyPUnQEgRvMt14Q2k4WNjfNjnY+Q2iUNPmYmZD9STRjwNjm6FbL22NObQKvBnP HbtXrJeryWZlTe0aAdwwX9ppTpTwsTi1zM+NnmNFD4P4x6/hUPj3z3aBRAJ/LsnvkGjR fw8DsKk+Gt91mPgZZogAB6VHiKkKtNm/bZyTZdFVeWzZScaY4FGz+QZthEsMkjXRJ9N9 QO+17AvWejW2lgVpgHHq8eZjOrhkTgJdEgwmhbZScXxV2Qs7+ZrJvz8dfFqn0Zu6n35N P055W6El7YY7g7M7VuKZP3EsQhbK+lgA5Ca6PjYolHWOjcV1OnfMlfeDcKlFgeVJH5rs 2ZCA== X-Gm-Message-State: AG10YOQ+VPwIixf5CBEW/Y7QmxIDfNrFZsyCROAjH83I/EEA/tpyZ+9KNBgCtNdu9a8uhdnc X-Received: by 10.140.20.183 with SMTP id 52mr12260523qgj.38.1455839318697; Thu, 18 Feb 2016 15:48:38 -0800 (PST) Received: from localhost.localdomain (pool-72-71-243-249.cncdnh.fast00.myfairpoint.net. [72.71.243.249]) by smtp.googlemail.com with ESMTPSA id w70sm3711845qge.13.2016.02.18.15.48.37 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 18 Feb 2016 15:48:38 -0800 (PST) From: David Long To: Catalin Marinas , Will Deacon , Sandeepa Prabhu , William Cohen , Pratyush Anand , Steve Capper , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 7/8] arm64: Add kernel return probes support (kretprobes) Date: Thu, 18 Feb 2016 18:48:20 -0500 Message-Id: <1455839301-26464-8-git-send-email-dave.long@linaro.org> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1455839301-26464-1-git-send-email-dave.long@linaro.org> References: <1455839301-26464-1-git-send-email-dave.long@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160218_154859_946289_DBD19E76 X-CRM114-Status: GOOD ( 19.42 ) X-Spam-Score: -2.7 (--) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Petr Mladek , Viresh Kumar , John Blackwood , Feng Kan , Zi Shen Lim , Dave P Martin , Yang Shi , Vladimir Murzin , Kees Cook , "Suzuki K. Poulose" , Marc Zyngier , Mark Brown , =?UTF-8?q?Alex=20Benn=C3=A9e?= , Ard Biesheuvel , Greg Kroah-Hartman , Mark Salyzyn , James Morse , Christoffer Dall , Andrew Morton , Robin Murphy , Jens Wiklander , Balamurugan Shanmugam MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Spam-Status: No, score=-4.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Sandeepa Prabhu The pre-handler of this special 'trampoline' kprobe executes the return probe handler functions and restores original return address in ELR_EL1. This way the saved pt_regs still hold the original register context to be carried back to the probed kernel function. Signed-off-by: Sandeepa Prabhu Signed-off-by: David A. Long --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/kprobes.c | 75 ++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 75 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index c395386..72412de 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -82,6 +82,7 @@ config ARM64 select HAVE_RCU_TABLE_FREE select HAVE_SYSCALL_TRACEPOINTS select HAVE_KPROBES + select HAVE_KRETPROBES if HAVE_KPROBES select IOMMU_DMA if IOMMU_SUPPORT select IRQ_DOMAIN select IRQ_FORCED_THREADING diff --git a/arch/arm64/kernel/kprobes.c b/arch/arm64/kernel/kprobes.c index 0ce4637..a699d03 100644 --- a/arch/arm64/kernel/kprobes.c +++ b/arch/arm64/kernel/kprobes.c @@ -533,7 +533,80 @@ int __kprobes longjmp_break_handler(struct kprobe *p, struct pt_regs *regs) void __kprobes __used *trampoline_probe_handler(struct pt_regs *regs) { - return (void *) 0; + struct kretprobe_instance *ri = NULL; + struct hlist_head *head, empty_rp; + struct hlist_node *tmp; + unsigned long flags, orig_ret_addr = 0; + unsigned long trampoline_address = + (unsigned long)&kretprobe_trampoline; + + INIT_HLIST_HEAD(&empty_rp); + kretprobe_hash_lock(current, &head, &flags); + + /* + * It is possible to have multiple instances associated with a given + * task either because multiple functions in the call path have + * a return probe installed on them, and/or more than one return + * probe was registered for a target function. + * + * We can handle this because: + * - instances are always inserted at the head of the list + * - when multiple return probes are registered for the same + * function, the first instance's ret_addr will point to the + * real return address, and all the rest will point to + * kretprobe_trampoline + */ + hlist_for_each_entry_safe(ri, tmp, head, hlist) { + if (ri->task != current) + /* another task is sharing our hash bucket */ + continue; + + if (ri->rp && ri->rp->handler) { + __this_cpu_write(current_kprobe, &ri->rp->kp); + get_kprobe_ctlblk()->kprobe_status = KPROBE_HIT_ACTIVE; + ri->rp->handler(ri, regs); + __this_cpu_write(current_kprobe, NULL); + } + + orig_ret_addr = (unsigned long)ri->ret_addr; + recycle_rp_inst(ri, &empty_rp); + + if (orig_ret_addr != trampoline_address) + /* + * This is the real return address. Any other + * instances associated with this task are for + * other calls deeper on the call stack + */ + break; + } + + kretprobe_assert(ri, orig_ret_addr, trampoline_address); + /* restore the original return address */ + instruction_pointer(regs) = orig_ret_addr; + reset_current_kprobe(); + kretprobe_hash_unlock(current, &flags); + + hlist_for_each_entry_safe(ri, tmp, &empty_rp, hlist) { + hlist_del(&ri->hlist); + kfree(ri); + } + + /* return 1 so that post handlers not called */ + return (void *) orig_ret_addr; +} + +void __kprobes arch_prepare_kretprobe(struct kretprobe_instance *ri, + struct pt_regs *regs) +{ + ri->ret_addr = (kprobe_opcode_t *)regs->regs[30]; + + /* replace return addr (x30) with trampoline */ + regs->regs[30] = (long)&kretprobe_trampoline; +} + +int __kprobes arch_trampoline_kprobe(struct kprobe *p) +{ + return 0; } int __init arch_init_kprobes(void)