From patchwork Wed May 26 17:49:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 12282577 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76A6AC47088 for ; Wed, 26 May 2021 18:51:20 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3E4D6613BE for ; Wed, 26 May 2021 18:51:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3E4D6613BE Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Mime-Version: Message-Id:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=t41Xz1A4SurX/oPGI3nSTwwJ7NvBcPkTUcyx/clq3AE=; b=gbg hOQVyGpllAlIGOjJ8B6PTD+FUw5mkR9mR3+uO2UyL0tRXlvrmTDECpBfzvA2t5hriuazwQatqztUX zSk8SbQXyuPmhNlZOLnMkdmiTUJbK9yPFa2IiMQSpgoKNELcZyjoUueb1kA+y8KYyVjAQHHgjRDsn cRV/dcColTq/lzSnm9VDn7Q0hXHBsg+vCru9ywtGZcNJ8gfW0Tx8EK4dzTIalJL4mulzUtQBX5VQa 3zfUWKbXFHq+bIKuWwxwwnHLpgVTpcXufxJQZ5UTLo+QZ9T5WLD2PWlU1hQWi41CyYL36dagRx840 Z1cMwvyvdwaJYAnGUHIIP0+FPNV4INw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1llyXY-00Gf0P-98; Wed, 26 May 2021 18:46:05 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1llxf5-00GFty-Vi for linux-arm-kernel@lists.infradead.org; Wed, 26 May 2021 17:49:46 +0000 Received: by mail-yb1-xb49.google.com with SMTP id n129-20020a2527870000b02904ed02e1aab5so2510784ybn.21 for ; Wed, 26 May 2021 10:49:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to:cc; bh=qZi7cmueV7wBE3Gk2EJDG+0VBz+GpDJTgpakxgVpTuE=; b=E+Fa+Yuve4RMBtBT4V8H5HtQ1zyqYmDwsQYS4+hLDoxrYKb1Ufz95jdH2zPV3aSqJJ ofYBcZLslakpGHwfgddv0whO1Ekqdlve4r2d5EOkqtwBHQB4kUg+Ia4CXuV4NWeXfj75 o+rRuXgmTTD4tZHNlJn+sdJUr9KsXZvpaXw669ZW2XhbrB7BvBNv1rx9EV5b/OOs9Lwo +dUcmUtvurWcf2I76GK+iR29jIkbfzZsKRH0bBvsQ78sKhEYZlpUQtHI33fw1gqqiBK9 892N99ckhsmonJovd8BuMnKfW7qqUEEEayJb91lvdaQTfD++WuLzHYDnh+vJeIuTChOC tShA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=qZi7cmueV7wBE3Gk2EJDG+0VBz+GpDJTgpakxgVpTuE=; b=tjNjTNpdCzuM84YEtwSpgAkp+5BCKeGiaHv6Bh4J3OoNq29aSsU3po5ZtExz2r8uw1 OCEenNkzNtggpufhMq0+Z1i3cmmRjA+d9ewwMXWiDJ7rz4lKCMz0j3JQpxaXofQu6lPp lgcCYGMWNJU1uS3apeJPcmYvC1kaudVT+H96UdXrgDoZNvnOIoaNfmautMqK5sTwnu8l jkR8tGIygcYBIWKaVOzv51WJH80nN6FIEGZNXRMCreh7sYNplECmeZa/fIhGxMwKGkF+ PJ3XEI7Qp9/BaR8YZAYdN8cP0gUX5tIAWcMuGOBq+HJHkaHbVtNS5OOg3DycPUBJTo00 A69A== X-Gm-Message-State: AOAM531PfoIlKpIrsGOTXpB4qmP2ch4Kda2LxpAgiwpnzsy/7kMYW2Aw U8raOdFS60o5EZQskUL1ScHj/io= X-Google-Smtp-Source: ABdhPJz5rOKQkrJN9AcuBTOcCHpYIPoIazhdfqkfHp97XnC54o89ydBa0fF8Papo2yMft1vZoAs2iRc= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:a9df:d12:c875:45b7]) (user=pcc job=sendgmr) by 2002:a25:bb08:: with SMTP id z8mr52930638ybg.103.1622051381486; Wed, 26 May 2021 10:49:41 -0700 (PDT) Date: Wed, 26 May 2021 10:49:25 -0700 Message-Id: <20210526174927.2477847-1-pcc@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v6 1/3] arm64: Change the on_*stack functions to take a size argument From: Peter Collingbourne To: Mark Brown , Mark Rutland , Will Deacon , Catalin Marinas , Andrey Konovalov , Evgenii Stepanov , Ard Biesheuvel Cc: Peter Collingbourne , Linux ARM X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210526_104944_081240_F8226562 X-CRM114-Status: GOOD ( 17.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org unwind_frame() was previously implicitly checking that the frame record is in bounds of the stack by enforcing that FP is both aligned to 16 and in bounds of the stack. Once the FP alignment requirement is relaxed to 8 this will not be sufficient because it does not account for the case where FP points to 8 bytes before the end of the stack. Make the check explicit by changing the on_*stack functions to take a size argument and adjusting the callers to pass the appropriate sizes. Signed-off-by: Peter Collingbourne Link: https://linux-review.googlesource.com/id/Ib7a3eb3eea41b0687ffaba045ceb2012d077d8b4 Reviewed-by: Mark Rutland Tested-by: Mark Rutland --- v6: - rebase onto for-next/stacktrace arch/arm64/include/asm/processor.h | 12 +++++------ arch/arm64/include/asm/sdei.h | 7 ++++--- arch/arm64/include/asm/stacktrace.h | 32 ++++++++++++++--------------- arch/arm64/kernel/ptrace.c | 2 +- arch/arm64/kernel/sdei.c | 16 ++++++++------- arch/arm64/kernel/stacktrace.c | 2 +- 6 files changed, 37 insertions(+), 34 deletions(-) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 9df3feeee890..7a094aafec20 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -329,13 +329,13 @@ long get_tagged_addr_ctrl(struct task_struct *task); * of header definitions for the use of task_stack_page. */ -#define current_top_of_stack() \ -({ \ - struct stack_info _info; \ - BUG_ON(!on_accessible_stack(current, current_stack_pointer, &_info)); \ - _info.high; \ +#define current_top_of_stack() \ +({ \ + struct stack_info _info; \ + BUG_ON(!on_accessible_stack(current, current_stack_pointer, 1, &_info)); \ + _info.high; \ }) -#define on_thread_stack() (on_task_stack(current, current_stack_pointer, NULL)) +#define on_thread_stack() (on_task_stack(current, current_stack_pointer, 1, NULL)) #endif /* __ASSEMBLY__ */ #endif /* __ASM_PROCESSOR_H */ diff --git a/arch/arm64/include/asm/sdei.h b/arch/arm64/include/asm/sdei.h index 63e0b92a5fbb..8bc30a5c4569 100644 --- a/arch/arm64/include/asm/sdei.h +++ b/arch/arm64/include/asm/sdei.h @@ -42,8 +42,9 @@ unsigned long sdei_arch_get_entry_point(int conduit); struct stack_info; -bool _on_sdei_stack(unsigned long sp, struct stack_info *info); -static inline bool on_sdei_stack(unsigned long sp, +bool _on_sdei_stack(unsigned long sp, unsigned long size, + struct stack_info *info); +static inline bool on_sdei_stack(unsigned long sp, unsigned long size, struct stack_info *info) { if (!IS_ENABLED(CONFIG_VMAP_STACK)) @@ -51,7 +52,7 @@ static inline bool on_sdei_stack(unsigned long sp, if (!IS_ENABLED(CONFIG_ARM_SDE_INTERFACE)) return false; if (in_nmi()) - return _on_sdei_stack(sp, info); + return _on_sdei_stack(sp, size, info); return false; } diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h index 4b33ca620679..1801399204d7 100644 --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -69,14 +69,14 @@ extern void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk, DECLARE_PER_CPU(unsigned long *, irq_stack_ptr); -static inline bool on_stack(unsigned long sp, unsigned long low, - unsigned long high, enum stack_type type, - struct stack_info *info) +static inline bool on_stack(unsigned long sp, unsigned long size, + unsigned long low, unsigned long high, + enum stack_type type, struct stack_info *info) { if (!low) return false; - if (sp < low || sp >= high) + if (sp < low || sp + size < sp || sp + size > high) return false; if (info) { @@ -87,38 +87,38 @@ static inline bool on_stack(unsigned long sp, unsigned long low, return true; } -static inline bool on_irq_stack(unsigned long sp, +static inline bool on_irq_stack(unsigned long sp, unsigned long size, struct stack_info *info) { unsigned long low = (unsigned long)raw_cpu_read(irq_stack_ptr); unsigned long high = low + IRQ_STACK_SIZE; - return on_stack(sp, low, high, STACK_TYPE_IRQ, info); + return on_stack(sp, size, low, high, STACK_TYPE_IRQ, info); } static inline bool on_task_stack(const struct task_struct *tsk, - unsigned long sp, + unsigned long sp, unsigned long size, struct stack_info *info) { unsigned long low = (unsigned long)task_stack_page(tsk); unsigned long high = low + THREAD_SIZE; - return on_stack(sp, low, high, STACK_TYPE_TASK, info); + return on_stack(sp, size, low, high, STACK_TYPE_TASK, info); } #ifdef CONFIG_VMAP_STACK DECLARE_PER_CPU(unsigned long [OVERFLOW_STACK_SIZE/sizeof(long)], overflow_stack); -static inline bool on_overflow_stack(unsigned long sp, +static inline bool on_overflow_stack(unsigned long sp, unsigned long size, struct stack_info *info) { unsigned long low = (unsigned long)raw_cpu_ptr(overflow_stack); unsigned long high = low + OVERFLOW_STACK_SIZE; - return on_stack(sp, low, high, STACK_TYPE_OVERFLOW, info); + return on_stack(sp, size, low, high, STACK_TYPE_OVERFLOW, info); } #else -static inline bool on_overflow_stack(unsigned long sp, +static inline bool on_overflow_stack(unsigned long sp, unsigned long size, struct stack_info *info) { return false; } #endif @@ -128,21 +128,21 @@ static inline bool on_overflow_stack(unsigned long sp, * context. */ static inline bool on_accessible_stack(const struct task_struct *tsk, - unsigned long sp, + unsigned long sp, unsigned long size, struct stack_info *info) { if (info) info->type = STACK_TYPE_UNKNOWN; - if (on_task_stack(tsk, sp, info)) + if (on_task_stack(tsk, sp, size, info)) return true; if (tsk != current || preemptible()) return false; - if (on_irq_stack(sp, info)) + if (on_irq_stack(sp, size, info)) return true; - if (on_overflow_stack(sp, info)) + if (on_overflow_stack(sp, size, info)) return true; - if (on_sdei_stack(sp, info)) + if (on_sdei_stack(sp, size, info)) return true; return false; diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index eb2f73939b7b..499b6b2f9757 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -122,7 +122,7 @@ static bool regs_within_kernel_stack(struct pt_regs *regs, unsigned long addr) { return ((addr & ~(THREAD_SIZE - 1)) == (kernel_stack_pointer(regs) & ~(THREAD_SIZE - 1))) || - on_irq_stack(addr, NULL); + on_irq_stack(addr, sizeof(unsigned long), NULL); } /** diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index 2c7ca449dd51..c524f96f97c4 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -162,31 +162,33 @@ static int init_sdei_scs(void) return err; } -static bool on_sdei_normal_stack(unsigned long sp, struct stack_info *info) +static bool on_sdei_normal_stack(unsigned long sp, unsigned long size, + struct stack_info *info) { unsigned long low = (unsigned long)raw_cpu_read(sdei_stack_normal_ptr); unsigned long high = low + SDEI_STACK_SIZE; - return on_stack(sp, low, high, STACK_TYPE_SDEI_NORMAL, info); + return on_stack(sp, size, low, high, STACK_TYPE_SDEI_NORMAL, info); } -static bool on_sdei_critical_stack(unsigned long sp, struct stack_info *info) +static bool on_sdei_critical_stack(unsigned long sp, unsigned long size, + struct stack_info *info) { unsigned long low = (unsigned long)raw_cpu_read(sdei_stack_critical_ptr); unsigned long high = low + SDEI_STACK_SIZE; - return on_stack(sp, low, high, STACK_TYPE_SDEI_CRITICAL, info); + return on_stack(sp, size, low, high, STACK_TYPE_SDEI_CRITICAL, info); } -bool _on_sdei_stack(unsigned long sp, struct stack_info *info) +bool _on_sdei_stack(unsigned long sp, unsigned long size, struct stack_info *info) { if (!IS_ENABLED(CONFIG_VMAP_STACK)) return false; - if (on_sdei_critical_stack(sp, info)) + if (on_sdei_critical_stack(sp, size, info)) return true; - if (on_sdei_normal_stack(sp, info)) + if (on_sdei_normal_stack(sp, size, info)) return true; return false; diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 36cf05d5eb9e..5c70f247645b 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -78,7 +78,7 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) if (fp & 0xf) return -EINVAL; - if (!on_accessible_stack(tsk, fp, &info)) + if (!on_accessible_stack(tsk, fp, 16, &info)) return -EINVAL; if (test_bit(info.type, frame->stacks_done)) From patchwork Wed May 26 17:49:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 12282585 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E6E5C47088 for ; Wed, 26 May 2021 18:54:40 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D3853613BC for ; Wed, 26 May 2021 18:54:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D3853613BC Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=4SA7OAdjesrA2vXWTsEtMalnCUXHTK809r/jkOni2oM=; b=iaNuXPnPrZ+h76MZiGcpEv/uro 3zuR5tQg8pj69XGhBOxgo+GtJYMtJ4/GAk5HW7/F77ZGHITXT63feUPO/Jexo45qKUh5YR7SEBvmw VCq+su81jEyUYc1U5tuavRkCaK6wrIHB66SSxLnZ6U8AQVWu4+QrQXBrWG8w5obGV4fPUFhTdVnBr USyKYkEvDxpxBQzYtsukEJSGToVt3aMBhb80ntfpXS1HtgfxTR5OjxJDFezqOd84ESKk1gLxq3bFR ElVR8YLs6tWAC55h5zjQcAx63g9UrSYMGvwKksqtYSjsseCdywo0VyPpkyhiDeQ85TLC2n8qXfFJA SDFvf92A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1llycE-00Ghrx-P1; Wed, 26 May 2021 18:50:53 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1llxf8-00GFv4-05 for linux-arm-kernel@lists.infradead.org; Wed, 26 May 2021 17:49:48 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id j7-20020a258b870000b029052360b1e3e2so2550766ybl.8 for ; Wed, 26 May 2021 10:49:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=GqmVVXezyMmHPgCnOKEarqOWfLrnu9GStscZ2c33AM4=; b=h8YeQi0+TXAtIgFS5KMdfkHQZ+8vBKkQDrmUewNfsQjC+qSZPKRYehwyWtHl2cF99e ocYtFUX7yph7M/KafScINOFQpepuWFH/0h/AOh82Y35O76V0EuFt8NEhzsbvJLsFk8Rw CebLxWzfEp8fA3n1lzIFqtXkL75JtDITFbKDgOjBVuY420bv9welt5XaZ31io7yjxDIA Igp/3N8u4LUYjOoZaWC2MnoAr9o27PhOl/gc3uq5UblmFWpgtujPx1ORdQNKagp7qph2 U076BIDHjs2SdcaBVBimuUWDdHBkQtepWns2u3zSkZBoR3wU6XJ1kiGKeCPeN0inxkUV 6nLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=GqmVVXezyMmHPgCnOKEarqOWfLrnu9GStscZ2c33AM4=; b=WpWsCZTxM5bwVMK0bQduHeVMkGdRQBlt8O937acbrtNeQpeli6jGQLexhs/+gnqEKc H6qW98VZUBQ70ML0G1Ke763R9mSc7MGAva2Pugb4i5BNgwyOSBGYJv9ITv9sJ/Y+b+Vn aaH1dDrzxMSIYEXvQ56qE2xM2/1HkYSQn4LDwhTtWWIkGPH+imFvT9SCCx0RYYRT8CJM vBcQGTtancMYqbOCBqtYWW+bZ3SXIXypS7uEar0hugrt57UUDJfV2XIjXE1+rgRmx8P/ rP7OvDQZYCf8OH4tgQn6163qdQIgONbdRae0EFN4JNMDyRE9XeulnAaN6rzBbQXdhNuO EqmQ== X-Gm-Message-State: AOAM533NIPBi49vQiKeMLim/nmyE4QkHWI0tny+ZHJP3mZJaGvU8ZAz0 bRGxm0Gi1iKlFohCOSUoMc/xvq8= X-Google-Smtp-Source: ABdhPJxef3Weo5XHTOZpuQzcBc1xK121ALoMxcYWcL0ObL3u+Fzve7HB8wYJteHXd8HOnW+1nww/cbg= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:a9df:d12:c875:45b7]) (user=pcc job=sendgmr) by 2002:a25:11c6:: with SMTP id 189mr52382886ybr.154.1622051383686; Wed, 26 May 2021 10:49:43 -0700 (PDT) Date: Wed, 26 May 2021 10:49:26 -0700 In-Reply-To: <20210526174927.2477847-1-pcc@google.com> Message-Id: <20210526174927.2477847-2-pcc@google.com> Mime-Version: 1.0 References: <20210526174927.2477847-1-pcc@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v6 2/3] arm64: stacktrace: Relax frame record alignment requirement to 8 bytes From: Peter Collingbourne To: Mark Brown , Mark Rutland , Will Deacon , Catalin Marinas , Andrey Konovalov , Evgenii Stepanov , Ard Biesheuvel Cc: Peter Collingbourne , Linux ARM X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210526_104946_108337_E45E3E9D X-CRM114-Status: GOOD ( 16.13 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The AAPCS places no requirements on the alignment of the frame record. In theory it could be placed anywhere, although it seems sensible to require it to be aligned to 8 bytes. With an upcoming enhancement to tag-based KASAN Clang will begin creating frame records located at an address that is only aligned to 8 bytes. Accommodate such frame records in the stack unwinding code. As pointed out by Mark Rutland, the userspace stack unwinding code has the same problem, so fix it there as well. Signed-off-by: Peter Collingbourne Link: https://linux-review.googlesource.com/id/Ia22c375230e67ca055e9e4bb639383567f7ad268 Acked-by: Andrey Konovalov Reviewed-by: Mark Rutland Tested-by: Mark Rutland --- v6: - rebase onto for-next/stacktrace v5: - fix rebase v4: - rebase to 5.13rc1 v2: - fix it in the userspace unwinding code as well arch/arm64/kernel/perf_callchain.c | 2 +- arch/arm64/kernel/stacktrace.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/perf_callchain.c b/arch/arm64/kernel/perf_callchain.c index 88ff471b0bce..4a72c2727309 100644 --- a/arch/arm64/kernel/perf_callchain.c +++ b/arch/arm64/kernel/perf_callchain.c @@ -116,7 +116,7 @@ void perf_callchain_user(struct perf_callchain_entry_ctx *entry, tail = (struct frame_tail __user *)regs->regs[29]; while (entry->nr < entry->max_stack && - tail && !((unsigned long)tail & 0xf)) + tail && !((unsigned long)tail & 0x7)) tail = user_backtrace(tail, entry); } else { #ifdef CONFIG_COMPAT diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 5c70f247645b..b189de5ca6cb 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -75,7 +75,7 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) if (fp == (unsigned long)task_pt_regs(tsk)->stackframe) return -ENOENT; - if (fp & 0xf) + if (fp & 0x7) return -EINVAL; if (!on_accessible_stack(tsk, fp, 16, &info)) From patchwork Wed May 26 17:49:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 12282595 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF5FBC47088 for ; Wed, 26 May 2021 18:57:21 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 70B8E613C3 for ; Wed, 26 May 2021 18:57:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 70B8E613C3 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=hLXDZ/pQkm0DZexEi1uuhAbOHvPzGAI/XPiFd3gug2Q=; b=re9jkY05ZV8CKGs1k/AcRP8nln 1GLZauKtoJspThtvmcSJaPApunIf32upC4B9ySQaA8An5cxqY1Fe+W2XZp3hKSmedKyq4RT2sAJs1 oP600YIgC+zz81LJFlzmpkcxk9/9W78EfHpTSE8OV9wWeQVDaLyqlPRqZsrPtr1eUpx2AAT6MNzg8 nmd51WcTgg7Pi68chtKzCVdBQFFi3KxSVVATumDQCF7L/LufrVvLuoQYTcrCFICey5Jyy6qD9pAmP NS1rar01KhkDfdXFKyZhDEr8GepPuZSRudlIe5dkD4OTfsxpzqU99MtVeBzdtU/HcMfTajpsDApBA 0o59GSDg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1llyeY-00Gixm-CV; Wed, 26 May 2021 18:53:16 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1llxf9-00GFvo-NP for linux-arm-kernel@lists.infradead.org; Wed, 26 May 2021 17:49:50 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id c9-20020a2580c90000b02904f86395a96dso2499016ybm.19 for ; Wed, 26 May 2021 10:49:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ZAReDArHj00EpnlJwcqJaDoXpaiz37/03m+cdvn/wWA=; b=e3FOyW111m4ypztTr8AkbvyJ0/2eJvrhLzdy6ZNtDkUzBo9tgBGhIDbUmE23nc0P0x 97lXORii7Vfuu36y9PKxGTHfM+boFeTjvPuMQjpdQaw3n/gxcxostv9o7NXAefxxf09h imwRUVxDsHhlyie1gQiFDT+FQwzPK6RtA8WPqa21tDAW5KxWIK3tO2yN8GRxmB31Pv9r RKxaAhqtCX5aIVx42elCyQnNjrtPvWRqPlSW279heDoU2R2tnuuWmhspnrgkc9jFcxSH QZwEh988VL697lsgnwakDXNz71FG/DabSalgkGmuWCAdJuVQcWSgiZ4xS8qrGsqmabew f1PQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ZAReDArHj00EpnlJwcqJaDoXpaiz37/03m+cdvn/wWA=; b=ID4t0sLOUKKHP1IqFs7YvkfnvddV5iHYSJ77fllww+ociY8fV2k+MJyCUkOTLtwJqv MS10/TpZfac5sY7kJ8ml3vLXfAX9PSon2TO7yzfp4J62LOAlViCyqjYEe3Jo+uPJ4Q9f DNw85XoUHziBfYc0oqLtcvmCm8+igFyWkd/MAj5Iazv6nl/ln348T08UcR8gn3ZKCgJP uC8kUHq1tJY6PbnMt8xLWSxrvHapSZKH+3kbgBPXSCI2nwzZyU1K8J3YiH+1Y12sWzFQ MXjvlVgZ9J1NoOfRCRAi4GX2ecKccQFuDEzEPT0q/sOv3Pgtyl+ZiLQ1JYSvqzeBncZ9 md3Q== X-Gm-Message-State: AOAM5320kxNOgavNVmQPkzNRRXlXcOr8gGhLBBmJW38BHVr6yYtJYFnY S4AX9PZ6O4Kah5qcE96Dqar+Z+M= X-Google-Smtp-Source: ABdhPJz2Wn2hvuKQPzJ3BWfCL7BalHipxcV2SzYFp4CmNCCKCzkp7cbTM7mQfpY6PnE16UyD7DxlZh0= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:a9df:d12:c875:45b7]) (user=pcc job=sendgmr) by 2002:a25:16d4:: with SMTP id 203mr29882221ybw.281.1622051385871; Wed, 26 May 2021 10:49:45 -0700 (PDT) Date: Wed, 26 May 2021 10:49:27 -0700 In-Reply-To: <20210526174927.2477847-1-pcc@google.com> Message-Id: <20210526174927.2477847-3-pcc@google.com> Mime-Version: 1.0 References: <20210526174927.2477847-1-pcc@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v6 3/3] kasan: arm64: support specialized outlined tag mismatch checks From: Peter Collingbourne To: Mark Brown , Mark Rutland , Will Deacon , Catalin Marinas , Andrey Konovalov , Evgenii Stepanov , Ard Biesheuvel Cc: Peter Collingbourne , Linux ARM X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210526_104947_819115_383F130D X-CRM114-Status: GOOD ( 32.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org By using outlined checks we can achieve a significant code size improvement by moving the tag-based ASAN checks into separate functions. Unlike the existing CONFIG_KASAN_OUTLINE mode these functions have a custom calling convention that preserves most registers and is specialized to the register containing the address and the type of access, and as a result we can eliminate the code size and performance overhead of a standard calling convention such as AAPCS for these functions. This change depends on a separate series of changes to Clang [1] to support outlined checks in the kernel, although the change works fine without them (we just don't get outlined checks). This is because the flag -mllvm -hwasan-inline-all-checks=0 has no effect until the Clang changes land. The flag was introduced in the Clang 9.0 timeframe as part of the support for outlined checks in userspace and because our minimum Clang version is 10.0 we can pass it unconditionally. Outlined checks require a new runtime function with a custom calling convention. Add this function to arch/arm64/lib. I measured the code size of defconfig + tag-based KASAN, as well as boot time (i.e. time to init launch) on a DragonBoard 845c with an Android arm64 GKI kernel. The results are below: code size boot time CONFIG_KASAN_INLINE=y before 92824064 6.18s CONFIG_KASAN_INLINE=y after 38822400 6.65s CONFIG_KASAN_OUTLINE=y 39215616 11.48s We can see straight away that specialized outlined checks beat the existing CONFIG_KASAN_OUTLINE=y on both code size and boot time for tag-based ASAN. As for the comparison between CONFIG_KASAN_INLINE=y before and after we saw similar performance numbers in userspace [2] and decided that since the performance overhead is minimal compared to the overhead of tag-based ASAN itself as well as compared to the code size improvements we would just replace the inlined checks with the specialized outlined checks without the option to select between them, and that is what I have implemented in this patch. Signed-off-by: Peter Collingbourne Acked-by: Andrey Konovalov Reviewed-by: Mark Rutland Tested-by: Mark Rutland Link: https://linux-review.googlesource.com/id/I1a30036c70ab3c3ee78d75ed9b87ef7cdc3fdb76 Link: [1] https://reviews.llvm.org/D90426 Link: [2] https://reviews.llvm.org/D56954 --- v4: - rebase to 5.13rc1 v3: - adopt Mark Rutland's suggested changes - move frame record alignment patches behind this one v2: - use calculations in the stack spills and restores - improve the comment at the top of the function - add a BTI instruction arch/arm64/include/asm/asm-prototypes.h | 6 ++ arch/arm64/include/asm/module.lds.h | 17 +++++- arch/arm64/lib/Makefile | 2 + arch/arm64/lib/kasan_sw_tags.S | 76 +++++++++++++++++++++++++ mm/kasan/sw_tags.c | 7 +++ scripts/Makefile.kasan | 1 + 6 files changed, 107 insertions(+), 2 deletions(-) create mode 100644 arch/arm64/lib/kasan_sw_tags.S diff --git a/arch/arm64/include/asm/asm-prototypes.h b/arch/arm64/include/asm/asm-prototypes.h index 1c9a3a0c5fa5..ec1d9655f885 100644 --- a/arch/arm64/include/asm/asm-prototypes.h +++ b/arch/arm64/include/asm/asm-prototypes.h @@ -23,4 +23,10 @@ long long __ashlti3(long long a, int b); long long __ashrti3(long long a, int b); long long __lshrti3(long long a, int b); +/* + * This function uses a custom calling convention and cannot be called from C so + * this prototype is not entirely accurate. + */ +void __hwasan_tag_mismatch(unsigned long addr, unsigned long access_info); + #endif /* __ASM_PROTOTYPES_H */ diff --git a/arch/arm64/include/asm/module.lds.h b/arch/arm64/include/asm/module.lds.h index 810045628c66..a11ccadd47d2 100644 --- a/arch/arm64/include/asm/module.lds.h +++ b/arch/arm64/include/asm/module.lds.h @@ -1,7 +1,20 @@ -#ifdef CONFIG_ARM64_MODULE_PLTS SECTIONS { +#ifdef CONFIG_ARM64_MODULE_PLTS .plt 0 (NOLOAD) : { BYTE(0) } .init.plt 0 (NOLOAD) : { BYTE(0) } .text.ftrace_trampoline 0 (NOLOAD) : { BYTE(0) } -} #endif + +#ifdef CONFIG_KASAN_SW_TAGS + /* + * Outlined checks go into comdat-deduplicated sections named .text.hot. + * Because they are in comdats they are not combined by the linker and + * we otherwise end up with multiple sections with the same .text.hot + * name in the .ko file. The kernel module loader warns if it sees + * multiple sections with the same name so we use this sections + * directive to force them into a single section and silence the + * warning. + */ + .text.hot : { *(.text.hot) } +#endif +} diff --git a/arch/arm64/lib/Makefile b/arch/arm64/lib/Makefile index d31e1169d9b8..8e60d76a1b47 100644 --- a/arch/arm64/lib/Makefile +++ b/arch/arm64/lib/Makefile @@ -18,3 +18,5 @@ obj-$(CONFIG_CRC32) += crc32.o obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o obj-$(CONFIG_ARM64_MTE) += mte.o + +obj-$(CONFIG_KASAN_SW_TAGS) += kasan_sw_tags.o diff --git a/arch/arm64/lib/kasan_sw_tags.S b/arch/arm64/lib/kasan_sw_tags.S new file mode 100644 index 000000000000..5b04464c045e --- /dev/null +++ b/arch/arm64/lib/kasan_sw_tags.S @@ -0,0 +1,76 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2020 Google LLC + */ + +#include +#include + +/* + * Report a tag mismatch detected by tag-based KASAN. + * + * A compiler-generated thunk calls this with a non-AAPCS calling + * convention. Upon entry to this function, registers are as follows: + * + * x0: fault address (see below for restore) + * x1: fault description (see below for restore) + * x2 to x15: callee-saved + * x16 to x17: safe to clobber + * x18 to x30: callee-saved + * sp: pre-decremented by 256 bytes (see below for restore) + * + * The caller has decremented the SP by 256 bytes, and created a + * structure on the stack as follows: + * + * sp + 0..15: x0 and x1 to be restored + * sp + 16..231: free for use + * sp + 232..247: x29 and x30 (same as in GPRs) + * sp + 248..255: free for use + * + * Note that this is not a struct pt_regs. + * + * To call a regular AAPCS function we must save x2 to x15 (which we can + * store in the gaps), and create a frame record (for which we can use + * x29 and x30 spilled by the caller as those match the GPRs). + * + * The caller expects x0 and x1 to be restored from the structure, and + * for the structure to be removed from the stack (i.e. the SP must be + * incremented by 256 prior to return). + */ +SYM_CODE_START(__hwasan_tag_mismatch) +#ifdef BTI_C + BTI_C +#endif + add x29, sp, #232 + stp x2, x3, [sp, #8 * 2] + stp x4, x5, [sp, #8 * 4] + stp x6, x7, [sp, #8 * 6] + stp x8, x9, [sp, #8 * 8] + stp x10, x11, [sp, #8 * 10] + stp x12, x13, [sp, #8 * 12] + stp x14, x15, [sp, #8 * 14] +#ifndef CONFIG_SHADOW_CALL_STACK + str x18, [sp, #8 * 18] +#endif + + mov x2, x30 + bl kasan_tag_mismatch + + ldp x0, x1, [sp] + ldp x2, x3, [sp, #8 * 2] + ldp x4, x5, [sp, #8 * 4] + ldp x6, x7, [sp, #8 * 6] + ldp x8, x9, [sp, #8 * 8] + ldp x10, x11, [sp, #8 * 10] + ldp x12, x13, [sp, #8 * 12] + ldp x14, x15, [sp, #8 * 14] +#ifndef CONFIG_SHADOW_CALL_STACK + ldr x18, [sp, #8 * 18] +#endif + ldp x29, x30, [sp, #8 * 29] + + /* remove the structure from the stack */ + add sp, sp, #256 + ret +SYM_CODE_END(__hwasan_tag_mismatch) +EXPORT_SYMBOL(__hwasan_tag_mismatch) diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c index 9df8e7f69e87..9362938abbfa 100644 --- a/mm/kasan/sw_tags.c +++ b/mm/kasan/sw_tags.c @@ -207,3 +207,10 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, return &alloc_meta->free_track[i]; } + +void kasan_tag_mismatch(unsigned long addr, unsigned long access_info, + unsigned long ret_ip) +{ + kasan_report(addr, 1 << (access_info & 0xf), access_info & 0x10, + ret_ip); +} diff --git a/scripts/Makefile.kasan b/scripts/Makefile.kasan index 3d791908ed36..801c415bac59 100644 --- a/scripts/Makefile.kasan +++ b/scripts/Makefile.kasan @@ -50,6 +50,7 @@ endif CFLAGS_KASAN := -fsanitize=kernel-hwaddress \ $(call cc-param,hwasan-instrument-stack=$(stack_enable)) \ $(call cc-param,hwasan-use-short-granules=0) \ + $(call cc-param,hwasan-inline-all-checks=0) \ $(instrumentation_flags) endif # CONFIG_KASAN_SW_TAGS