From patchwork Fri Jul 15 16:47:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jisheng Zhang X-Patchwork-Id: 12919476 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6A4F7C433EF for ; Fri, 15 Jul 2022 16:57:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=pj1TmWV/fEKe+Oc5DQ9yCwf9cVgM4HQpjDudJIiIYWs=; b=b+be+ein56Wi+P dBJE/EycrkZGi/u9hkWiUuvFRtkQoDPO5OduEjqEpfYLrp7Vxu7IsGKUHOdlQ/V7IDBcS+DNP8ljW CBmUzoJEdEidylBH6G2cctSq+y0G35KEsm7qX6XYkSm+GOdxc/d0DMlyh3+RzaNF2e7rx7BC+aOwj NpUFD/R4L+C2g6nJ9VjiZkf3DWCVTxr0Ycu2fUmoeRm/VIbID+ggHWvuXxd5JUQ/wsxjiudkI2Bgm yTpCt3GgEWbpkwHt/yZ6yJDForXigvxjXebZmI2k4a3pLg2Y/GoEL9FLb1lWid7I0KfXIiqs5UnVk phlVDm4qrEEkzq7SWOiw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oCOcg-008yGM-P3; Fri, 15 Jul 2022 16:57:02 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oCOcd-008y9E-Re for linux-riscv@lists.infradead.org; Fri, 15 Jul 2022 16:57:01 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 1DF26B82D21; Fri, 15 Jul 2022 16:56:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D5B95C34115; Fri, 15 Jul 2022 16:56:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657904209; bh=KupXbF85Vdym0NnWb28OdRC9H0j8O4pevvKskb1neSw=; h=From:To:Cc:Subject:Date:From; b=t1zLcEg/BHu+kQUkd82G+gcJkSr2urc9hPw8L6N4osje+sZMWqLOp+syqqaAD/+0a 36U/rKCmdrsRYaTLIou6X7pDNkH9xtC/KYQoCitBHH1LBNVRg1CgKlqr+mBddA04Mc VoHaIpuGVpgJ2XMS6GH+cmrSxEoIle/5632oUS17uZECy4cJ8EbI3UagVOKC9P/9BG V9+brqCBFtma47RzYbp707MNJAJopEhIMalvUWbaZYDg223JAMhcuEni5wzfc0YLih 8VDCIN03XvC3wSvIioQnEz7P7jVojxo1vaZRyxV8o9jgPVMbE87D+XdQx81GRwLVnV l2vKlKh1CPTmg== From: Jisheng Zhang To: Paul Walmsley , Palmer Dabbelt , Albert Ou Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v5] riscv: add irq stack support Date: Sat, 16 Jul 2022 00:47:54 +0800 Message-Id: <20220715164754.2343-1-jszhang@kernel.org> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220715_095700_230717_27F7220F X-CRM114-Status: GOOD ( 20.31 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Currently, IRQs are still handled on the kernel stack of the current task on riscv platforms. If the task has a deep call stack at the time of interrupt, and handling the interrupt also requires a deep stack, it's possible to see stack overflow. Before this patch, the stack_max_size of a v5.17-rc1 kernel running on a lichee RV board gave: ~ # cat /sys/kernel/debug/tracing/stack_max_size 3736 After this patch, ~ # cat /sys/kernel/debug/tracing/stack_max_size 3176 We reduce the max kernel stack usage by 560 bytes! Signed-off-by: Jisheng Zhang --- since v4: - rebase on v5.19-rcN - fix build error when randomized task_struct - use irq stack allocation method in v2, I.E use vmalloc when VMAP_STACK otherwise use __get_free_pages. since v3: - Always use vmalloc to allocate irq stack. Thank Arnd. since v2: - rebase on v5.18-rcN - update commit msg, I.E remove the "it's possible to reduce the THREAD_SIZE to 8KB for RV64 platforms..." since v1: - add __ro_after_init to the irq_stack[] array. arch/riscv/include/asm/thread_info.h | 1 + arch/riscv/kernel/asm-offsets.c | 2 ++ arch/riscv/kernel/entry.S | 39 +++++++++++++++++++++++++--- arch/riscv/kernel/irq.c | 16 ++++++++++++ 4 files changed, 55 insertions(+), 3 deletions(-) diff --git a/arch/riscv/include/asm/thread_info.h b/arch/riscv/include/asm/thread_info.h index 78933ac04995..bc17b7070572 100644 --- a/arch/riscv/include/asm/thread_info.h +++ b/arch/riscv/include/asm/thread_info.h @@ -25,6 +25,7 @@ #endif #define THREAD_SIZE (PAGE_SIZE << THREAD_SIZE_ORDER) +#define IRQ_STACK_SIZE THREAD_SIZE /* * By aligning VMAP'd stacks to 2 * THREAD_SIZE, we can detect overflow by * checking sp & (1 << THREAD_SHIFT), which we can do cheaply in the entry diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c index df9444397908..9e32748af0e8 100644 --- a/arch/riscv/kernel/asm-offsets.c +++ b/arch/riscv/kernel/asm-offsets.c @@ -37,6 +37,8 @@ void asm_offsets(void) OFFSET(TASK_TI_PREEMPT_COUNT, task_struct, thread_info.preempt_count); OFFSET(TASK_TI_KERNEL_SP, task_struct, thread_info.kernel_sp); OFFSET(TASK_TI_USER_SP, task_struct, thread_info.user_sp); + OFFSET(TASK_TI_CPU, task_struct, thread_info.cpu); + OFFSET(TASK_STACK, task_struct, stack); OFFSET(TASK_THREAD_F0, task_struct, thread.fstate.f[0]); OFFSET(TASK_THREAD_F1, task_struct, thread.fstate.f[1]); diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S index 2e5b88ca11ce..2f2a68b3938a 100644 --- a/arch/riscv/kernel/entry.S +++ b/arch/riscv/kernel/entry.S @@ -126,12 +126,45 @@ skip_context_tracking: */ bge s4, zero, 1f - la ra, ret_from_exception + /* preserve the sp */ + move s0, sp - /* Handle interrupts */ move a0, sp /* pt_regs */ + + /* + * Compare sp with the base of the task stack. + * If the top ~(THREAD_SIZE - 1) bits match, we are on a task stack, + * and should switch to the irq stack. + */ +.if (TASK_STACK > 2047) + li t0, TASK_STACK + add t0, t0, tp + REG_L t0, 0(tp) +.else + REG_L t0, TASK_STACK(tp) +.endif + xor t0, t0, s0 + li t1, ~(THREAD_SIZE - 1) + and t0, t0, t1 + bnez t0, 2f + + la t1, irq_stack + REG_L t2, TASK_TI_CPU(tp) + slli t2, t2, RISCV_LGPTR + add t1, t1, t2 + REG_L t2, 0(t1) + li t1, IRQ_STACK_SIZE + /* switch to the irq stack */ + add sp, t2, t1 + +2: + /* Handle interrupts */ la a1, generic_handle_arch_irq - jr a1 + jalr a1 + + /* Restore sp */ + move sp, s0 + j ret_from_exception 1: /* * Exceptions run with interrupts enabled or disabled depending on the diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c index 7207fa08d78f..f20cbfd42e82 100644 --- a/arch/riscv/kernel/irq.c +++ b/arch/riscv/kernel/irq.c @@ -10,6 +10,8 @@ #include #include +void *irq_stack[NR_CPUS] __ro_after_init; + int arch_show_interrupts(struct seq_file *p, int prec) { show_ipi_stats(p, prec); @@ -18,7 +20,21 @@ int arch_show_interrupts(struct seq_file *p, int prec) void __init init_IRQ(void) { + int cpu; + irqchip_init(); if (!handle_arch_irq) panic("No interrupt controller found."); + + for_each_possible_cpu(cpu) { +#ifdef CONFIG_VMAP_STACK + void *s = __vmalloc_node(IRQ_STACK_SIZE, THREAD_ALIGN, + THREADINFO_GFP, cpu_to_node(cpu), + __builtin_return_address(0)); +#else + void *s = (void *)__get_free_pages(GFP_KERNEL, get_order(IRQ_STACK_SIZE)); +#endif + + irq_stack[cpu] = s; + } }