From patchwork Thu Jan 25 06:21:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 13530086 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1FED1C47DDF for ; Thu, 25 Jan 2024 06:30:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7E8358D0007; Thu, 25 Jan 2024 01:30:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7708D8D0001; Thu, 25 Jan 2024 01:30:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 54EA68D0007; Thu, 25 Jan 2024 01:30:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 3CBCC8D0001 for ; Thu, 25 Jan 2024 01:30:00 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 1AD4D1407FF for ; Thu, 25 Jan 2024 06:30:00 +0000 (UTC) X-FDA: 81716858160.08.67F5DC3 Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by imf11.hostedemail.com (Postfix) with ESMTP id 4A08440007 for ; Thu, 25 Jan 2024 06:29:57 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=lz4Q7XvG; dmarc=none; spf=pass (imf11.hostedemail.com: domain of debug@rivosinc.com designates 209.85.215.175 as permitted sender) smtp.mailfrom=debug@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706164197; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ElZ8v003TL24bpwLuLWIXAZR4LH6JaW8OFYo6Q46gr4=; b=hM0EiHrZIvwm68IhhCz4S46snzQ7BM9mssCFHvyQtaCFBNLT19HsW2S+/o/mhNTydc6nuf Xnwfl0/PVcY68jMpm4Ww89A/RoMLhVoABvtMDfsBmjSzM7INBUt/1aQwS1AUJRTEG8ouBF dT1AsMv0fuCD+APcoV52XFi+GXvNs0k= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=lz4Q7XvG; dmarc=none; spf=pass (imf11.hostedemail.com: domain of debug@rivosinc.com designates 209.85.215.175 as permitted sender) smtp.mailfrom=debug@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706164197; a=rsa-sha256; cv=none; b=Gik+PvGSp9TfO1yyt6p+Z/+CyPNe5V/MsXsQhMmHs8kz16Sc55PQQSoyG90lRGKK3wt7eB 1029nkzT4BLLMsfTuQiVMnEPDUOSbviToz6ZdTWORXFNx56+zeiJesG4hNMIaMGTnOgnU6 AjnRdOUw3S0xB1G/7CK7o4nwwuXjtwU= Received: by mail-pg1-f175.google.com with SMTP id 41be03b00d2f7-5cf2d73a183so294458a12.1 for ; Wed, 24 Jan 2024 22:29:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1706164196; x=1706768996; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ElZ8v003TL24bpwLuLWIXAZR4LH6JaW8OFYo6Q46gr4=; b=lz4Q7XvGSolcYbUms225MGmMJCmMDiKfsHTP2+gBkNi9FecWQWLVJ6hf69Us8mrDxn XUfHlQak/eznhONTPJODwnocXwc3FoQgVJtdwqxvCG7Nu/Y3uTwYy6AIC3uezKZU5jZI bp/aopqpyo2k58tB5VMH6ClRgUo7vE7dYmjfQhLetU52gpVUv/dvLUdDXwre8hxrE1X3 KWhi8meBBvmKcSl38eRyalSNDSHTFG4yZqN2JZdnRVSzLi6mQDRe9YbRhY6Id+srSpcc K0w2fyHtWM26dxozea1/exWx+hOPpW5rhs7v1IWQBo1kD9jU8vj96WQixBcnrKA9XPEe SFNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706164196; x=1706768996; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ElZ8v003TL24bpwLuLWIXAZR4LH6JaW8OFYo6Q46gr4=; b=pMPhiTsqi19geoLrwndgXBbrxuPSXwiOEMdtt8dizrNWamERa+yU031kJUj/tBZPie DCQ+pc7VWAVfaxiRpc7kawg82ZyNjpTPbpw2Lv1ZZtdKVJv4AfCNiL9yRWmkZ1qsc2Cp WL3LtMz1g+qESGdDDEjtsqHTDdgg9udgMVAP0FO2DB6Jwn9LF8GSs/e708jeXKzx5t91 2SxqGCk+y0wH7T6kLASTH8CO1n5vQnwuY4rwDk+et1dmPCL55uFFQZfLkgux8UB1sUw1 R9vyJgOinUGxT0pVELJx7CmUIwRjsDtRm8tZ9e8d9w2yyjOtHPkO9GET+lC1kH7RGNRi ewXA== X-Gm-Message-State: AOJu0Yw3jM9EV8zcdolUW83t+zGjinI0SEst519lkBsHC0gDFK7N/EwO nEooGkNpMDjFzt89dKC6oOdoHYfxcGhu1/j3keRMLfTNV7JUvGS0Wh9yZk8sp0Y= X-Google-Smtp-Source: AGHT+IFX1+d0cQXBDGIq+89psVK7k1OJN/xRk32F1GoAKNNphyd9nJ4SL45lMaDh/Km+dlBcYHzICw== X-Received: by 2002:a05:6a21:3513:b0:19b:7d67:5b35 with SMTP id zc19-20020a056a21351300b0019b7d675b35mr808275pzb.36.1706164196007; Wed, 24 Jan 2024 22:29:56 -0800 (PST) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id t19-20020a056a00139300b006dd870b51b8sm3201139pfg.126.2024.01.24.22.29.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 24 Jan 2024 22:29:55 -0800 (PST) From: debug@rivosinc.com To: rick.p.edgecombe@intel.com, broonie@kernel.org, Szabolcs.Nagy@arm.com, kito.cheng@sifive.com, keescook@chromium.org, ajones@ventanamicro.com, paul.walmsley@sifive.com, palmer@dabbelt.com, conor.dooley@microchip.com, cleger@rivosinc.com, atishp@atishpatra.org, alex@ghiti.fr, bjorn@rivosinc.com, alexghiti@rivosinc.com Cc: corbet@lwn.net, aou@eecs.berkeley.edu, oleg@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, ebiederm@xmission.com, shuah@kernel.org, brauner@kernel.org, debug@rivosinc.com, guoren@kernel.org, samitolvanen@google.com, evan@rivosinc.com, xiao.w.wang@intel.com, apatel@ventanamicro.com, mchitale@ventanamicro.com, waylingii@gmail.com, greentime.hu@sifive.com, heiko@sntech.de, jszhang@kernel.org, shikemeng@huaweicloud.com, david@redhat.com, charlie@rivosinc.com, panqinglin2020@iscas.ac.cn, willy@infradead.org, vincent.chen@sifive.com, andy.chiu@sifive.com, gerg@kernel.org, jeeheng.sia@starfivetech.com, mason.huo@starfivetech.com, ancientmodern4@gmail.com, mathis.salmen@matsal.de, cuiyunhui@bytedance.com, bhe@redhat.com, chenjiahao16@huawei.com, ruscur@russell.cc, bgray@linux.ibm.com, alx@kernel.org, baruch@tkos.co.il, zhangqing@loongson.cn, catalin.marinas@arm.com, revest@chromium.org, josh@joshtriplett.org, joey.gouly@arm.com, shr@devkernel.io, omosnace@redhat.com, ojeda@kernel.org, jhubbard@nvidia.com, linux-doc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-kselftest@vger.kernel.org Subject: [RFC PATCH v1 16/28] riscv/shstk: If needed allocate a new shadow stack on clone Date: Wed, 24 Jan 2024 22:21:41 -0800 Message-ID: <20240125062739.1339782-17-debug@rivosinc.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240125062739.1339782-1-debug@rivosinc.com> References: <20240125062739.1339782-1-debug@rivosinc.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4A08440007 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: mkesiuounidkakuky6rhwofsaagbjapn X-HE-Tag: 1706164197-491857 X-HE-Meta: U2FsdGVkX18OhqF/c0wT6P8CQIv8DHqJWBy8dNTQznamDny9EocBa8Q6kW4V2/xMm5JmL6d30J6p4ENaahrb5p1KhQWCBs2MJslriPnb3ER0zv2rcluHzTxzRPqCLQdjuEzGiJHd96xcjI8KFywUyepKOWEeabosKUjTFfdf2wCeLCDPc+jvrv+VWKGY4sAKz8q+oJb8EVVl30BrqENsijpQ380Y7hQBPi8nJRqxj4ZIxH7zLdB/xBUlIDIinBp5h89fIUESgZ31z7MZl2j1K3T0lSzL53lA2IDmBm/YdlzpD36yT8JJ53noPB7XhgJJb09JpiND45nFo1CCZ/BkdZRwd7SmYao43svQnpZV2Jc1U2stOamO03Baa3cQdTCDR5AEfBUm37F3TkTCbPgmwLzBBGzCOZ+RoljA3mgBKzuUQLV2QDX1zmNfEqdhG5YL/01JVxEB2KUHWcwh+wS5BnoRri9C1H4X+7KAHVADI49FfsrGsMihZilWi6u3zEqkXhpan5aygaG8qma0SWPDgUCjH1nj5hjtsMPlxG2G8j+AUjRhK3TaxDreNZnCr9E2CsxRvEpSltTY9aawBO9GwL91QFF+VmgYQY++kLg9+6LWsDL6dCCoim9IgFTkFI7JnrG/tT66yVaUSNzSmR8p5uYVHEzZtiKN0GnnS2XmH9YJIVaRZoSQByLlOHDIu9vyjDXucCJpSjpONYxZyKr5OOckAEfjM6ULc3rKt66lo82H/FTQZfheP26OABLbYi/lSpy5aJ1Ff+L/nUJKNBwa3A5IG1x6sMklCpKAlnr71A1YusUg/o2UBnUpA38yINnd78xSwiCtO5WN8sUykvcv+XnO2eb2q3n6Vwk1FGDwtLYz8Gq9S5sN2eUF5x3/sn4JwlY/IhG25HDVMsGXC6VdJGAVpvwHDcghhd4+OLSxz7SNEPzVVeYFRBACUWNBA+ofH7Gqpbi6Zv5AkYbBjRO +MXrFZlk btGqV16llNJSttuzJmF4gSyoP6CNIq0TFQ6FwsZQEO3x7SO7kFPskS5ZZUiIvaajE30/l2zQxLWdihY7PuR1hSk4bA6zlKUmfqk4s83qKeH9UbU2AmVMvmkfIqsi3ywqDrxwc6+a7VFQfn8T67wf7OBLTx+vvni2i+DfyJ14wn4rYk8UsRJ/jCbxN8AYxrizmW76udr8x8or6kkrBO2mntdHZ2GxXeAJbzLFbIaWcS1gY4c5p+pmoC3NvfJGuPWuBXR41D74toX5qgnwTIjNfqs7T4rFrksfG7VHOg7347kTczY6pfWIKIY8B8FbFU1eUKDEpck3E5GuB1dLkJV1tto7RgZODk1Fo+tdZD2aymf7TXF3Bj2ppGOzrZflEwRkqPwF9VumKJkgMTNanTh8xEzy6ejafczZbdDmjRo/iG5XerM3He15Ixs8z7GFLFTnaeKhm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Deepak Gupta Userspace specifies VM_CLONE to share address space and spawn new thread. `clone` allow userspace to specify a new stack for new thread. However there is no way to specify new shadow stack base address without changing API. This patch allocates a new shadow stack whenever VM_CLONE is given. In case of VM_FORK, parent is suspended until child finishes and thus can child use parent shadow stack. In case of !VM_CLONE, COW kicks in because entire address space is copied from parent to child. `clone3` is extensible and can provide mechanisms using which shadow stack as an input parameter can be provided. This is not settled yet and being extensively discussed on mailing list. Once that's settled, this commit will adapt to that. Signed-off-by: Deepak Gupta --- arch/riscv/include/asm/usercfi.h | 39 ++++++++++ arch/riscv/kernel/process.c | 10 +++ arch/riscv/kernel/usercfi.c | 121 +++++++++++++++++++++++++++++++ 3 files changed, 170 insertions(+) diff --git a/arch/riscv/include/asm/usercfi.h b/arch/riscv/include/asm/usercfi.h index 080d7077d12c..eb9a0905e72b 100644 --- a/arch/riscv/include/asm/usercfi.h +++ b/arch/riscv/include/asm/usercfi.h @@ -8,6 +8,9 @@ #ifndef __ASSEMBLY__ #include +struct task_struct; +struct kernel_clone_args; + #ifdef CONFIG_RISCV_USER_CFI struct cfi_status { unsigned long ubcfi_en : 1; /* Enable for backward cfi. */ @@ -17,6 +20,42 @@ struct cfi_status { unsigned long shdw_stk_size; /* size of shadow stack */ }; +unsigned long shstk_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args); +void shstk_release(struct task_struct *tsk); +void set_shstk_base(struct task_struct *task, unsigned long shstk_addr, unsigned long size); +void set_active_shstk(struct task_struct *task, unsigned long shstk_addr); +bool is_shstk_enabled(struct task_struct *task); + +#else + +static inline unsigned long shstk_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args) +{ + return 0; +} + +static inline void shstk_release(struct task_struct *tsk) +{ + +} + +static inline void set_shstk_base(struct task_struct *task, unsigned long shstk_addr, + unsigned long size) +{ + +} + +static inline void set_active_shstk(struct task_struct *task, unsigned long shstk_addr) +{ + +} + +static inline bool is_shstk_enabled(struct task_struct *task) +{ + return false; +} + #endif /* CONFIG_RISCV_USER_CFI */ #endif /* __ASSEMBLY__ */ diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c index c249cf3d8083..a2b2a686a545 100644 --- a/arch/riscv/kernel/process.c +++ b/arch/riscv/kernel/process.c @@ -26,6 +26,7 @@ #include #include #include +#include register unsigned long gp_in_global __asm__("gp"); @@ -194,6 +195,7 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) void exit_thread(struct task_struct *tsk) { + shstk_release(tsk); return; } @@ -202,6 +204,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) unsigned long clone_flags = args->flags; unsigned long usp = args->stack; unsigned long tls = args->tls; + unsigned long ssp = 0; struct pt_regs *childregs = task_pt_regs(p); memset(&p->thread.s, 0, sizeof(p->thread.s)); @@ -217,11 +220,18 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) p->thread.s[0] = (unsigned long)args->fn; p->thread.s[1] = (unsigned long)args->fn_arg; } else { + /* allocate new shadow stack if needed. In case of CLONE_VM we have to */ + ssp = shstk_alloc_thread_stack(p, args); + if (IS_ERR_VALUE(ssp)) + return PTR_ERR((void *)ssp); + *childregs = *(current_pt_regs()); /* Turn off status.VS */ riscv_v_vstate_off(childregs); if (usp) /* User fork */ childregs->sp = usp; + if (ssp) /* if needed, set new ssp */ + set_active_shstk(p, ssp); if (clone_flags & CLONE_SETTLS) childregs->tp = tls; childregs->a0 = 0; /* Return value of fork() */ diff --git a/arch/riscv/kernel/usercfi.c b/arch/riscv/kernel/usercfi.c index 35ede2cbc05b..36cac0d653f5 100644 --- a/arch/riscv/kernel/usercfi.c +++ b/arch/riscv/kernel/usercfi.c @@ -19,6 +19,41 @@ #define SHSTK_ENTRY_SIZE sizeof(void *) +bool is_shstk_enabled(struct task_struct *task) +{ + return task->thread_info.user_cfi_state.ubcfi_en ? true : false; +} + +void set_shstk_base(struct task_struct *task, unsigned long shstk_addr, unsigned long size) +{ + task->thread_info.user_cfi_state.shdw_stk_base = shstk_addr; + task->thread_info.user_cfi_state.shdw_stk_size = size; +} + +unsigned long get_shstk_base(struct task_struct *task, unsigned long *size) +{ + if (size) + *size = task->thread_info.user_cfi_state.shdw_stk_size; + return task->thread_info.user_cfi_state.shdw_stk_base; +} + +void set_active_shstk(struct task_struct *task, unsigned long shstk_addr) +{ + task->thread_info.user_cfi_state.user_shdw_stk = shstk_addr; +} + +/* + * If size is 0, then to be compatible with regular stack we want it to be as big as + * regular stack. Else PAGE_ALIGN it and return back + */ +static unsigned long calc_shstk_size(unsigned long size) +{ + if (size) + return PAGE_ALIGN(size); + + return PAGE_ALIGN(min_t(unsigned long long, rlimit(RLIMIT_STACK), SZ_4G)); +} + /* * Writes on shadow stack can either be `sspush` or `ssamoswap`. `sspush` can happen * implicitly on current shadow stack pointed to by CSR_SSP. `ssamoswap` takes pointer to @@ -148,3 +183,89 @@ SYSCALL_DEFINE3(map_shadow_stack, unsigned long, addr, unsigned long, size, unsi return allocate_shadow_stack(addr, aligned_size, size, set_tok); } + +/* + * This gets called during clone/clone3/fork. And is needed to allocate a shadow stack for + * cases where CLONE_VM is specified and thus a different stack is specified by user. We + * thus need a separate shadow stack too. How does separate shadow stack is specified by + * user is still being debated. Once that's settled, remove this part of the comment. + * This function simply returns 0 if shadow stack are not supported or if separate shadow + * stack allocation is not needed (like in case of !CLONE_VM) + */ +unsigned long shstk_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args) +{ + unsigned long addr, size; + + /* If shadow stack is not supported, return 0 */ + if (!cpu_supports_shadow_stack()) + return 0; + + /* + * If shadow stack is not enabled on the new thread, skip any + * switch to a new shadow stack. + */ + if (is_shstk_enabled(tsk)) + return 0; + + /* + * For CLONE_VFORK the child will share the parents shadow stack. + * Set base = 0 and size = 0, this is special means to track this state + * so the freeing logic run for child knows to leave it alone. + */ + if (args->flags & CLONE_VFORK) { + set_shstk_base(tsk, 0, 0); + return 0; + } + + /* + * For !CLONE_VM the child will use a copy of the parents shadow + * stack. + */ + if (!(args->flags & CLONE_VM)) + return 0; + + /* + * reaching here means, CLONE_VM was specified and thus a separate shadow + * stack is needed for new cloned thread. Note: below allocation is happening + * using current mm. + */ + size = calc_shstk_size(args->stack_size); + addr = allocate_shadow_stack(0, size, 0, false); + if (IS_ERR_VALUE(addr)) + return addr; + + set_shstk_base(tsk, addr, size); + + return addr + size; +} + +void shstk_release(struct task_struct *tsk) +{ + unsigned long base = 0, size = 0; + /* If shadow stack is not supported or not enabled, nothing to release */ + if (!cpu_supports_shadow_stack() || + !is_shstk_enabled(tsk)) + return; + + /* + * When fork() with CLONE_VM fails, the child (tsk) already has a + * shadow stack allocated, and exit_thread() calls this function to + * free it. In this case the parent (current) and the child share + * the same mm struct. Move forward only when they're same. + */ + if (!tsk->mm || tsk->mm != current->mm) + return; + + /* + * We know shadow stack is enabled but if base is NULL, then + * this task is not managing its own shadow stack (CLONE_VFORK). So + * skip freeing it. + */ + base = get_shstk_base(tsk, &size); + if (!base) + return; + + vm_munmap(base, size); + set_shstk_base(tsk, 0, 0); +}