From patchwork Wed Sep 21 16:45:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12983978 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CB5CBC6FA82 for ; Wed, 21 Sep 2022 16:46:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=7avUCiWoJxGE/1vrduQ4BjNBWxrTeMFa6sAU3iDTIBk=; b=VM3nAtPGQ5Nm6k zxq8Vm/pmX2p6r2CrFYvKaiUmIATdqXXfvVjlT0x4itoOPo9zTSU3uLdyCUg3vt3Ne4f59VRw1qkX klBBo+4BW8eHCupk553dYzNq6Wez5v7zUaWYi5gnmVHfBMnLu3oufPCsdoFfuJne3ZIK/BCXQ1f35 /RWhyuHKi6idlMowBEwAlkwJC1h4z6sjl6ktqUxOcBSTTrFVj3aKF7i6whsyiJ3A2nGI4BCBuRdJW 6AuetRw17Bil1KvmtRNJALhtDb4LiJuqKAztL/hMjxLJ1JqV5YtPvjS1uIw1gNJuQQ7ELfhj2seYe CagIgKDp6WXWqxSwK57A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2rc-00C23S-KF; Wed, 21 Sep 2022 16:46:20 +0000 Received: from mail-wr1-x42a.google.com ([2a00:1450:4864:20::42a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2rZ-00C21V-In for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:46:18 +0000 Received: by mail-wr1-x42a.google.com with SMTP id z6so10915093wrq.1 for ; Wed, 21 Sep 2022 09:46:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=Yw6P8uWcw4cBGcbaqsZpk8wARZfWeyzrk6CPKlNIqGs=; b=SJDfAUTkNX08HeNv7Zt2yt4svVIXntA1AMaa3KipMqgzA/3XzxtASXnplHXZSNHv1q 5wzt27EorXVk95cQ67SrbS+GhuA3YkFDJGNUe03MB+9C6AwctOUlIwgb234iKr9qOSQ/ 3mROmQWL5oP9tpkgx9/PUodmt13LYjo2DXgx0t3XflkYEQGjwiVDUB7Wiu+ETvQa3qdy L/EfGmivT+ER/3lQGNoQr2BMKkcri8Yd8VZ7yBILiMyqAi41SfJ5aoH42BlPWj82Rqzo 1s7zerN5lZpu9Nwf1vCNhvHLCXQ8XXuYCfhbwVb44/CbomwOTHo5ScSsXlcKDpcL8uIn W8rQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=Yw6P8uWcw4cBGcbaqsZpk8wARZfWeyzrk6CPKlNIqGs=; b=WNUEeML+1x5NbJqC/3y20pcAfd27Inu71lbnjr8XVvn9JRYwyiT8a4t2xOJE4xtqLM H7+iawQg0TRo/+57hDY+bShLeoO+rW6VHNKIX3CXXKnpj42NJHjsJ7w69h4CDWkrncKF D1uuNZk9c3W3vCdcgIE0gP/iYEpD20Y1lJEtxzu+zXzbbZC2H+DIgjLhnsLxK/yX1Qyg 7hXmY+aPRUg+316/KFXRW41/hGSCFi8gWfnq5DhP3SsuiPv+US38G+1R5fiX+Kn7uLsD VPU90JOmi3FuphJM860z9v/vaZYbG8k9fePlqLrkCINvjV7Aiw6zeLpGUXvo8NjOoas5 1SOw== X-Gm-Message-State: ACrzQf12S/EfOIpH+3V+wI1VLSlN7kmSgrfiH2Tpf+U5LAUMX7X3r0XD fviL23hPH6rJxARf01AaSszaDXZSR8HmoYpmkuWA66lahhZ4Qg== X-Google-Smtp-Source: AMsMyM7tO9Me9/88M6xHNncGBG95VQNc3dh6D/LBflbB8ZzRe4kwnfCLFPGUFg2KGHsW3v+MT7D7FeSPljtL5aHmFEE= X-Received: by 2002:adf:fb8f:0:b0:225:2def:221e with SMTP id a15-20020adffb8f000000b002252def221emr17848376wrr.130.1663778774822; Wed, 21 Sep 2022 09:46:14 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:45:38 -0700 Message-ID: Subject: [PATCH 01/17] riscv: Rename __switch_to_aux -> fpu To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_094617_637822_0E776C40 X-CRM114-Status: UNSURE ( 9.72 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org The name of __switch_to_aux is not clear and rename it with the determine function: __switch_to_fpu. Next we could add other regs' switch. Signed-off-by: Guo Ren Signed-off-by: Guo Ren Signed-off-by: Greentime Hu Reviewed-by: Anup Patel Reviewed-by: Palmer Dabbelt --- arch/riscv/include/asm/switch_to.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) -- 2.25.1 diff --git a/arch/riscv/include/asm/switch_to.h b/arch/riscv/include/asm/switch_to.h index 11463489fec6..df1aa589b7fd 100644 --- a/arch/riscv/include/asm/switch_to.h +++ b/arch/riscv/include/asm/switch_to.h @@ -46,7 +46,7 @@ static inline void fstate_restore(struct task_struct *task, } } -static inline void __switch_to_aux(struct task_struct *prev, +static inline void __switch_to_fpu(struct task_struct *prev, struct task_struct *next) { struct pt_regs *regs; @@ -65,7 +65,7 @@ static __always_inline bool has_fpu(void) static __always_inline bool has_fpu(void) { return false; } #define fstate_save(task, regs) do { } while (0) #define fstate_restore(task, regs) do { } while (0) -#define __switch_to_aux(__prev, __next) do { } while (0) +#define __switch_to_fpu(__prev, __next) do { } while (0) #endif extern struct task_struct *__switch_to(struct task_struct *, @@ -76,7 +76,7 @@ do { \ struct task_struct *__prev = (prev); \ struct task_struct *__next = (next); \ if (has_fpu()) \ - __switch_to_aux(__prev, __next); \ + __switch_to_fpu(__prev, __next); \ ((last) = __switch_to(__prev, __next)); \ } while (0) From patchwork Wed Sep 21 16:46:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12983990 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C7D08ECAAD8 for ; Wed, 21 Sep 2022 16:47:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=nfdVB31cP1Y9/C1jVHMKUxPSTsO5G6+kZD+yFWAUcxc=; b=jRXDudfEEAj/73 wBjZ2cazMcTDEwIOUahHyouG15olelOecldn9iUfQUuUdqibijFYqi0KJozieG5HPNkrYYCzVcO5I OzIi/rXamnMYy5vPjc0JvqGsw40zD5INcPZzKTY52Tn5UHz0yFPVa8DBoCkg7pw/3XuvL0BRX3WyL 4kSuImjmBEgG2vXMuEq03VrDPZcSTvntaEUAIRqJ39qfgeUMeAZ4a17QcK66tPp6hZNTmtMQTf9m5 zvBIa5fbBF3sT2RLfBdr/P1Yl+TcpBw86k0RhCmRklwaHj4L5kOsXo06cZxZmEPEjofeOnZQQMs4/ YtvPMOPucdAek1s3XAOw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2sR-00C2Ky-7x; Wed, 21 Sep 2022 16:47:11 +0000 Received: from mail-wm1-x32b.google.com ([2a00:1450:4864:20::32b]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2sM-00C2Hu-To for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:47:09 +0000 Received: by mail-wm1-x32b.google.com with SMTP id l8so4881690wmi.2 for ; Wed, 21 Sep 2022 09:47:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=IaicDJv7f5Dt9tD+h7CLRz4v0jAjgSVUGTvRxbodkhg=; b=3sU0tPRItlL5eswzbcZrS4epiOJWfRU+1EV1iop2VTBj6puiryrMUEyPJFqxe0BnA+ aVN+YAx5VmmRkK0ihSj8MfVf2mBL90mxb7iJMsgQYnNIddO2fYS/5ksgSXZ0m/jllPhA XGIG6JJLAbnzlUXXB29J0C0yC+uLpLcygnr5NM7uPj/0XnAZVD42sQ4C6bwDK83qhfcD MKahRQP7U4qXuxN1OiJA7BzLcYZXfvJJX9isG6W7r48g1sHqHZpVOfgiH873L18YgbKw fsO3AtOY+YqEAZi4TegCZVejeFph7xDeKxwGpgIeGR+r7+b6yidK3HiOVsDlJkUzXnk/ LOLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=IaicDJv7f5Dt9tD+h7CLRz4v0jAjgSVUGTvRxbodkhg=; b=poIiwAEy+EFlreXHc5sMca9Ay81o3SydQ5NIRgcrzjOfQotGqCRBZhQenB2XQWB1LR TyfR30MDTFvlJL2ohMje80kuWxrlL7Ib/JAPU1XD4Yk2vLhkualFJFSa7xE6vvjE2SuU uYLNlX4uEKcfqAHU2h/iLoM1UtrX4ADDUMosdAdeesmLZIdbeUt/HxsNwb9nFmElpJsd U7QGQHAD7chx4fdhIdpolgX1gGA0nTKZVur1Q/nbH1rSwZKJ+pQhch8gS4HA4vzBNVw3 ExuVoWoYQ51AuS8rzErbg3eI1ilcFqCWvZ2azB/AP0KPw5JFExuRUNdCQU1+mYE0y2vI t72w== X-Gm-Message-State: ACrzQf2+FlMPgJiZEVmMwPWUkf3OaOxWZ93HyP3Pdskt8Mu4OUFCGd94 o5dESZAIF8VdJt3QjvKBjnLpX+gJGfzSj9tyW8hLrbHcI8Burg== X-Google-Smtp-Source: AMsMyM5nEL3KU8p8Dn2FopGVo+1ah3Jp+xENkt72+BNWlaGcks0EoborsXg8xdnM+ntSybWhhrpbfpKhFvD9l5+Ypew= X-Received: by 2002:a05:600c:1c84:b0:3b3:ef37:afd3 with SMTP id k4-20020a05600c1c8400b003b3ef37afd3mr6312674wms.155.1663778822974; Wed, 21 Sep 2022 09:47:02 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:46:27 -0700 Message-ID: Subject: [PATCH 02/17] riscv: Extending cpufeature.c to detect V-extension To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_094706_990940_DB93AF25 X-CRM114-Status: UNSURE ( 8.91 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Current cpufeature.c doesn't support detecting V-extension, because "rv64" also contain a 'v' letter and we need to skip it. Signed-off-by: Guo Ren Signed-off-by: Guo Ren Reviewed-by: Anup Patel Reviewed-by: Greentime Hu Signed-off-by: Greentime Hu Reviewed-by: Palmer Dabbelt --- arch/riscv/include/uapi/asm/hwcap.h | 1 + arch/riscv/kernel/cpufeature.c | 1 + 2 files changed, 2 insertions(+) -- 2.25.1 diff --git a/arch/riscv/include/uapi/asm/hwcap.h b/arch/riscv/include/uapi/asm/hwcap.h index 46dc3f5ee99f..c52bb7bbbabe 100644 --- a/arch/riscv/include/uapi/asm/hwcap.h +++ b/arch/riscv/include/uapi/asm/hwcap.h @@ -21,5 +21,6 @@ #define COMPAT_HWCAP_ISA_F (1 << ('F' - 'A')) #define COMPAT_HWCAP_ISA_D (1 << ('D' - 'A')) #define COMPAT_HWCAP_ISA_C (1 << ('C' - 'A')) +#define COMPAT_HWCAP_ISA_V (1 << ('V' - 'A')) #endif /* _UAPI_ASM_RISCV_HWCAP_H */ diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 553d755483ed..8d4448c2d4f4 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -83,6 +83,7 @@ void __init riscv_fill_hwcap(void) isa2hwcap['f'] = isa2hwcap['F'] = COMPAT_HWCAP_ISA_F; isa2hwcap['d'] = isa2hwcap['D'] = COMPAT_HWCAP_ISA_D; isa2hwcap['c'] = isa2hwcap['C'] = COMPAT_HWCAP_ISA_C; + isa2hwcap['v'] = isa2hwcap['V'] = COMPAT_HWCAP_ISA_V; elf_hwcap = 0; From patchwork Wed Sep 21 16:46:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12983991 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C7881ECAAD8 for ; Wed, 21 Sep 2022 16:47:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=5EL3GWxiTTtyfd4+F5rZsSNYk3IIDcXT0xMHcLi3B3k=; b=AbeKjaXVJT9oiu l//MCktMc5VPcOfWRoohMAWJtwUeOUVDMy63FMn76d7mK/1KO+1hvvMzfZQO/n0j9/9t8jHsyRU8N 7+5g9WDP1bqGwOVqk0KmDGOYmVw3N3+ma6CBnWVKoVTTmBXUv/FnkQqsCtKK8YOIrE3J5NOmIO/L5 SKquub4cYnUJuSm0iGxT1OdrDwoa3yM5QufZPKOhGqw6UU5MpQB90TgjQQyg5V31nI9BrbEdeXxLT 0jACS1TTSz9NhD3u9+bhUQPpSV0W2qznzD5ajSoLQPuF8Zgsn1VLpzlFDYEMNFAyn2sRpupDnz83q bq8dY77IlSbg7lBPVvpA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2sj-00C2TV-Hz; Wed, 21 Sep 2022 16:47:29 +0000 Received: from mail-wr1-x434.google.com ([2a00:1450:4864:20::434]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2sg-00C2QX-UI for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:47:28 +0000 Received: by mail-wr1-x434.google.com with SMTP id y5so10910346wrh.3 for ; Wed, 21 Sep 2022 09:47:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=SfPOL29gvKfJCXS2UHaX/u5ZZWYH7zgP7jlLqWnd/Ds=; b=Ys43HplyPAH2OimY3/+SAvtATb42EFFeFFwUaIv53YGZtCWV02x8QeS/rcczF/2d2M 62JkJlpxzuBcKkDULSkDSC2sEgPcd0VLseRi+E4IxnNg/R8GvcLpLSTtS0wTfhHdzMvi CdzZU1NB/iX3/oLDupobdE1j4w1UsolFFma6ffsbFpqE3PCydR7iXwdxn4/xP8qDnjO9 rInjANPy7rx627Kgce6PxLOsSslEk0eFxjhdAVmSE/XSYC1Zn3eNBgn5B/HbOnBfaI4V 4wrAMoJbyWqPrdFwUarCEH6xx2p2jha40nzhyzfGDUPrdwLsiEYIADa60ZSw3S3PM2wc Peew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=SfPOL29gvKfJCXS2UHaX/u5ZZWYH7zgP7jlLqWnd/Ds=; b=njhQ1JfSQJPYficMDGdvcQTadPgomv22SApgyCEXzapAr81ybFoTmSDjG40NIRFgyA XvD4j7HVYNkDt+1y15VyeMfxVq5rUezJMGibbic6EdHG6Z18FW1OczCmFoe9HALi6Dt2 qqvYhjhdiMRjXK3muOc8eUdTKCshWEpQa/CnlwaEp2SP8T475OXkf8NVn4ipLO7gEp7Q M9LOex2GlOEyq+9pN/7xUngUV59QWDAir0Tmc++iKK1GnkSAainNINA4PA0rpG0qflpb 3gos0b0HozzZwFLPAzGdMJbkpuylFH88mHVS/CMzUimoZUYBebcKkuTaSk9H1wXDj7+W C6ag== X-Gm-Message-State: ACrzQf2hcFIgWPe3Y1nZ7pAGeBIyOApnIgYo/jrQUTfsC+jSeMQf6+VA gtIIn5/MqbuDO3onqDlhyB96LPYe3OHdUHGpUj4pzvSHrQ85sQ== X-Google-Smtp-Source: AMsMyM7NQZoEgDMWXgGtxSKkvcPa/gwF02Vef1WSEnyMddUo+J6Q/uArjxyi//tn6IM8O/DcvdfysDY/ONADduJ5rcs= X-Received: by 2002:adf:dcc8:0:b0:22a:b9e3:bab8 with SMTP id x8-20020adfdcc8000000b0022ab9e3bab8mr17948321wrm.341.1663778843031; Wed, 21 Sep 2022 09:47:23 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:46:47 -0700 Message-ID: Subject: [PATCH 03/17] riscv: Add new csr defines related to vector extension To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_094726_999812_75922B25 X-CRM114-Status: UNSURE ( 8.06 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Follow the riscv vector spec to add new csr numbers. [guoren@linux.alibaba.com: first porting for new vector related csr] Acked-by: Guo Ren Co-developed-by: Guo Ren Signed-off-by: Guo Ren Co-developed-by: Vincent Chen Signed-off-by: Vincent Chen Signed-off-by: Greentime Hu Reviewed-by: Palmer Dabbelt --- arch/riscv/include/asm/csr.h | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) -- 2.25.1 diff --git a/arch/riscv/include/asm/csr.h b/arch/riscv/include/asm/csr.h index 0e571f6483d9..78a68f29c3ba 100644 --- a/arch/riscv/include/asm/csr.h +++ b/arch/riscv/include/asm/csr.h @@ -24,6 +24,12 @@ #define SR_FS_CLEAN _AC(0x00004000, UL) #define SR_FS_DIRTY _AC(0x00006000, UL) +#define SR_VS _AC(0x00000600, UL) /* Vector Status */ +#define SR_VS_OFF _AC(0x00000000, UL) +#define SR_VS_INITIAL _AC(0x00000200, UL) +#define SR_VS_CLEAN _AC(0x00000400, UL) +#define SR_VS_DIRTY _AC(0x00000600, UL) + #define SR_XS _AC(0x00018000, UL) /* Extension Status */ #define SR_XS_OFF _AC(0x00000000, UL) #define SR_XS_INITIAL _AC(0x00008000, UL) @@ -31,9 +37,9 @@ #define SR_XS_DIRTY _AC(0x00018000, UL) #ifndef CONFIG_64BIT -#define SR_SD _AC(0x80000000, UL) /* FS/XS dirty */ +#define SR_SD _AC(0x80000000, UL) /* FS/VS/XS dirty */ #else -#define SR_SD _AC(0x8000000000000000, UL) /* FS/XS dirty */ +#define SR_SD _AC(0x8000000000000000, UL) /* FS/VS/XS dirty */ #endif #ifdef CONFIG_64BIT @@ -297,6 +303,12 @@ #define CSR_MIMPID 0xf13 #define CSR_MHARTID 0xf14 +#define CSR_VSTART 0x8 +#define CSR_VCSR 0xf +#define CSR_VL 0xc20 +#define CSR_VTYPE 0xc21 +#define CSR_VLENB 0xc22 + #ifdef CONFIG_RISCV_M_MODE # define CSR_STATUS CSR_MSTATUS # define CSR_IE CSR_MIE From patchwork Wed Sep 21 16:47:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12983992 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8C80AC6FA82 for ; Wed, 21 Sep 2022 16:47:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=baY4sJ0azBfLKFQoZ7SH9gQlD+zXZ0MJjl128nDCbGY=; b=owgj76F5g67qco StZ12/Qvrh5Atpppxi2BFy9tQr+2hedbJb7S2tbTzLpnxpnAHNcqChxksqobXYvUkaJxImJL2AGyY Y1ZIrBkoJPZKj3t28bsZMI8Lhftrc8fpTiCcWVBYndC0J2CuXoTrnBLdhZSppR7XIqHpBTN59c4xd MU+WZ6U6cNObsLytdhZKvuv1QAH+lmLc+UYpvSj2hw/sLTEBuGAx+SPcN6xD81PwH5VeBtWk4tI5s jG9/oNJk32nO9GWsJ1hpC9Y8mRstx/00Hu2Dpmo9USCaBQTYpuIYgfHvh2Gb05jR2ROUH+AUPHTcf W1J/yNzbZydZzZsgN6KA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2t2-00C2dV-40; Wed, 21 Sep 2022 16:47:48 +0000 Received: from mail-wr1-x42a.google.com ([2a00:1450:4864:20::42a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2sz-00C2bx-QW for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:47:47 +0000 Received: by mail-wr1-x42a.google.com with SMTP id t14so10850740wrx.8 for ; Wed, 21 Sep 2022 09:47:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=OQfg0vHlGZAcqp1vCrJ92OrmG/vN4V+b6Jsl+hTmAIg=; b=pdGoYuOwx5DRYGSfzFyEKt1SIzRLSlYt0AH/lnNM4mV/nAeBTrPtNki0ju9YvB9Pov DvB83tOQJgt6ktyTzTzh7EWL8/1Llk4t+Pw0/737ZQX4kJZoNU2KAFws7BmD+DW+kEh4 8BU9Yw/mUeHca4reinffWHDSH7QgzBU1D/0glbo7sMQUsPWs7h4e2Dz9WtEAEaM2mX4R aJJz+Krd6wW8DUL3KJNdNzGANyaz9U1EoqiTtwhjvO8c2wwWvoMWXR2UwE9bHyomVYj8 mgf/gg/EL6L1fIwG/qsJi8RYYSK7//zhEBsoui2ItB3kxOW8OZPUfYaYFDjXf2witTMd GWdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=OQfg0vHlGZAcqp1vCrJ92OrmG/vN4V+b6Jsl+hTmAIg=; b=y8UWvgGPK+CnJkJjrNXNbxKFdfYhf9yM9N1J9J7osLJDrjRYzktPIhH3dvEzHXC1OI I7bxXwqL7iMZtKwNLVDwz49vD+rpn6d2GLZeIV0mS6bPWDMTsLpMMbuAsdzDgzVE//yV Rg8MAO1lA1dKT8TcEmDh5nx3KLiYulkw4uVwKmzFGbpqFmtaFmGMpYNtG5FiIvKJig0E 6nchHGEKTPpLCNBlW6WVeqwVUt4sdGTCHzsQEMu61LkMOUXDwIt+ygO7UIJRK7fIVgYg /NyfeyRQ72ZcCWC9QWGUF31pDkXYJ8lwPWfKPkUov3bGGdckx4cVrHn21o3up/Ovpp4I wM2A== X-Gm-Message-State: ACrzQf03MyTxPac+Tl6/VFjvUJsvvRfVlevWxO8tdOeyZPHc+C4HzS3L leow4fBppEWYD33z9EICmMRhntMELHY2WcvusyCGLyFy96IEaA== X-Google-Smtp-Source: AMsMyM4k1FZ5lHKI9gbfA8Xh+YXoymj5pjr6SD3REsoCNgCjzEpjNlpjR28QJXxV9kzeLeR1TLCDBiQkwS2M+QR07Qo= X-Received: by 2002:a05:6000:243:b0:226:d241:11e6 with SMTP id m3-20020a056000024300b00226d24111e6mr17453615wrz.187.1663778864114; Wed, 21 Sep 2022 09:47:44 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:47:08 -0700 Message-ID: Subject: [PATCH 04/17] riscv: Add vector feature to compile To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_094745_884649_72F19C01 X-CRM114-Status: GOOD ( 12.08 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This patch adds a new config option which could enable assembler's vector feature. Signed-off-by: Guo Ren Co-developed-by: Greentime Hu Signed-off-by: Greentime Hu --- arch/riscv/Kconfig | 15 +++++++++++++-- arch/riscv/Makefile | 1 + 2 files changed, 14 insertions(+), 2 deletions(-) -- 2.25.1 diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index ed66c31e4655..e294d85bfb7d 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -432,7 +432,17 @@ config FPU If you don't know what to do here, say Y. -endmenu # "Platform type" +config VECTOR + bool "VECTOR support" + depends on GCC_VERSION >= 120000 || CLANG_VERSION >= 130000 + default n + help + Say N here if you want to disable all vector related procedure + in the kernel. + + If you don't know what to do here, say Y. + +endmenu menu "Kernel features" @@ -556,6 +566,7 @@ config CMDLINE_EXTEND cases where the provided arguments are insufficient and you don't want to or cannot modify them. + config CMDLINE_FORCE bool "Always use the default kernel command string" help @@ -648,7 +659,7 @@ config XIP_PHYS_ADDR be linked for and stored to. This address is dependent on your own flash usage. -endmenu # "Boot options" +endmenu config BUILTIN_DTB bool diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile index 3fa8ef336822..1ec17f3d6d09 100644 --- a/arch/riscv/Makefile +++ b/arch/riscv/Makefile @@ -50,6 +50,7 @@ riscv-march-$(CONFIG_ARCH_RV32I) := rv32ima riscv-march-$(CONFIG_ARCH_RV64I) := rv64ima riscv-march-$(CONFIG_FPU) := $(riscv-march-y)fd riscv-march-$(CONFIG_RISCV_ISA_C) := $(riscv-march-y)c +riscv-march-$(CONFIG_VECTOR) := $(riscv-march-y)v # Newer binutils versions default to ISA spec version 20191213 which moves some # instructions from the I extension to the Zicsr and Zifencei extensions. From patchwork Wed Sep 21 16:47:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12983993 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3A321ECAAD8 for ; Wed, 21 Sep 2022 16:48:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=EasbZXpNjyPFJcTspq59FynoMbgc09BAUtOvJq9OKpg=; b=Qde9qjz9ZMZoXe 430yaUDc6nH9NNV1hW+9rqwsH1dh/qh3ejoQVewQAcqdEOq9tE+RvOUL1b3RDcbR3jMbADnfVfolp ApSeP3OO0E/yJYfZA1jJh8HpEVhzPDr6MNlcNMHTykeZ0tlkuztVcGiGSD6bYcTaW4OI5pfxe9j6a aeWyxo6/3QhLFKDKPrbf5l6qGRPEf7U3eEEvJa4HJ5AkrFbTA0HADfFTKPbQIM678ICFNLTciPUUq xVCGL7pfUo/GldYkDeJsiVI3Rk8S9IXetVLLq8ZG/OWmbvOMGTQVZfs0zEJa2m2DrFyArXyDPF5G4 /4XJN/sblSr3mjzHuNLA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2tP-00C2oI-Rb; Wed, 21 Sep 2022 16:48:11 +0000 Received: from mail-wr1-x435.google.com ([2a00:1450:4864:20::435]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2tM-00C2kX-7L for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:48:09 +0000 Received: by mail-wr1-x435.google.com with SMTP id n15so3950354wrq.5 for ; Wed, 21 Sep 2022 09:48:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=UPPE0WNw9dds50E1SR1uWrRZWS8hSokvrPlI99Jv2ZM=; b=OsrPQefhMNudSFxowvYvSAGzIlzsJfb3yMkrGo9uFTgDmOJRb352txkXujKz1WQQFt vXk61RRoYWd5bMljF3IFpkd6I3HzPolT41WVY6BZpcXptQl0ICcvCMM18QEWwEcsim1T w8Fte6Q5Ca1D6DLhvqh47/JgRpOnNUYIsaM1aI1mgLFgBe7T3aPo9xFRrFTv/2o/h4+Y 6rYKp52bgfC+lY5Y860IcAyf9ZKL1Ij6hizvJypPQDFl5dd88vzJoRaTXfuwVcWRaMdp Bp+Xz6XLNg2AeBINvgAovExnYPR9WrFGWncXxcGURLpYLOzurj/Z47S0xeOHLxFKw2KU 3v4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=UPPE0WNw9dds50E1SR1uWrRZWS8hSokvrPlI99Jv2ZM=; b=Akbw2buOsutQeIkfDLq7vUlJ2ddc6oIDOgEjwUeeByW1oTBhEoMteB5yubM2oHCr/+ dziExDYm8yiNoAW15ZwtCjG6emMOpc5lZDm15nmXmU0ZxK1KafvQjOIZrfS3X0SUt40c KXfoxrEI8j7I/B2lmPD79tc/dU7VPj7M2Of0nBM20mhwc53i/dJsEWvCs3GnpGxJ6JgF f5ZAPYkICtVCUdlDrNwjaUMXnWGJRZPAGoNxMCq0uAct3wmlJitb0xkWwPS6ZzrhHTBZ d23w0cNmFjIaZA33+mRbUtAIfnaR4jop1vruJfe51OJ1lsTazSdmND/kLMcZqzXSn4aI D0cg== X-Gm-Message-State: ACrzQf2kxcb+chsfNgeqNcZs/xzueR+/Kjh5Mak5wdxdTPqFxQvxdbg6 rEHJt8aCLrZuQP3IRbUzp24+LvghQkIDCdZnYIS5i8oWCy1TXw== X-Google-Smtp-Source: AMsMyM7mMm2lVfTRU8X4Qt6FrQipuqjXAQTPMOdPKdBZPGPDtuf5a7vNk/YQuLJd4DOPxvaW5PnrRtHS3p9shmyxV/o= X-Received: by 2002:a5d:588f:0:b0:22b:623:ad04 with SMTP id n15-20020a5d588f000000b0022b0623ad04mr8689581wrf.607.1663778883146; Wed, 21 Sep 2022 09:48:03 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:47:27 -0700 Message-ID: Subject: [PATCH 05/17] riscv: Add has_vector/riscv_vsize to save vector features. To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_094808_287439_87373B96 X-CRM114-Status: GOOD ( 17.25 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This patch is used to detect vector support status of CPU and use riscv_vsize to save the size of all the vector registers. It assumes all harts has the same capabilities in SMP system. [guoren@linux.alibaba.com: add has_vector checking] Co-developed-by: Guo Ren Signed-off-by: Guo Ren Co-developed-by: Vincent Chen Signed-off-by: Vincent Chen Signed-off-by: Greentime Hu --- arch/riscv/include/asm/vector.h | 14 +++++ arch/riscv/kernel/cpufeature.c | 19 +++++++ arch/riscv/kernel/riscv_ksyms.c | 6 +++ arch/riscv/kernel/vector.S | 93 +++++++++++++++++++++++++++++++++ 4 files changed, 132 insertions(+) create mode 100644 arch/riscv/include/asm/vector.h create mode 100644 arch/riscv/kernel/vector.S -- 2.25.1 diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h new file mode 100644 index 000000000000..16304b0c6a6f --- /dev/null +++ b/arch/riscv/include/asm/vector.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2020 SiFive + */ + +#ifndef __ASM_RISCV_VECTOR_H +#define __ASM_RISCV_VECTOR_H + +#include + +void rvv_enable(void); +void rvv_disable(void); + +#endif /* ! __ASM_RISCV_VECTOR_H */ diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 8d4448c2d4f4..0487ab19b234 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -30,6 +30,14 @@ static DECLARE_BITMAP(riscv_isa, RISCV_ISA_EXT_MAX) __read_mostly; __ro_after_init DEFINE_STATIC_KEY_ARRAY_FALSE(riscv_isa_ext_keys, RISCV_ISA_EXT_KEY_MAX); EXPORT_SYMBOL(riscv_isa_ext_keys); +#ifdef CONFIG_FPU +__ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_fpu); +#endif +#ifdef CONFIG_VECTOR +#include +__ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_vector); +unsigned long riscv_vsize __read_mostly; +#endif /** * riscv_isa_extension_base() - Get base extension word @@ -249,6 +257,16 @@ void __init riscv_fill_hwcap(void) if (j >= 0) static_branch_enable(&riscv_isa_ext_keys[j]); } + +#ifdef CONFIG_VECTOR + if (elf_hwcap & COMPAT_HWCAP_ISA_V) { + static_branch_enable(&cpu_hwcap_vector); + /* There are 32 vector registers with vlenb length. */ + rvv_enable(); + riscv_vsize = csr_read(CSR_VLENB) * 32; + rvv_disable(); + } +#endif } #ifdef CONFIG_RISCV_ALTERNATIVE @@ -328,3 +346,4 @@ void __init_or_module riscv_cpufeature_patch_func(struct alt_entry *begin, } } #endif +} diff --git a/arch/riscv/kernel/riscv_ksyms.c b/arch/riscv/kernel/riscv_ksyms.c index 5ab1c7e1a6ed..3489d2a20ca3 100644 --- a/arch/riscv/kernel/riscv_ksyms.c +++ b/arch/riscv/kernel/riscv_ksyms.c @@ -15,3 +15,9 @@ EXPORT_SYMBOL(memmove); EXPORT_SYMBOL(__memset); EXPORT_SYMBOL(__memcpy); EXPORT_SYMBOL(__memmove); + +#ifdef CONFIG_VECTOR +#include +EXPORT_SYMBOL(rvv_enable); +EXPORT_SYMBOL(rvv_disable); +#endif diff --git a/arch/riscv/kernel/vector.S b/arch/riscv/kernel/vector.S new file mode 100644 index 000000000000..9f7dc70c4443 --- /dev/null +++ b/arch/riscv/kernel/vector.S @@ -0,0 +1,93 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2012 Regents of the University of California + * Copyright (C) 2017 SiFive + * Copyright (C) 2019 Alibaba Group Holding Limited + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation, version 2. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include + +#include +#include +#include + +#define vstatep a0 +#define datap a1 +#define x_vstart t0 +#define x_vtype t1 +#define x_vl t2 +#define x_vcsr t3 +#define incr t4 +#define status t5 + +ENTRY(__vstate_save) + li status, SR_VS + csrs CSR_STATUS, status + + csrr x_vstart, CSR_VSTART + csrr x_vtype, CSR_VTYPE + csrr x_vl, CSR_VL + csrr x_vcsr, CSR_VCSR + vsetvli incr, x0, e8, m8, ta, ma + vse8.v v0, (datap) + add datap, datap, incr + vse8.v v8, (datap) + add datap, datap, incr + vse8.v v16, (datap) + add datap, datap, incr + vse8.v v24, (datap) + + REG_S x_vstart, RISCV_V_STATE_VSTART(vstatep) + REG_S x_vtype, RISCV_V_STATE_VTYPE(vstatep) + REG_S x_vl, RISCV_V_STATE_VL(vstatep) + REG_S x_vcsr, RISCV_V_STATE_VCSR(vstatep) + + csrc CSR_STATUS, status + ret +ENDPROC(__vstate_save) + +ENTRY(__vstate_restore) + li status, SR_VS + csrs CSR_STATUS, status + + vsetvli incr, x0, e8, m8, ta, ma + vle8.v v0, (datap) + add datap, datap, incr + vle8.v v8, (datap) + add datap, datap, incr + vle8.v v16, (datap) + add datap, datap, incr + vle8.v v24, (datap) + + REG_L x_vstart, RISCV_V_STATE_VSTART(vstatep) + REG_L x_vtype, RISCV_V_STATE_VTYPE(vstatep) + REG_L x_vl, RISCV_V_STATE_VL(vstatep) + REG_L x_vcsr, RISCV_V_STATE_VCSR(vstatep) + vsetvl x0, x_vl, x_vtype + csrw CSR_VSTART, x_vstart + csrw CSR_VCSR, x_vcsr + + csrc CSR_STATUS, status + ret +ENDPROC(__vstate_restore) + +ENTRY(rvv_enable) + li status, SR_VS + csrs CSR_STATUS, status + ret +ENDPROC(rvv_enable) + +ENTRY(rvv_disable) + li status, SR_VS + csrc CSR_STATUS, status + ret +ENDPROC(rvv_disable) From patchwork Wed Sep 21 16:47:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12983994 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C2791C6FA82 for ; Wed, 21 Sep 2022 16:48:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=sevLiGq1Kqj5V2xb3QGGtDbZdo/N4p+2FChUOBqKvio=; b=0weYBGj94SldAF JZXezLjYtIlbPLRBqtyVjaRvCm3WoQh5EVkKnWMeLVDD3TBd059cNX+4HwlYXxEVCqamN0OGgGf4n Ifbqxv2xD0VwqL3AoRjSFpm0vlbfyjY+GCafPyL3K7vAwLav+q9gu53XIi9Tc99EE31lSLal/GpvT z4WRuCTjNy1SmvGnARsBJS760SSYv+CFmXwUhdLK9Ip95pHAA7klONI8JO0BKgLfQl6RVw/9c3spO lxmdEXhnSu8F6EFDu3/BOpikikqXmc/jkwGSGaUd7qa0nsiH5lEGktPTCpo4/l80vZntakmRvBAg2 ZEJ4bJe0jiezjfvWcAeQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2td-00C2us-Be; Wed, 21 Sep 2022 16:48:25 +0000 Received: from mail-wr1-x430.google.com ([2a00:1450:4864:20::430]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2ta-00C2s4-Qw for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:48:24 +0000 Received: by mail-wr1-x430.google.com with SMTP id bq9so10895769wrb.4 for ; Wed, 21 Sep 2022 09:48:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=bCwFGGiYjvgxFVbbvI83I9ZZV9BKobbourF/lTUeiTk=; b=Ei/PzGmXjiM06pJA0uzqeeOlNWOjgBr9044m2+nSEqji3tR7WHIexkzoK+wmLB7BHb wjBpsyoZsxrgu/a3RCzbGgmvSkocfLd0R9oJtqzyTNJMgbpnCTdL/jMHFUR6wvX/iNV8 izPPR41atsjCag7jxYJvtqn9nA8+RuB6H8npMm+owRY9dCIcGeEjqMrV/WK0rikpF78o Gbp/9H2JtLA2YRHvHvnpYWJB7K47NQVE0FWKirkHsHYRj6N+Chd1hFUSfSL819r0NnbG tulK/ox/kvMUUW2vDTZkhbwnxcs+fbNTfhzJ0YqEsyjFIm6zS/oBhbDMsnRL9NMlAQX9 zNcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=bCwFGGiYjvgxFVbbvI83I9ZZV9BKobbourF/lTUeiTk=; b=nxvgxvJMpA9NFwsw57BQCOcrApqbiQoJWipsnAAWIy9CUp9GAKfDt7LmXtJ6j6Oaqx WuhDXovcXCerNpCHXPFbcvTjsNmKnUHdnqwgcUjMbv87D3QxzScJSQMFYAzwsAsrHL/v WCzqJZFe90BGsEsiJgGgJanxqABXjHIX4cqcVbJ8bT7uQYDH/IOPvDjLAN6k8qp7yAB3 vnEyox1aKIKjgdavp9NPaF1HJdr/DkOP9sOwkJFxfZkiUzjns9TL1GvVhyj849N0wzoG OR/+eCfhQ1zedmUENSluSV+QGxeWjMKWiSas4toiBphM2KgQQi0dtJlsn7O/V4rWb4sq /sbg== X-Gm-Message-State: ACrzQf0kTbuMzliVnFsw5xgJj1EWOV0IwID0DzvzhDn4ETcR38zD//Qw rMKNRHyrkqVlum70Rz9HjPZh7KRrhs3aQ0X1zvJ7oRv0A2fNUg== X-Google-Smtp-Source: AMsMyM5CkzTsg1tH2O/s4oC7YcvnLgGuV40sXjGbBGdfSbzQFpEFcyehwFKqEgrsrdZ3rj3fGAqxQpd97yeu1OYaLKw= X-Received: by 2002:a05:6000:243:b0:226:d241:11e6 with SMTP id m3-20020a056000024300b00226d24111e6mr17455107wrz.187.1663778898900; Wed, 21 Sep 2022 09:48:18 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:47:43 -0700 Message-ID: Subject: [PATCH 06/17] riscv: Reset vector register To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_094822_913059_272AA2DB X-CRM114-Status: GOOD ( 11.46 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Reset vector registers at boot-time and disable vector instructions execution for kernel mode. Signed-off-by: Guo Ren Co-developed-by: Vincent Chen Signed-off-by: Vincent Chen Co-developed-by: Han-Kuan Chen Signed-off-by: Han-Kuan Chen Co-developed-by: Greentime Hu Signed-off-by: Greentime Hu Reviewed-by: Palmer Dabbelt --- arch/riscv/kernel/entry.S | 6 +++--- arch/riscv/kernel/head.S | 35 +++++++++++++++++++++++++++++------ 2 files changed, 32 insertions(+), 9 deletions(-) -- 2.25.1 diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S index b9eda3fcbd6d..1e9987376591 100644 --- a/arch/riscv/kernel/entry.S +++ b/arch/riscv/kernel/entry.S @@ -77,10 +77,10 @@ _save_context: * Disable user-mode memory access as it should only be set in the * actual user copy routines. * - * Disable the FPU to detect illegal usage of floating point in kernel - * space. + * Disable the FPU/Vector to detect illegal usage of floating point + * or vector in kernel space. */ - li t0, SR_SUM | SR_FS + li t0, SR_SUM | SR_FS | SR_VS REG_L s0, TASK_TI_USER_SP(tp) csrrc s1, CSR_STATUS, t0 diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S index b865046e4dbb..2c81ca42ec4e 100644 --- a/arch/riscv/kernel/head.S +++ b/arch/riscv/kernel/head.S @@ -140,10 +140,10 @@ secondary_start_sbi: .option pop /* - * Disable FPU to detect illegal usage of - * floating point in kernel space + * Disable FPU & VECTOR to detect illegal usage of + * floating point or vector in kernel space */ - li t0, SR_FS + li t0, SR_FS | SR_VS csrc CSR_STATUS, t0 /* Set trap vector to spin forever to help debug */ @@ -234,10 +234,10 @@ pmp_done: .option pop /* - * Disable FPU to detect illegal usage of - * floating point in kernel space + * Disable FPU & VECTOR to detect illegal usage of + * floating point or vector in kernel space */ - li t0, SR_FS + li t0, SR_FS | SR_VS csrc CSR_STATUS, t0 #ifdef CONFIG_RISCV_BOOT_SPINWAIT @@ -431,6 +431,29 @@ ENTRY(reset_regs) csrw fcsr, 0 /* note that the caller must clear SR_FS */ #endif /* CONFIG_FPU */ + +#ifdef CONFIG_VECTOR + csrr t0, CSR_MISA + li t1, COMPAT_HWCAP_ISA_V + and t0, t0, t1 + beqz t0, .Lreset_regs_done + + /* + * Clear vector registers and reset vcsr + * VLMAX has a defined value, VLEN is a constant, + * and this form of vsetvli is defined to set vl to VLMAX. + */ + li t1, SR_VS + csrs CSR_STATUS, t1 + csrs CSR_VCSR, x0 + vsetvli t1, x0, e8, m8, ta, ma + vmv.v.i v0, 0 + vmv.v.i v8, 0 + vmv.v.i v16, 0 + vmv.v.i v24, 0 + /* note that the caller must clear SR_VS */ +#endif /* CONFIG_VECTOR */ + .Lreset_regs_done: ret END(reset_regs) From patchwork Wed Sep 21 16:48:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12983995 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BA6FFECAAD8 for ; Wed, 21 Sep 2022 16:48:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=DXIrnseRzr9p1MIW0m3z15fv08F4JnyaeFZKx6XJ+Fs=; b=S16li9e+azEaSO xhaG6EqM32ByVhC/oHJGvtC7FbyoE3EQLJZI4WnaQVdx6Q0ciWUnL8REXPD/P+3TMs13wIV4E0v1Z 5Q0KecMOgGks1bGAUAhRtZ3lYTA/DTotvkJC3v/psLRuMeb5L7nBtZ9unCbD+uKNHGCUuJoJ1jqxq 33Zgel8qbZ1+Av2dmtujF+68T19DI4hzn2GnQmgJyz6esh+3iSdNh0mFfrAZIaUF4cHKTeXWmrWDI eovnGA4bHcV6AqT4JW16DZmUW7EXgNZQFNAVJgwuH+X/GYuMGDpTO9T8UZlTNdqFE8nM2mA1tsxqK QTexH0fNR7csECcC5KQA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2tw-00C33P-Vi; Wed, 21 Sep 2022 16:48:45 +0000 Received: from mail-wr1-x430.google.com ([2a00:1450:4864:20::430]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2tt-00C2s4-7h for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:48:42 +0000 Received: by mail-wr1-x430.google.com with SMTP id bq9so10897917wrb.4 for ; Wed, 21 Sep 2022 09:48:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=YCN2tVuugvi8qTfs+QkeI544lSmy4+FJUIa8ytD8sjY=; b=sJrxzeFxvLWTi2acUQTe8MR9BfkAvZ78komlubFTwPNNnj6DdlcU9MzFvzqqyJrwCu GaATgzwENxOwuhTZZyEd3lVZsUUeX+Dlr9DlKh2K+/2UCws/iwg9JQbi6VksIshJNCqu SNgNALGl8kjFqLjd4QSbekQHuS4TgslDPiOLGfA9WTi49By/9eBCOwertGucKpAdn8NP PluDui0eEgXeUQszcM7fo+G+rwZ14+7fssNTxUBQG2N3MjDWgFDRtHlekY6CVzdPy7Au UubTtuuTegZNtRxaJTuEJfgqTGY+hvoo6MOrICh+Jsk5eq824i6lTiXks5AjbmTb0R03 aaCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=YCN2tVuugvi8qTfs+QkeI544lSmy4+FJUIa8ytD8sjY=; b=Tia/ZjaWvnCjjas8IG+UTkkGQItgTROmcNHEl1sA5gaErIRNyu8WWa2lqwAidtTaeM j9uaBgaQMqMrKeM6aoDlGJucdwmIXNGOek5Banmr2uqQshJ+JQIJaWV2MxVJy5JVbZaS wZRJzthyi1ZucNjg0/W5aW+SQcfcLDPZQfZashE7OwVh5qm8MruogGhJ014VyHCAIhdJ sVYWcFT8biIGdPK8iYT73lCQ3XzsA+rkVKuAU39z09Gh/H0oJrh/pz0tVEhEucCchBoG jYPZHiEX1ZTzBhH9VdrR8ySJjuwp3dhOlc2CUID9/XhcRlEnefnx0kxsENUL7yBsaSc5 Oi+Q== X-Gm-Message-State: ACrzQf1XjXlbMIw6I2mOL+5RMEFlbBRJr5z5o/PWSL2jdWnWVIX4aNih nkd1iHa/3Ww1xsuxqhdVb5VNVgNUnNP56IXPqINk6vUhddRaXg== X-Google-Smtp-Source: AMsMyM6TJUz28ah59o30o4bgEO2bdaJzODFws/bNSqDhn366WPrqA7Na4NFBsDwSovWdk9bfQAC3NcYul7N7vWVp1BQ= X-Received: by 2002:adf:fb8f:0:b0:225:2def:221e with SMTP id a15-20020adffb8f000000b002252def221emr17854387wrr.130.1663778920538; Wed, 21 Sep 2022 09:48:40 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:48:04 -0700 Message-ID: Subject: [PATCH 07/17] riscv: Add vector struct and assembler definitions To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_094841_317044_8786CDE8 X-CRM114-Status: GOOD ( 11.18 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Add vector state context struct in struct thread and asm-offsets.c definitions. The vector registers will be saved in datap pointer of __riscv_v_state. It will be dynamically allocated in kernel space. It will be put right after the __riscv_v_state data structure in user space. Co-developed-by: Vincent Chen Signed-off-by: Vincent Chen Signed-off-by: Greentime Hu --- arch/riscv/include/asm/processor.h | 1 + arch/riscv/include/uapi/asm/ptrace.h | 17 +++++++++++++++++ arch/riscv/kernel/asm-offsets.c | 6 ++++++ 3 files changed, 24 insertions(+) -- 2.25.1 diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h index 19eedd4af4cd..95917a2b24f9 100644 --- a/arch/riscv/include/asm/processor.h +++ b/arch/riscv/include/asm/processor.h @@ -39,6 +39,7 @@ struct thread_struct { unsigned long s[12]; /* s[0]: frame pointer */ struct __riscv_d_ext_state fstate; unsigned long bad_cause; + struct __riscv_v_state vstate; }; /* Whitelist the fstate from the task_struct for hardened usercopy */ diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h index 882547f6bd5c..6ee1ca2edfa7 100644 --- a/arch/riscv/include/uapi/asm/ptrace.h +++ b/arch/riscv/include/uapi/asm/ptrace.h @@ -77,6 +77,23 @@ union __riscv_fp_state { struct __riscv_q_ext_state q; }; +struct __riscv_v_state { + unsigned long vstart; + unsigned long vl; + unsigned long vtype; + unsigned long vcsr; + void *datap; + /* + * In signal handler, datap will be set a correct user stack offset + * and vector registers will be copied to the address of datap + * pointer. + * + * In ptrace syscall, datap will be set to zero and the vector + * registers will be copied to the address right after this + * structure. + */ +}; + #endif /* __ASSEMBLY__ */ #endif /* _UAPI_ASM_RISCV_PTRACE_H */ diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c index df9444397908..37e3e6a8d877 100644 --- a/arch/riscv/kernel/asm-offsets.c +++ b/arch/riscv/kernel/asm-offsets.c @@ -75,6 +75,12 @@ void asm_offsets(void) OFFSET(TSK_STACK_CANARY, task_struct, stack_canary); #endif + OFFSET(RISCV_V_STATE_VSTART, __riscv_v_state, vstart); + OFFSET(RISCV_V_STATE_VL, __riscv_v_state, vl); + OFFSET(RISCV_V_STATE_VTYPE, __riscv_v_state, vtype); + OFFSET(RISCV_V_STATE_VCSR, __riscv_v_state, vcsr); + OFFSET(RISCV_V_STATE_DATAP, __riscv_v_state, datap); + DEFINE(PT_SIZE, sizeof(struct pt_regs)); OFFSET(PT_EPC, pt_regs, epc); OFFSET(PT_RA, pt_regs, ra); From patchwork Wed Sep 21 16:48:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12983996 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 633DDC6FA8B for ; Wed, 21 Sep 2022 16:49:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=nUrSVsb82w/p74jRrFTSCBggmVuhMXYPfSnRvi4wzPQ=; b=frxBRttu2GW+ve niEjDgVsNuR4hZiFKwFk/l4ltM6V4Dt+iLBCgvpgU4ikK+OC+VsFn/C9dQBc++1yVR+W6uUh8OHWv 1uQNVrIFqSZXzSv0J4UEYYeOr0UHWzgRETlJDN6Mn55yXBZ3fw/Ky5fCLG60bwJY7wCWdt7CqQfpi s5Q5cYorySTwyqyyJdd2CubkXRHV6micrMYJ1HNmtPyacjnCZeyFdFDavyXBiMKpp/HugI+Nq3yf1 RjIIUMe2tLZmffSVAf/hHOHHbpyU20aSqyWm5dTxhvN0xt1Uwj6Lm1mwYmH9InMJ3zT7DEhR8nPxH TzDtZnY3BJsLsJajjH1w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2uG-00C3DW-Mz; Wed, 21 Sep 2022 16:49:04 +0000 Received: from mail-wm1-x334.google.com ([2a00:1450:4864:20::334]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2uD-00C3Ah-Pn for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:49:03 +0000 Received: by mail-wm1-x334.google.com with SMTP id iv17so265994wmb.4 for ; Wed, 21 Sep 2022 09:48:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=I4ghk0ykr10Y3ofFJtkABGhBT0o8tksw4duAtzlyuag=; b=ufWgpSBC3zx+JozVzv1YnL4PD1IlUDuwEBenyq5ffpeNwVCWi4TbbNE3V5kwyRML6b 74WM/72bPXkZweaSs5hJPbcwbO2II6+saAmPKdegqDRyrvPdM7Jj/T56nOq/RDlzTrsr wty1AmKiEtr/mdVoKN34u224gH6dBo5l8tY6fDA2L4t5GQ3rPfUnQv5BkH6ec6ri3vS/ fTvqi97CKux9nnip0U1TcQcBHt/96CNwoAwVbdJenjVQn3qg+V8RJLXhFnqj9oxQFqDE VfgTN93d2iD+VffESK9Bu3cFHKagPgJGlUEtVFJsb4AoWLUQGsrQcdc5q/pDyeCM8lLA YCCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=I4ghk0ykr10Y3ofFJtkABGhBT0o8tksw4duAtzlyuag=; b=qHlNVPYgjHHzvojEqIzqZoOpEJ47SIxdPGCxviPd3nIo2/VHpzdSyZzp39nheNGJCK Jr2huNnsNJoii2nYPuPKtnl9E2JJDyDVSn+as4QasicqRXJsVUKEaN2wQErbi123687k aXWujCWAXO/2OnEPmBStsG02c2jPKx3Us8QlmLHWsTiKuXY0TUvASfb8p+OKGyhiwM/q EKtdjVx1r3TWFeH5pwhko3Db91Uy8ukhOAoYJxFfmbmLvIp1bPVqQ7SxWD7UJr/PHGew kn81RPCIT5nnzlkduYiZit4PPTEMx8iiwEM/dRVmnLXMj5ee3dYp3JmUSa+sPAK1VwOs 48DQ== X-Gm-Message-State: ACrzQf268CQhLgdbffzPsvKC3kcdn7IInoBeMMf6KPf6+/dMRTFnVz2v ec4dWMOIvORRp5H5GOKLQ5jTqd/nMPhxDFBgEcmNbZ7rDcK7gQ== X-Google-Smtp-Source: AMsMyM6Zs33ybe+9ae0erCh4a8TIZXu8Rb3M+eNQ8pmcnz3QDfCrwObxhqzj/iiJQRuLvkF5kVzgfE7W4ItvVVqPDR8= X-Received: by 2002:a05:600c:1c84:b0:3b3:ef37:afd3 with SMTP id k4-20020a05600c1c8400b003b3ef37afd3mr6317887wms.155.1663778937623; Wed, 21 Sep 2022 09:48:57 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:48:21 -0700 Message-ID: Subject: [PATCH 08/17] riscv: Add task switch support for vector To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_094901_866707_B684261B X-CRM114-Status: GOOD ( 18.70 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This patch adds task switch support for vector. It supports partial lazy save and restore mechanism. It also supports all lengths of vlen. [guoren@linux.alibaba.com: First available porting to support vector context switching] [nick.knight@sifive.com: Rewrite vector.S to support dynamic vlen, xlen and code refine] [vincent.chen@sifive.com: Fix the might_sleep issue in vstate_save, vstate_restore] [andrew@sifive.com: Optimize task switch codes of vector] [ruinland.tsai@sifive.com: Fix the arch_release_task_struct free wrong datap issue] Suggested-by: Andrew Waterman Co-developed-by: Nick Knight Signed-off-by: Nick Knight Co-developed-by: Guo Ren Signed-off-by: Guo Ren Co-developed-by: Vincent Chen Signed-off-by: Vincent Chen Co-developed-by: Ruinland Tsai Signed-off-by: Ruinland Tsai Signed-off-by: Greentime Hu Reported-by: kernel test robot Reported-by: kernel test robot --- arch/riscv/include/asm/switch_to.h | 66 ++++++++++++++++++++++++++++++ arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/process.c | 43 +++++++++++++++++++ 3 files changed, 110 insertions(+) childregs->sp = usp; -- 2.25.1 diff --git a/arch/riscv/include/asm/switch_to.h b/arch/riscv/include/asm/switch_to.h index df1aa589b7fd..527951c033d4 100644 --- a/arch/riscv/include/asm/switch_to.h +++ b/arch/riscv/include/asm/switch_to.h @@ -7,11 +7,13 @@ #define _ASM_RISCV_SWITCH_TO_H #include +#include #include #include #include #include #include +#include #ifdef CONFIG_FPU extern void __fstate_save(struct task_struct *save_to); @@ -68,6 +70,68 @@ static __always_inline bool has_fpu(void) { return false; } #define __switch_to_fpu(__prev, __next) do { } while (0) #endif +#ifdef CONFIG_VECTOR +extern struct static_key_false cpu_hwcap_vector; +static __always_inline bool has_vector(void) +{ + return static_branch_likely(&cpu_hwcap_vector); +} +extern unsigned long riscv_vsize; +extern void __vstate_save(struct __riscv_v_state *save_to, void *datap); +extern void __vstate_restore(struct __riscv_v_state *restore_from, void *datap); + +static inline void __vstate_clean(struct pt_regs *regs) +{ + regs->status = (regs->status & ~(SR_VS)) | SR_VS_CLEAN; +} + +static inline void vstate_off(struct task_struct *task, + struct pt_regs *regs) +{ + regs->status = (regs->status & ~SR_VS) | SR_VS_OFF; +} + +static inline void vstate_save(struct task_struct *task, + struct pt_regs *regs) +{ + if ((regs->status & SR_VS) == SR_VS_DIRTY) { + struct __riscv_v_state *vstate = &(task->thread.vstate); + + __vstate_save(vstate, vstate->datap); + __vstate_clean(regs); + } +} + +static inline void vstate_restore(struct task_struct *task, + struct pt_regs *regs) +{ + if ((regs->status & SR_VS) != SR_VS_OFF) { + struct __riscv_v_state *vstate = &(task->thread.vstate); + + __vstate_restore(vstate, vstate->datap); + __vstate_clean(regs); + } +} + +static inline void __switch_to_vector(struct task_struct *prev, + struct task_struct *next) +{ + struct pt_regs *regs; + + regs = task_pt_regs(prev); + if (unlikely(regs->status & SR_SD)) + vstate_save(prev, regs); + vstate_restore(next, task_pt_regs(next)); +} + +#else +static __always_inline bool has_vector(void) { return false; } +#define riscv_vsize (0) +#define vstate_save(task, regs) do { } while (0) +#define vstate_restore(task, regs) do { } while (0) +#define __switch_to_vector(__prev, __next) do { } while (0) +#endif + extern struct task_struct *__switch_to(struct task_struct *, struct task_struct *); @@ -77,6 +141,8 @@ do { \ struct task_struct *__next = (next); \ if (has_fpu()) \ __switch_to_fpu(__prev, __next); \ + if (has_vector()) \ + __switch_to_vector(__prev, __next); \ ((last) = __switch_to(__prev, __next)); \ } while (0) diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 33bb60a354cd..35752fb6d145 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -55,6 +55,7 @@ obj-$(CONFIG_MMU) += vdso.o vdso/ obj-$(CONFIG_RISCV_M_MODE) += traps_misaligned.o obj-$(CONFIG_FPU) += fpu.o +obj-$(CONFIG_VECTOR) += vector.o obj-$(CONFIG_SMP) += smpboot.o obj-$(CONFIG_SMP) += smp.o obj-$(CONFIG_SMP) += cpu_ops.o diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c index ceb9ebab6558..e88a37fc77ed 100644 --- a/arch/riscv/kernel/process.c +++ b/arch/riscv/kernel/process.c @@ -124,6 +124,25 @@ void start_thread(struct pt_regs *regs, unsigned long pc, */ fstate_restore(current, regs); } + + if (has_vector()) { + struct __riscv_v_state *vstate = &(current->thread.vstate); + + /* Enable vector and allocate memory for vector registers. */ + if (!vstate->datap) { + vstate->datap = kzalloc(riscv_vsize, GFP_KERNEL); + if (WARN_ON(!vstate->datap)) + return; + } + regs->status |= SR_VS_INITIAL; + + /* + * Restore the initial value to the vector register + * before starting the user program. + */ + vstate_restore(current, regs); + } + regs->epc = pc; regs->sp = sp; @@ -148,15 +167,29 @@ void flush_thread(void) fstate_off(current, task_pt_regs(current)); memset(¤t->thread.fstate, 0, sizeof(current->thread.fstate)); #endif +#ifdef CONFIG_VECTOR + /* Reset vector state */ + vstate_off(current, task_pt_regs(current)); + memset(¤t->thread.vstate, 0, RISCV_V_STATE_DATAP); +#endif } int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) { fstate_save(src, task_pt_regs(src)); *dst = *src; + dst->thread.vstate.datap = NULL; + return 0; } +void arch_release_task_struct(struct task_struct *tsk) +{ + /* Free the vector context of datap. */ + if (has_vector() && tsk->thread.vstate.datap) + kfree(tsk->thread.vstate.datap); +} + int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) { unsigned long clone_flags = args->flags; @@ -175,7 +208,17 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) p->thread.ra = (unsigned long)ret_from_kernel_thread; p->thread.s[0] = (unsigned long)args->fn; p->thread.s[1] = (unsigned long)args->fn_arg; + p->thread.vstate.datap = NULL; } else { + /* Allocate the datap for the user process if datap is NULL */ + if (has_vector() && !p->thread.vstate.datap) { + void *datap = kzalloc(riscv_vsize, GFP_KERNEL); + /* Failed to allocate memory. */ + if (!datap) + return -ENOMEM; + p->thread.vstate.datap = datap; + memset(&p->thread.vstate, 0, RISCV_V_STATE_DATAP); + } *childregs = *(current_pt_regs()); if (usp) /* User fork */ From patchwork Wed Sep 21 16:48:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12983997 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AC63AECAAD8 for ; Wed, 21 Sep 2022 16:49:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=4HiqkRHWaIXYFi/h6+so7erk69hJVr/6FrjJH+sY0F0=; b=AYob1TTWABW9oL 0d8SYqXIepVmWWBhXgvRf1DOVlL541BbizDKvPLz8LcGLVWm9IMdE1udzLxKNYt5bZ5gOyk4vwMZ1 QE3XllVcCjzqZjZ32N7qvHTbPxfszwVIDvnZf/u6xy/K9Ew/lZBLFqfHYEbSOLgxnOSAJeP1yTYjh 8B++lpcU8sw2Nx/RvJDwGIDlMqbaZWU3EfM0YpsrAG8voUudXGcqwSqvKE0oIjSF2VwrhES0hVJL7 yqMta0E9Lj/ZIxk4f3d09MDmfT5eS2GoK7UABqV6aGF9fFXSI9Rfoj3mMWc8thXaRuQj1hvZgt1/w MUnDtGkq4cLpVbNV19gw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2ub-00C3PT-Nl; Wed, 21 Sep 2022 16:49:25 +0000 Received: from mail-wr1-x42d.google.com ([2a00:1450:4864:20::42d]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2uY-00C3Ju-BY for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:49:23 +0000 Received: by mail-wr1-x42d.google.com with SMTP id bq9so10901173wrb.4 for ; Wed, 21 Sep 2022 09:49:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=gqSubrlqj2HaC74aHL2k1nAHbEWi0zWIGWa6FjxCUv4=; b=1Oe5GFWBFxZ0m8tz7dwBUncAuUDhdLS9Cm/v+0waLu4MThjl+0JQLjCEyb+k+OesGH gvS2SpAK9cmJSsuC+aYp23/VJfLchg0mpgAZ4S5aAcN+/ly7ciIZ0Nlka+gn//DuQnY2 a0+DQvUwvj6x6SUx2Wcr4GvA+6MlloQImgVakX2I5PVXgug/eg2bQozbxcYf4wzRjqej Clj0h4s2018JUn+kIiDob6QSKyxmZJ+bknUejti/q0zDYc41iMze6moT6GBMMlM6gVQa yE2WicxUR/NwLmWbjw1QxZtKqp52bb7emGnTYlduX1DEUmyzSZ1sXPLYttftHnmnaYb2 GRXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=gqSubrlqj2HaC74aHL2k1nAHbEWi0zWIGWa6FjxCUv4=; b=OkyDSd8k/hUD4gYjk/iHCCAyVp9nkbNApz8J3z/uUKithG4Diav4LeFy9G/yJMNJ7Q FoejsfNiDXmyu37zIlP7RyAXRhKecebZ5+kQ/Wf8ItmoPP21rBql35RTDFx0oSF4Jtfm +8Z0+OZ7wawlxRYl5N0TtlFhrEEve/KylUJETELONcdksrCIDu0QtZOIir5+MnVCpJ4E 0WVg2D/Kd6TPpsFo9bCjrBz0+H3cMvfT+ONvqPmhW+gD2l+IjwuHj7VxZC2fuAKBLJi8 JRa8GLNQ9PybBSw9kuyeTY1KztagRoK5ZtbrlxESvcAVl05BXrG63rHovXTnXiiiDxa4 Hz6w== X-Gm-Message-State: ACrzQf1hhaN1P0oUykFJkcg89SPq1uBZUSW3SHZEzYZojxYT/KbxU9z+ zmhdCgxsb5zp9AVdGXMAoFYaC9DVBSFP+y6HVNKV2HRg98HBew== X-Google-Smtp-Source: AMsMyM4WgQdXcSPCxGSFXzzMMjBk07N2VN8k1uzFowO0xmdYYrt7lvPi7S7Q8JTKwSX1fSjk97NQdzS7lnUUC5jS3s4= X-Received: by 2002:a5d:588f:0:b0:22b:623:ad04 with SMTP id n15-20020a5d588f000000b0022b0623ad04mr8692524wrf.607.1663778954422; Wed, 21 Sep 2022 09:49:14 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:48:38 -0700 Message-ID: Subject: [PATCH 09/17] riscv: Add ptrace vector support To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_094922_418869_CDB135FE X-CRM114-Status: GOOD ( 19.20 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This patch adds ptrace support for riscv vector. The vector registers will be saved in datap pointer of __riscv_v_state. This pointer will be set right after the __riscv_v_state data structure then it will be put in ubuf for ptrace system call to get or set. It will check if the datap got from ubuf is set to the correct address or not when the ptrace system call is trying to set the vector registers. Co-developed-by: Vincent Chen Signed-off-by: Vincent Chen Signed-off-by: Greentime Hu --- arch/riscv/include/uapi/asm/ptrace.h | 6 +++ arch/riscv/kernel/ptrace.c | 71 ++++++++++++++++++++++++++++ include/uapi/linux/elf.h | 1 + 3 files changed, 78 insertions(+) #define NT_LOONGARCH_LSX 0xa02 /* LoongArch Loongson SIMD Extension registers */ -- 2.25.1 diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h index 6ee1ca2edfa7..2491875be80d 100644 --- a/arch/riscv/include/uapi/asm/ptrace.h +++ b/arch/riscv/include/uapi/asm/ptrace.h @@ -94,6 +94,12 @@ struct __riscv_v_state { */ }; +/* + * According to spec: The number of bits in a single vector register, + * VLEN >= ELEN, which must be a power of 2, and must be no greater than + * 2^16 = 65536bits = 8192bytes + */ +#define RISCV_MAX_VLENB (8192) #endif /* __ASSEMBLY__ */ #endif /* _UAPI_ASM_RISCV_PTRACE_H */ diff --git a/arch/riscv/kernel/ptrace.c b/arch/riscv/kernel/ptrace.c index 2ae8280ae475..cce459ff551d 100644 --- a/arch/riscv/kernel/ptrace.c +++ b/arch/riscv/kernel/ptrace.c @@ -27,6 +27,9 @@ enum riscv_regset { #ifdef CONFIG_FPU REGSET_F, #endif +#ifdef CONFIG_VECTOR + REGSET_V, +#endif }; static int riscv_gpr_get(struct task_struct *target, @@ -83,6 +86,64 @@ static int riscv_fpr_set(struct task_struct *target, } #endif +#ifdef CONFIG_VECTOR +static int riscv_vr_get(struct task_struct *target, + const struct user_regset *regset, + struct membuf to) +{ + struct __riscv_v_state *vstate = &target->thread.vstate; + + /* + * Ensure the vector registers have been saved to the memory before + * copying them to membuf. + */ + if (target == current) + vstate_save(current, task_pt_regs(current)); + + /* Copy vector header from vstate. */ + membuf_write(&to, vstate, RISCV_V_STATE_DATAP); + membuf_zero(&to, sizeof(void *)); +#if __riscv_xlen == 32 + membuf_zero(&to, sizeof(__u32)); +#endif + + /* Copy all the vector registers from vstate. */ + return membuf_write(&to, vstate->datap, riscv_vsize); +} + +static int riscv_vr_set(struct task_struct *target, + const struct user_regset *regset, + unsigned int pos, unsigned int count, + const void *kbuf, const void __user *ubuf) +{ + int ret, size; + struct __riscv_v_state *vstate = &target->thread.vstate; + + /* Copy rest of the vstate except datap and __padding. */ + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, vstate, 0, + RISCV_V_STATE_DATAP); + if (unlikely(ret)) + return ret; + + /* Skip copy datap. */ + size = sizeof(vstate->datap); + count -= size; + ubuf += size; +#if __riscv_xlen == 32 + /* Skip copy _padding. */ + size = sizeof(vstate->__padding); + count -= size; + ubuf += size; +#endif + + /* Copy all the vector registers. */ + pos = 0; + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, vstate->datap, + 0, riscv_vsize); + return ret; +} +#endif + static const struct user_regset riscv_user_regset[] = { [REGSET_X] = { .core_note_type = NT_PRSTATUS, @@ -102,6 +163,16 @@ static const struct user_regset riscv_user_regset[] = { .set = riscv_fpr_set, }, #endif +#ifdef CONFIG_VECTOR + [REGSET_V] = { + .core_note_type = NT_RISCV_VECTOR, + .align = 16, + .n = (32 * RISCV_MAX_VLENB)/sizeof(__u32), + .size = sizeof(__u32), + .regset_get = riscv_vr_get, + .set = riscv_vr_set, + }, +#endif }; static const struct user_regset_view riscv_user_native_view = { diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h index c7b056af9ef0..5a5056c6a2a1 100644 --- a/include/uapi/linux/elf.h +++ b/include/uapi/linux/elf.h @@ -439,6 +439,7 @@ typedef struct elf64_shdr { #define NT_MIPS_DSP 0x800 /* MIPS DSP ASE registers */ #define NT_MIPS_FP_MODE 0x801 /* MIPS floating-point mode */ #define NT_MIPS_MSA 0x802 /* MIPS SIMD registers */ +#define NT_RISCV_VECTOR 0x900 /* RISC-V vector registers */ #define NT_LOONGARCH_CPUCFG 0xa00 /* LoongArch CPU config registers */ #define NT_LOONGARCH_CSR 0xa01 /* LoongArch control and status registers */ From patchwork Wed Sep 21 16:48:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12983998 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6F20DC6FA82 for ; Wed, 21 Sep 2022 16:49:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=XvXfgqpteKS561cvU4tQ2897kCwibucMnvL8DpMOPUs=; b=hlHAcLr6fxgxoE XM26O6W1Gd1GUfhPPDUAJLsFyT2tHSaKHs6RWjQQ0t5qn0G6i3sIrKOYohEfEmpw0MmYbPfcDB2e3 1Tj0/OQWkfAzeJZxJQ7DLHYVvGGGyXYybdtOxzDypJ+souan5C0B52F/L7uA7eYtKwhHR18XsSWCZ z2Cd73ZwqASNpNXlttr8nnSRvm+jNiJHYw4mR4zo/RcMq2iuLHXfEv+YTH4PIQCWp2eY4ifxnw8Fa zLqyR0VmsRrM3aTEuRSkgVU0uMemwZzWjKh1HmTjVP7Vx/qAFnSYHwFQBNuE0MOJp9DimOyN00qS7 Euxh/uTZo2pDZiEXx/Jw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2un-00C3Vx-OO; Wed, 21 Sep 2022 16:49:37 +0000 Received: from mail-wr1-x432.google.com ([2a00:1450:4864:20::432]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2uk-00C3TL-F9 for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:49:36 +0000 Received: by mail-wr1-x432.google.com with SMTP id y5so10923663wrh.3 for ; Wed, 21 Sep 2022 09:49:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=4K1uDO9M+U3uhs0pS3WI7ZtfRXH39QR/ccbpGWue5u8=; b=ZYUL8hKVRVwZHczz+pjLeA7hUvH88E6L/P8A1Q7z9NkC+ONfAoz7OALb3GTRVAjxoy u3PexYABDFopnpyyOPT7HcC4tNHy1t5E5OPTlr8HDYzY77lkf46HQ9E/k03ptjweuHfw F302cByhgGuDF203jKkdcAxaSdhvqQi27GzbV1Cv/NAwj6pOMqHtI14tA6Fu5SCil7xQ VUFgm7kDOMyGCboajL7lVmGeWYmcxRVYgDrB3igGYsruDAIhRvQJ6Oq+UsXXpLSx2v8C N/gIJAnd3AAjXfZFYWSnweEhInpiawpydaCM9CCTZsmX1zcM/1gpaWTDkBo04ZVjJovL 0fHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=4K1uDO9M+U3uhs0pS3WI7ZtfRXH39QR/ccbpGWue5u8=; b=ooWK+k+zUpFNdejjObUG7u/Xm2jzjFVSL9mMHH/n/PBXMw0wdngJL0qiAF6+Zi5xL7 AvPgiMsHCtfi8zMc0D1WaWKAiodtMhqn8miFBz1RS5e+XELPOL7KYY4zSaHHjcDEabzn Pf5xJi2nu3MDRnYnOumwHZH3OqECMomoA9TepvpWWtZhTztfKNkor/fC2BLnxEsrDa+t QHx2+da51A+EFS83U2nYBznHgAifLQ5gZ/Wi7fWKJRlz5qhXopI5zlOAdED2xSy1n08w PqtwXmYAEHElvpbFtzPTBmblj+S2rhuOnOpKySGcYdMWNRD07VKtAfpiGJC1bM2VcFXo kXbA== X-Gm-Message-State: ACrzQf2IjA49rGGw73EHlvIL+LmwMkGLlGJ2qSJGMS3L17DLGbQbjfKQ unD2lvjTTCNeZGqpLc3OUgCei4Nn40fDZSQ5fTPE1lNPFNwZnQ== X-Google-Smtp-Source: AMsMyM43Zr2+QEQqDJmmUhXf9yZ2Xc2hLk+VzDJSYQG1MqhQiipkN2DV4Ec/8ahNl6zfUZV97mjqnhRqYjwe+aaKoww= X-Received: by 2002:a05:6000:1886:b0:22a:2944:a09 with SMTP id a6-20020a056000188600b0022a29440a09mr18104425wri.391.1663778971832; Wed, 21 Sep 2022 09:49:31 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:48:55 -0700 Message-ID: Subject: [PATCH 10/17] riscv: Add sigcontext save/restore for vector To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_094934_532466_4164CECF X-CRM114-Status: GOOD ( 22.46 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This patch adds sigcontext save/restore for vector. The vector registers will be saved in datap pointer. The datap pointer will be allocated dynamically when the task needs in kernel space. The datap pointer will be set right after the __riscv_v_state data structure to save all the vector registers in the signal handler stack. Co-developed-by: Vincent Chen Signed-off-by: Vincent Chen Signed-off-by: Greentime Hu --- arch/riscv/include/uapi/asm/sigcontext.h | 24 ++++ arch/riscv/kernel/asm-offsets.c | 2 + arch/riscv/kernel/signal.c | 165 ++++++++++++++++++++++- 3 files changed, 187 insertions(+), 4 deletions(-) } SYSCALL_DEFINE0(rt_sigreturn) @@ -103,13 +243,14 @@ SYSCALL_DEFINE0(rt_sigreturn) struct rt_sigframe __user *frame; struct task_struct *task; sigset_t set; + size_t frame_size = cal_rt_frame_size(); /* Always make any pending restarted system calls return -EINTR */ current->restart_block.fn = do_no_restart_syscall; frame = (struct rt_sigframe __user *)regs->sp; - if (!access_ok(frame, sizeof(*frame))) + if (!access_ok(frame, frame_size)) goto badframe; if (__copy_from_user(&set, &frame->uc.uc_sigmask, sizeof(set))) @@ -142,11 +283,20 @@ static long setup_sigcontext(struct rt_sigframe __user *frame, { struct sigcontext __user *sc = &frame->uc.uc_mcontext; long err; + void *sc_reserved_free_ptr = sc->__reserved; + /* sc_regs is structured the same as the start of pt_regs */ err = __copy_to_user(&sc->sc_regs, regs, sizeof(sc->sc_regs)); /* Save the floating-point state. */ if (has_fpu()) err |= save_fp_state(regs, &sc->sc_fpregs); + /* Save the vector state. */ + if (has_vector()) + err |= save_v_state(regs, &sc_reserved_free_ptr); + + /* Put END __riscv_ctx_hdr at the end. */ + err = __put_user(END_MAGIC, &((struct __riscv_ctx_hdr *)sc_reserved_free_ptr)->magic); + err = __put_user(END_HDR_SIZE, &((struct __riscv_ctx_hdr *)sc_reserved_free_ptr)->size); return err; } @@ -178,9 +328,10 @@ static int setup_rt_frame(struct ksignal *ksig, sigset_t *set, { struct rt_sigframe __user *frame; long err = 0; + size_t frame_size = cal_rt_frame_size(); - frame = get_sigframe(ksig, regs, sizeof(*frame)); - if (!access_ok(frame, sizeof(*frame))) + frame = get_sigframe(ksig, regs, frame_size); + if (!access_ok(frame, frame_size)) return -EFAULT; err |= copy_siginfo_to_user(&frame->info, &ksig->info); @@ -326,3 +477,9 @@ asmlinkage __visible void do_notify_resume(struct pt_regs *regs, if (thread_info_flags & _TIF_NOTIFY_RESUME) resume_user_mode_work(regs); } + +void init_rt_signal_env(void); +void __init init_rt_signal_env(void) +{ + rvv_sc_size = sizeof(struct __sc_riscv_v_state) + riscv_vsize; +} -- 2.25.1 diff --git a/arch/riscv/include/uapi/asm/sigcontext.h b/arch/riscv/include/uapi/asm/sigcontext.h index 84f2dfcfdbce..b8a0fd7d7cfc 100644 --- a/arch/riscv/include/uapi/asm/sigcontext.h +++ b/arch/riscv/include/uapi/asm/sigcontext.h @@ -8,6 +8,23 @@ #include +/* The Magic number for signal context frame header. */ +#define RVV_MAGIC 0x53465457 +#define END_MAGIC 0x0 + +/* The size of END signal context header. */ +#define END_HDR_SIZE 0x0 + +struct __riscv_ctx_hdr { + __u32 magic; + __u32 size; +}; + +struct __sc_riscv_v_state { + struct __riscv_ctx_hdr head; + struct __riscv_v_state v_state; +} __attribute__((aligned(16))); + /* * Signal context structure * @@ -17,6 +34,13 @@ struct sigcontext { struct user_regs_struct sc_regs; union __riscv_fp_state sc_fpregs; + /* + * 4K + 128 reserved for vector state and future expansion. + * This space is enough to store the vector context whose VLENB + * is less or equal to 128. + * (The size of the vector context is 4144 byte as VLENB is 128) + */ + __u8 __reserved[4224] __attribute__((__aligned__(16))); }; #endif /* _UAPI_ASM_RISCV_SIGCONTEXT_H */ diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c index 37e3e6a8d877..80316ef7bb78 100644 --- a/arch/riscv/kernel/asm-offsets.c +++ b/arch/riscv/kernel/asm-offsets.c @@ -75,6 +75,8 @@ void asm_offsets(void) OFFSET(TSK_STACK_CANARY, task_struct, stack_canary); #endif + OFFSET(RISCV_V_STATE_MAGIC, __riscv_ctx_hdr, magic); + OFFSET(RISCV_V_STATE_SIZE, __riscv_ctx_hdr, size); OFFSET(RISCV_V_STATE_VSTART, __riscv_v_state, vstart); OFFSET(RISCV_V_STATE_VL, __riscv_v_state, vl); OFFSET(RISCV_V_STATE_VTYPE, __riscv_v_state, vtype); diff --git a/arch/riscv/kernel/signal.c b/arch/riscv/kernel/signal.c index 38b05ca6fe66..41d9a02c7098 100644 --- a/arch/riscv/kernel/signal.c +++ b/arch/riscv/kernel/signal.c @@ -20,15 +20,16 @@ #include extern u32 __user_rt_sigreturn[2]; +static size_t rvv_sc_size; #define DEBUG_SIG 0 struct rt_sigframe { struct siginfo info; - struct ucontext uc; #ifndef CONFIG_MMU u32 sigreturn_code[2]; #endif + struct ucontext uc; }; #ifdef CONFIG_FPU @@ -85,16 +86,155 @@ static long save_fp_state(struct pt_regs *regs, #define restore_fp_state(task, regs) (0) #endif +#ifdef CONFIG_VECTOR +static long restore_v_state(struct pt_regs *regs, void **sc_reserved_ptr) +{ + long err; + struct __sc_riscv_v_state __user *state = (struct __sc_riscv_v_state *)(*sc_reserved_ptr); + void *datap; + __u32 magic; + __u32 size; + + /* Get magic number and check it. */ + err = __get_user(magic, &state->head.magic); + err = __get_user(size, &state->head.size); + if (unlikely(err)) + return err; + + if (magic != RVV_MAGIC || size != rvv_sc_size) + return -EINVAL; + + /* Copy everything of __sc_riscv_v_state except datap. */ + err = __copy_from_user(¤t->thread.vstate, &state->v_state, + RISCV_V_STATE_DATAP); + if (unlikely(err)) + return err; + + /* Copy the pointer datap itself. */ + err = __get_user(datap, &state->v_state.datap); + if (unlikely(err)) + return err; + + + /* Copy the whole vector content from user space datap. */ + err = __copy_from_user(current->thread.vstate.datap, datap, riscv_vsize); + if (unlikely(err)) + return err; + + vstate_restore(current, regs); + + /* Move sc_reserved_ptr to point the next signal context frame. */ + *sc_reserved_ptr += size; + + return err; +} + +static long save_v_state(struct pt_regs *regs, void **sc_reserved_free_ptr) +{ + /* + * Put __sc_riscv_v_state to the user's signal context space pointed + * by sc_reserved_free_ptr and the datap point the address right + * after __sc_riscv_v_state. + */ + struct __sc_riscv_v_state __user *state = (struct __sc_riscv_v_state *) + (*sc_reserved_free_ptr); + void *datap = state + 1; + long err; + + *sc_reserved_free_ptr += rvv_sc_size; + + err = __put_user(RVV_MAGIC, &state->head.magic); + err = __put_user(rvv_sc_size, &state->head.size); + + vstate_save(current, regs); + /* Copy everything of vstate but datap. */ + err = __copy_to_user(&state->v_state, ¤t->thread.vstate, + RISCV_V_STATE_DATAP); + if (unlikely(err)) + return err; + + /* Copy the pointer datap itself. */ + err = __put_user(datap, &state->v_state.datap); + if (unlikely(err)) + return err; + + /* Copy the whole vector content to user space datap. */ + err = __copy_to_user(datap, current->thread.vstate.datap, riscv_vsize); + + return err; +} +#else +#define save_v_state(task, regs) (0) +#define restore_v_state(task, regs) (0) +#endif + static long restore_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc) { long err; + void *sc_reserved_ptr = sc->__reserved; /* sc_regs is structured the same as the start of pt_regs */ err = __copy_from_user(regs, &sc->sc_regs, sizeof(sc->sc_regs)); /* Restore the floating-point state. */ if (has_fpu()) err |= restore_fp_state(regs, &sc->sc_fpregs); + + while (1 && !err) { + __u32 magic, size; + struct __riscv_ctx_hdr *head = (struct __riscv_ctx_hdr *)sc_reserved_ptr; + + err |= __get_user(magic, &head->magic); + err |= __get_user(size, &head->size); + if (err) + goto done; + + switch (magic) { + case 0: + if (size) + goto invalid; + goto done; + case RVV_MAGIC: + if (!has_vector()) + goto invalid; + if (size != rvv_sc_size) + goto invalid; + err |= restore_v_state(regs, &sc_reserved_ptr); + break; + default: + goto invalid; + } + } +done: return err; + +invalid: + return -EINVAL; +} + +static size_t cal_rt_frame_size(void) +{ + struct rt_sigframe __user *frame; + static size_t frame_size; + size_t total_context_size = 0; + size_t sc_reserved_size = sizeof(frame->uc.uc_mcontext.__reserved); + + if (frame_size) + goto done; + + frame_size = sizeof(*frame); + + if (has_vector()) + total_context_size += rvv_sc_size; + /* Preserved a __riscv_ctx_hdr for END signal context header. */ + total_context_size += sizeof(struct __riscv_ctx_hdr); + + if (total_context_size > sc_reserved_size) + frame_size += (total_context_size - sc_reserved_size); + + frame_size = round_up(frame_size, 16); +done: + return frame_size; + From patchwork Wed Sep 21 16:49:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12983999 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2E19AC6FA82 for ; Wed, 21 Sep 2022 16:50:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=X1mV7ADFZDEziNQmZsvQmYtjysILKeY80V2+GC67v7E=; b=I+4eftMf4Li5ZF IE5aNT2CmFHYvIYLLPL0hM8f3XXEh/S9EAYg1F2WiHVshxUg+RQGxrOh5E/k2BA+i4TgfJ54Wg6t/ ZyoCD/T9xjXNfJaGISPMQ7eyS7noj6fIsvOKwql5HkaUmOaJeUbsjFpVfikLuu/GvRXYC5NZSLc9Z p18Hc2n0dwiSGBjRXvNwv48xhFXdd8uha6175XXVI6gIj2BEoYclaA3M5Q84jIEdsHBCPKcJCNBnQ X8OqA5yjSH59pzWx9Yu+aIcNeiepQRpIGF4VwSv9XFOv/lLSyMz6TjhvHP5kpe+OMLn4TMydM4aEr vWLyEq29l7stJLJu6Daw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2v3-00C3e1-Le; Wed, 21 Sep 2022 16:49:53 +0000 Received: from mail-wm1-x329.google.com ([2a00:1450:4864:20::329]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2v0-00C3bD-RC for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:49:52 +0000 Received: by mail-wm1-x329.google.com with SMTP id h130-20020a1c2188000000b003b4cadadce7so8204047wmh.3 for ; Wed, 21 Sep 2022 09:49:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=RdNxjUT1Iwihmma6pljxet97EzfZHaVqcKftodv4caY=; b=RPEVVhA/vRIRNOuEYwYHgjEC3qLjcfP3x5evz6XTjIwL/kARoV3BYbhBoXPNSkgW1D 9AnWxEkIHVWFNv5PXJigpZE+t/wSiahAC7dZEYv8AZD+DUbWfBRJ17FxpOO7lIbzuBVh td/HV2Fon1RJqla08K/IcE2LjJdaB4jZUegnLksMM0apQ8mV4PexaM/XTfsPk0h4VWCd IS+ifakg8nDnldGzWTnLECMgjfsOuXluwdtZ3DPBEGrJsWN34Lrl75OJZj0Zce1nHQMZ qP7hPWEIlW/OGc4lJ1oQNRie0J/bdvDDirlmjJTvo+i+lPhfE2N9NtZK1V2gWVd4RVwr seJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=RdNxjUT1Iwihmma6pljxet97EzfZHaVqcKftodv4caY=; b=gbzhppS+nSwrvJ/twh1VUwKJXF2LFskn7aJjpE1Bcamu641Ysp7MWBws+yLLvLNjXO R/ACfN7yLqlLBB5ESFg3zXExJu4s7d8ZJ8kyrPPBfVwGqxOS+i1rZEvxs3GUsJ7CG9Sw 3kPIXbITDtix8mM08yr1yTsvljQvvrkCct2QkJVJfwk5j7VTritj5mEJVWL09BGX+1Cu 7g9vKFD79WhDp3OCj+G8+RRV7BHpLSw2nuluCDxn2kKjId4C23a6HCMzlA/h1yWUrAj/ PPZo1/636vk159v9Xi/weykF3bsmpPhJywpbi2vFaqT67CPmPIAUiTWBLS90M9nx7oOO Tx9A== X-Gm-Message-State: ACrzQf3sPY2D0SACwcsG5Qx7U1T20eVFx13zdqyRoy335KFe8cwft6/G aIzjB/2iY6XFeMIdtkYB6+StTwfSEqKSgwPpT+LfcYZcUrjuLQ== X-Google-Smtp-Source: AMsMyM4VIHs4G1awsBx+OjC3tOW2mzsVWUxHEA2MjLiBs4IVPBU8qtwH1dZHI5OtZi13U/FKkApa/rt9jUW4WG/zh7c= X-Received: by 2002:a05:600c:410d:b0:3b4:9454:f894 with SMTP id j13-20020a05600c410d00b003b49454f894mr6439149wmi.111.1663778988316; Wed, 21 Sep 2022 09:49:48 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:49:12 -0700 Message-ID: Subject: [PATCH 11/17] riscv: signal: Report signal frame size to userspace via auxv To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_094950_903241_156850F4 X-CRM114-Status: GOOD ( 15.05 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org The vector register belongs to the signal context. They need to be stored and restored as entering and leaving the signal handler. According to the V-extension specification, the maximum length of the vector registers can be 2^(XLEN-1). Hence, if userspace refers to the MINSIGSTKSZ to create a sigframe, it may not be enough. To resolve this problem, this patch refers to the commit 94b07c1f8c39c ("arm64: signal: Report signal frame size to userspace via auxv") to enable userspace to know the minimum required sigframe size through the auxiliary vector and use it to allocate enough memory for signal context. Signed-off-by: Greentime Hu Signed-off-by: Vincent Chen --- arch/riscv/include/asm/elf.h | 47 +++++++++++++++------------- arch/riscv/include/asm/processor.h | 2 ++ arch/riscv/include/uapi/asm/auxvec.h | 1 + arch/riscv/kernel/signal.c | 8 +++++ 4 files changed, 36 insertions(+), 22 deletions(-) + signal_minsigstksz = cal_rt_frame_size(); } -- 2.25.1 diff --git a/arch/riscv/include/asm/elf.h b/arch/riscv/include/asm/elf.h index 14fc7342490b..df9047b619e8 100644 --- a/arch/riscv/include/asm/elf.h +++ b/arch/riscv/include/asm/elf.h @@ -77,28 +77,31 @@ extern unsigned long elf_hwcap; #define COMPAT_ELF_PLATFORM (NULL) #ifdef CONFIG_MMU -#define ARCH_DLINFO \ -do { \ - /* \ - * Note that we add ulong after elf_addr_t because \ - * casting current->mm->context.vdso triggers a cast \ - * warning of cast from pointer to integer for \ - * COMPAT ELFCLASS32. \ - */ \ - NEW_AUX_ENT(AT_SYSINFO_EHDR, \ - (elf_addr_t)(ulong)current->mm->context.vdso); \ - NEW_AUX_ENT(AT_L1I_CACHESIZE, \ - get_cache_size(1, CACHE_TYPE_INST)); \ - NEW_AUX_ENT(AT_L1I_CACHEGEOMETRY, \ - get_cache_geometry(1, CACHE_TYPE_INST)); \ - NEW_AUX_ENT(AT_L1D_CACHESIZE, \ - get_cache_size(1, CACHE_TYPE_DATA)); \ - NEW_AUX_ENT(AT_L1D_CACHEGEOMETRY, \ - get_cache_geometry(1, CACHE_TYPE_DATA)); \ - NEW_AUX_ENT(AT_L2_CACHESIZE, \ - get_cache_size(2, CACHE_TYPE_UNIFIED)); \ - NEW_AUX_ENT(AT_L2_CACHEGEOMETRY, \ - get_cache_geometry(2, CACHE_TYPE_UNIFIED)); \ +#define ARCH_DLINFO \ +do { \ + NEW_AUX_ENT(AT_SYSINFO_EHDR, \ + (elf_addr_t)current->mm->context.vdso); \ + NEW_AUX_ENT(AT_L1I_CACHESIZE, \ + get_cache_size(1, CACHE_TYPE_INST)); \ + NEW_AUX_ENT(AT_L1I_CACHEGEOMETRY, \ + get_cache_geometry(1, CACHE_TYPE_INST)); \ + NEW_AUX_ENT(AT_L1D_CACHESIZE, \ + get_cache_size(1, CACHE_TYPE_DATA)); \ + NEW_AUX_ENT(AT_L1D_CACHEGEOMETRY, \ + get_cache_geometry(1, CACHE_TYPE_DATA)); \ + NEW_AUX_ENT(AT_L2_CACHESIZE, \ + get_cache_size(2, CACHE_TYPE_UNIFIED)); \ + NEW_AUX_ENT(AT_L2_CACHEGEOMETRY, \ + get_cache_geometry(2, CACHE_TYPE_UNIFIED)); \ + /* \ + * Should always be nonzero unless there's a kernel bug. \ + * If we haven't determined a sensible value to give to \ + * userspace, omit the entry: \ + */ \ + if (likely(signal_minsigstksz)) \ + NEW_AUX_ENT(AT_MINSIGSTKSZ, signal_minsigstksz); \ + else \ + NEW_AUX_ENT(AT_IGNORE, 0); \ } while (0) #define ARCH_HAS_SETUP_ADDITIONAL_PAGES struct linux_binprm; diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h index 95917a2b24f9..a09141ecf6aa 100644 --- a/arch/riscv/include/asm/processor.h +++ b/arch/riscv/include/asm/processor.h @@ -7,6 +7,7 @@ #define _ASM_RISCV_PROCESSOR_H #include +#include #include @@ -86,6 +87,7 @@ int riscv_of_parent_hartid(struct device_node *node, unsigned long *hartid); extern void riscv_fill_hwcap(void); extern int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src); +extern unsigned long signal_minsigstksz __ro_after_init; #endif /* __ASSEMBLY__ */ #endif /* _ASM_RISCV_PROCESSOR_H */ diff --git a/arch/riscv/include/uapi/asm/auxvec.h b/arch/riscv/include/uapi/asm/auxvec.h index 32c73ba1d531..6610d24e6662 100644 --- a/arch/riscv/include/uapi/asm/auxvec.h +++ b/arch/riscv/include/uapi/asm/auxvec.h @@ -33,5 +33,6 @@ /* entries in ARCH_DLINFO */ #define AT_VECTOR_SIZE_ARCH 7 +#define AT_MINSIGSTKSZ 51 #endif /* _UAPI_ASM_RISCV_AUXVEC_H */ diff --git a/arch/riscv/kernel/signal.c b/arch/riscv/kernel/signal.c index 41d9a02c7098..fb1f5012416e 100644 --- a/arch/riscv/kernel/signal.c +++ b/arch/riscv/kernel/signal.c @@ -478,8 +478,16 @@ asmlinkage __visible void do_notify_resume(struct pt_regs *regs, resume_user_mode_work(regs); } +unsigned long __ro_after_init signal_minsigstksz; + void init_rt_signal_env(void); void __init init_rt_signal_env(void) { rvv_sc_size = sizeof(struct __sc_riscv_v_state) + riscv_vsize; + /* + * Determine the stack space required for guaranteed signal delivery. + * The signal_minsigstksz will be populated into the AT_MINSIGSTKSZ entry + * in the auxiliary array at process startup. + */ From patchwork Wed Sep 21 16:49:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12984000 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D792AC6FA82 for ; Wed, 21 Sep 2022 16:50:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=VMhUOKsINOso+S65AmwWxy6s+kKQn88/Mmx3AE2wWes=; b=EiBvwlca9A0jZp SqUskYZH3sR1o60jzs0PReup/70TbTYdtLUu50iXrVjxaJOyhqMFShNWglI01MgJGGbQMaBsaQf4F t4vS7FiW7N/YZEtmcsSm4nmgvmd5RebYKEChpXyFXICp2lBU5G/QN0ky6t3RX2lb6OKAJIBVuzCLM +AkAXtCRSr1DRQLRfjvVrIZMfWNM6rXi3YycwkDc0WJuHdhPFZqWfyQCLGrHzoQSx1U6dE1+tRLW7 P7MEA1mZlGBf6+fHYvkJg9PlGxfBy1ke7s0EXPaTNfldvpnlCKXrpmjb/311PyUxodgIzhCuZFFXr RHaMqTDYk7iQE9gWjqrA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2vL-00C3oQ-1c; Wed, 21 Sep 2022 16:50:11 +0000 Received: from mail-wr1-x433.google.com ([2a00:1450:4864:20::433]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2vH-00C3lg-IP for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:50:09 +0000 Received: by mail-wr1-x433.google.com with SMTP id bq9so10906183wrb.4 for ; Wed, 21 Sep 2022 09:50:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=el0T6aQTw3mwJKznCQ4/AJLb+bj7u+7hEjPO5x8RmzQ=; b=sVZi9+PmVQf9wKvYLoPvf63PkGmcMBM4+1sbSRUQveKD/HGtRI0GjOyE1hucL4WPRi zeJ5vZTIzUlAOJiHZHflS6RojRu7UVrZzmWiYjGmrCSKDg/R64Z1jVYz/OuU/Kb+en2s GEgaRi/lMpUNbEVamyOFBqB4hzGl8TYaIqYaLQi1q6pwpd20shh+81kCogWC3ZmmL/ik qLvSwzF+FWNBynlXPn71bLhhGkoDn3yuCmRk4eqze+XqsTGf6Dxr6/EqajrGfYaJ8IFc /YRgeEH9DzNu2UhkidoZm1o9VHp+ZSN5BRMjNFduDYLNzR1Gy+2KATPGulMLpyclsbiM GKlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=el0T6aQTw3mwJKznCQ4/AJLb+bj7u+7hEjPO5x8RmzQ=; b=hff+CDXCNk7NXRlXg3JUdp+rHg+EGpHFCdC+SlQ01SUpMUviIs5UlKXF6bHxtLVH3e uOv0ZNLiaoxsQofflFcMGy38ugRBPK3b6JI8zUs9OBRNi9NaacRy1HRxho/MjQ6DcOxx IL+Kuac6l2SRr4NgTG1zXGdqDf66Pp5QidXOXdWCTjLlN+b06X8p7vUnXv4khRDzAEQE eRXsQebbXEQExQPcpwzjyhDWqLUV1/d4UGCUAWih5BprEHxrnDfcp1gBYe4cXippPOg2 OBiJpMkuGaWQtVQUiMHwg07Dg0WVNB/vbWvTs8BHeRT1ItDnqJUjzj66plXKKMu0gQ1a yMnw== X-Gm-Message-State: ACrzQf36e1SmYHgM3IWhnr41/2pIDc43ZPlA5apZ+MeGeEw426+sH4wo nFGlaw6WO+lpuXmviwrg2Y1XuS+vwEuiQkklhnyWVIehjIKEoQ== X-Google-Smtp-Source: AMsMyM5JlZDR6cqlQzpRsTquLjatg0nJestYBAde/LLcXLS4vR90fylQZYQjpbLJTDteIu2T77KHmQmj4Rtvf9rXaHM= X-Received: by 2002:a05:6000:1886:b0:22a:2944:a09 with SMTP id a6-20020a056000188600b0022a29440a09mr18105762wri.391.1663779004914; Wed, 21 Sep 2022 09:50:04 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:49:28 -0700 Message-ID: Subject: [PATCH 12/17] riscv: Add support for kernel mode vector To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_095007_632021_8E4947E0 X-CRM114-Status: GOOD ( 18.56 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Add kernel_rvv_begin() and kernel_rvv_end() function declarations and corresponding definitions in kernel_mode_vector.c These are needed to wrap uses of vector in kernel mode. Co-developed-by: Vincent Chen Signed-off-by: Vincent Chen Signed-off-by: Greentime Hu --- arch/riscv/include/asm/vector.h | 3 + arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/kernel_mode_vector.c | 132 +++++++++++++++++++++++++ arch/riscv/kernel/vector.S | 9 ++ 4 files changed, 145 insertions(+) create mode 100644 arch/riscv/kernel/kernel_mode_vector.c -- 2.25.1 diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index 16304b0c6a6f..a59841cc81fb 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -10,5 +10,8 @@ void rvv_enable(void); void rvv_disable(void); +void kernel_rvv_begin(void); +void kernel_rvv_end(void); +void vector_flush_cpu_state(void); #endif /* ! __ASM_RISCV_VECTOR_H */ diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 35752fb6d145..8c238415f800 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -56,6 +56,7 @@ obj-$(CONFIG_MMU) += vdso.o vdso/ obj-$(CONFIG_RISCV_M_MODE) += traps_misaligned.o obj-$(CONFIG_FPU) += fpu.o obj-$(CONFIG_VECTOR) += vector.o +obj-$(CONFIG_VECTOR) += kernel_mode_vector.o obj-$(CONFIG_SMP) += smpboot.o obj-$(CONFIG_SMP) += smp.o obj-$(CONFIG_SMP) += cpu_ops.o diff --git a/arch/riscv/kernel/kernel_mode_vector.c b/arch/riscv/kernel/kernel_mode_vector.c new file mode 100644 index 000000000000..0277168af0c5 --- /dev/null +++ b/arch/riscv/kernel/kernel_mode_vector.c @@ -0,0 +1,132 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2012 ARM Ltd. + * Author: Catalin Marinas + * Copyright (C) 2017 Linaro Ltd. + * Copyright (C) 2021 SiFive + */ +#include +#include +#include +#include +#include + +#include +#include + +DECLARE_PER_CPU(bool, vector_context_busy); +DEFINE_PER_CPU(bool, vector_context_busy); + +/* + * may_use_vector - whether it is allowable at this time to issue vector + * instructions or access the vector register file + * + * Callers must not assume that the result remains true beyond the next + * preempt_enable() or return from softirq context. + */ +static __must_check inline bool may_use_vector(void) +{ + /* + * vector_context_busy is only set while preemption is disabled, + * and is clear whenever preemption is enabled. Since + * this_cpu_read() is atomic w.r.t. preemption, vector_context_busy + * cannot change under our feet -- if it's set we cannot be + * migrated, and if it's clear we cannot be migrated to a CPU + * where it is set. + */ + return !in_irq() && !irqs_disabled() && !in_nmi() && + !this_cpu_read(vector_context_busy); +} + +/* + * Claim ownership of the CPU vector context for use by the calling context. + * + * The caller may freely manipulate the vector context metadata until + * put_cpu_vector_context() is called. + */ +static void get_cpu_vector_context(void) +{ + bool busy; + + preempt_disable(); + busy = __this_cpu_xchg(vector_context_busy, true); + + WARN_ON(busy); +} + +/* + * Release the CPU vector context. + * + * Must be called from a context in which get_cpu_vector_context() was + * previously called, with no call to put_cpu_vector_context() in the + * meantime. + */ +static void put_cpu_vector_context(void) +{ + bool busy = __this_cpu_xchg(vector_context_busy, false); + + WARN_ON(!busy); + preempt_enable(); +} + +/* + * kernel_rvv_begin(): obtain the CPU vector registers for use by the calling + * context + * + * Must not be called unless may_use_vector() returns true. + * Task context in the vector registers is saved back to memory as necessary. + * + * A matching call to kernel_rvv_end() must be made before returning from the + * calling context. + * + * The caller may freely use the vector registers until kernel_rvv_end() is + * called. + */ +void kernel_rvv_begin(void) +{ + if (WARN_ON(!has_vector())) + return; + + WARN_ON(!may_use_vector()); + + /* Acquire kernel mode vector */ + get_cpu_vector_context(); + + /* Save vector state, if any */ + vstate_save(current, task_pt_regs(current)); + + /* Enable vector */ + rvv_enable(); + + /* Invalidate vector regs */ + vector_flush_cpu_state(); +} +EXPORT_SYMBOL_GPL(kernel_rvv_begin); + +/* + * kernel_rvv_end(): give the CPU vector registers back to the current task + * + * Must be called from a context in which kernel_rvv_begin() was previously + * called, with no call to kernel_rvv_end() in the meantime. + * + * The caller must not use the vector registers after this function is called, + * unless kernel_rvv_begin() is called again in the meantime. + */ +void kernel_rvv_end(void) +{ + if (WARN_ON(!has_vector())) + return; + + /* Invalidate vector regs */ + vector_flush_cpu_state(); + + /* Restore vector state, if any */ + vstate_restore(current, task_pt_regs(current)); + + /* disable vector */ + rvv_disable(); + + /* release kernel mode vector */ + put_cpu_vector_context(); +} +EXPORT_SYMBOL_GPL(kernel_rvv_end); diff --git a/arch/riscv/kernel/vector.S b/arch/riscv/kernel/vector.S index 9f7dc70c4443..9c2de823c0d9 100644 --- a/arch/riscv/kernel/vector.S +++ b/arch/riscv/kernel/vector.S @@ -91,3 +91,12 @@ ENTRY(rvv_disable) csrc CSR_STATUS, status ret ENDPROC(rvv_disable) + +ENTRY(vector_flush_cpu_state) + vsetvli t0, x0, e8, m8, ta, ma + vmv.v.i v0, 0 + vmv.v.i v8, 0 + vmv.v.i v16, 0 + vmv.v.i v24, 0 + ret +ENDPROC(vector_flush_cpu_state) From patchwork Wed Sep 21 16:49:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12984001 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 60B1CC6FA82 for ; Wed, 21 Sep 2022 16:50:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=MujNX26FxUK9B199zWabMUiSzWDzBYr3u7q5PvMCOMc=; b=XXr0DhvJB2BSqD 8MG8k2OezzB69oOCSX4VWldF7/noZaZlmKUq5PePYypfDS7x4auwD8ARVZdRE9S/ASt7eqtHFbXjw GCiMf1BCTlphiLdnlLL2zMsUtN0r4axfF6+eeVfEQwoi5i6F6X8rAUsUzgpgnn4flhWxXhkFyi0yI 4IXl6vXWaPUfZn3ud+ayzqY+9OC8eHLo2eGESy0qu3eDV1nz+ALP/MLbZQXE0m58vEJxqIHwOU4gG s6Z6bXqj69AlWDF2HnzRYDG43ZVhQqSbZ2zIV8+cBxMCbncGZnhiG5k5SLG5OQKl3q4irILha3D8L xZ8dkkCAiSC9e8FIoLoA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2va-00C3wg-HZ; Wed, 21 Sep 2022 16:50:26 +0000 Received: from mail-wr1-x431.google.com ([2a00:1450:4864:20::431]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2vX-00C3uG-6d for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:50:24 +0000 Received: by mail-wr1-x431.google.com with SMTP id x18so4867762wrm.7 for ; Wed, 21 Sep 2022 09:50:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=R7FucNEvBYhx+py0T15xv+ntnu+xHn1Mgi3filkb55I=; b=Uh4TSWA7tCmxfRa440DFTfSj+dNvJmSB7UnK7rOE2vbhT9I8VlGvHvnM40vZWeMjjF EL9r5Xc2TDxq2Bu+Ighif08jv6DLLqKKC9ch7TZQ5fP/yG9bSzOVxWI0wkOWArdqdy2E McHemt69NFLkjJAFdXdpkOJ3a3VDy1+9GWBSBChO+Cl1oSoerSsP9hwlHMBBhpfp3htt 8DCCNB6GvMkHKvnRPTDHqxtHCvvZ+nc6BztDQS/LzKzLiMd94DYOfdFy0BPvu0Vh5lZB s9F/gQTcFQGw+y2+9NBU18gZFqaNlYhmjvn6lZ1o0/gtpnaIHkuSsmBQUON0CtaZD1Zh mg5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=R7FucNEvBYhx+py0T15xv+ntnu+xHn1Mgi3filkb55I=; b=JrceMKtpyP7JWLB8WfEGAqKLvfoA1UFQI0wMx+Pve+b2cVBSjlX9O/4dQqibQYDk3A zN18BBVjEzeS7yeV4U4XM0IRI98VTm0P3EZkMR1Ewq7xvvk1UMV9SwAeGHcf3N60FkTb oIG/9ydbDJ2TRXh9PHtgjI2iP2mnpLRmwprJ6kB0qpmAMRDUUMjZwyfpD9DZZZYCHSGu yFdTnorHVvexcXE7l/jAIjKvLL/904vbIW20Cpch177JzwN7neOmNw8f1zOAUbnv/rPD Wt+8m0Jiu+pkNSJiu24egwi8CKwBuA6p0x9LvHhAnjsutv3B8eoT9KR/ARijDjfEw6JC fDGw== X-Gm-Message-State: ACrzQf0ow78fGctKWaN03snO/8HkNNjetfNIXB8qUd82mWdazL6Mp8Gx /LlV+kvffujcgQXTqdFBxivANFvulNKfTG+kbKGGfcGkrhc8lw== X-Google-Smtp-Source: AMsMyM5/TMXrUrr1Ond+VacueK7HVCeGvk6wGIb5xyA4DGlH7ZUvIyT+K+VvOEmnn9qDcrqvr4oPDXCW3T0jCsqPAWc= X-Received: by 2002:adf:fb8f:0:b0:225:2def:221e with SMTP id a15-20020adffb8f000000b002252def221emr17858412wrr.130.1663779020529; Wed, 21 Sep 2022 09:50:20 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:49:44 -0700 Message-ID: Subject: [PATCH 13/17] riscv: Add vector extension XOR implementation To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_095023_263752_848E9840 X-CRM114-Status: GOOD ( 14.44 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This patch adds support for vector optimized XOR and it is tested in qemu. Co-developed-by: Han-Kuan Chen Signed-off-by: Han-Kuan Chen Signed-off-by: Greentime Hu --- arch/riscv/include/asm/xor.h | 82 ++++++++++++++++++++++++++++++++++++ arch/riscv/lib/Makefile | 1 + arch/riscv/lib/xor.S | 81 +++++++++++++++++++++++++++++++++++ 3 files changed, 164 insertions(+) create mode 100644 arch/riscv/include/asm/xor.h create mode 100644 arch/riscv/lib/xor.S -- 2.25.1 diff --git a/arch/riscv/include/asm/xor.h b/arch/riscv/include/asm/xor.h new file mode 100644 index 000000000000..d1f2eeb14afb --- /dev/null +++ b/arch/riscv/include/asm/xor.h @@ -0,0 +1,82 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2021 SiFive + */ + +#include +#include +#ifdef CONFIG_VECTOR +#include +#include + +void xor_regs_2_(unsigned long bytes, unsigned long * __restrict p1, + const unsigned long * __restrict p2); +void xor_regs_3_(unsigned long bytes, unsigned long * __restrict p1, + const unsigned long * __restrict p2, + const unsigned long * __restrict p3); +void xor_regs_4_(unsigned long bytes, unsigned long * __restrict p1, + const unsigned long * __restrict p2, + const unsigned long * __restrict p3, + const unsigned long * __restrict p4); +void xor_regs_5_(unsigned long bytes, unsigned long * __restrict p1, + const unsigned long * __restrict p2, + const unsigned long * __restrict p3, + const unsigned long * __restrict p4, + const unsigned long * __restrict p5); + +static void xor_rvv_2(unsigned long bytes, unsigned long * __restrict p1, + const unsigned long * __restrict p2) +{ + kernel_rvv_begin(); + xor_regs_2_(bytes, p1, p2); + kernel_rvv_end(); +} + +static void xor_rvv_3(unsigned long bytes, unsigned long * __restrict p1, + const unsigned long * __restrict p2, + const unsigned long * __restrict p3) +{ + kernel_rvv_begin(); + xor_regs_3_(bytes, p1, p2, p3); + kernel_rvv_end(); +} + +static void xor_rvv_4(unsigned long bytes, unsigned long * __restrict p1, + const unsigned long * __restrict p2, + const unsigned long * __restrict p3, + const unsigned long * __restrict p4) +{ + kernel_rvv_begin(); + xor_regs_4_(bytes, p1, p2, p3, p4); + kernel_rvv_end(); +} + +static void xor_rvv_5(unsigned long bytes, unsigned long * __restrict p1, + const unsigned long * __restrict p2, + const unsigned long * __restrict p3, + const unsigned long * __restrict p4, + const unsigned long * __restrict p5) +{ + kernel_rvv_begin(); + xor_regs_5_(bytes, p1, p2, p3, p4, p5); + kernel_rvv_end(); +} + +static struct xor_block_template xor_block_rvv = { + .name = "rvv", + .do_2 = xor_rvv_2, + .do_3 = xor_rvv_3, + .do_4 = xor_rvv_4, + .do_5 = xor_rvv_5 +}; + +#undef XOR_TRY_TEMPLATES +#define XOR_TRY_TEMPLATES \ + do { \ + xor_speed(&xor_block_8regs); \ + xor_speed(&xor_block_32regs); \ + if (has_vector()) { \ + xor_speed(&xor_block_rvv);\ + } \ + } while (0) +#endif diff --git a/arch/riscv/lib/Makefile b/arch/riscv/lib/Makefile index 25d5c9664e57..acd87ac86d24 100644 --- a/arch/riscv/lib/Makefile +++ b/arch/riscv/lib/Makefile @@ -7,3 +7,4 @@ lib-$(CONFIG_MMU) += uaccess.o lib-$(CONFIG_64BIT) += tishift.o obj-$(CONFIG_FUNCTION_ERROR_INJECTION) += error-inject.o +lib-$(CONFIG_VECTOR) += xor.o diff --git a/arch/riscv/lib/xor.S b/arch/riscv/lib/xor.S new file mode 100644 index 000000000000..3bc059e18171 --- /dev/null +++ b/arch/riscv/lib/xor.S @@ -0,0 +1,81 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2021 SiFive + */ +#include +#include +#include + +ENTRY(xor_regs_2_) + vsetvli a3, a0, e8, m8, ta, ma + vle8.v v0, (a1) + vle8.v v8, (a2) + sub a0, a0, a3 + vxor.vv v16, v0, v8 + add a2, a2, a3 + vse8.v v16, (a1) + add a1, a1, a3 + bnez a0, xor_regs_2_ + ret +END(xor_regs_2_) +EXPORT_SYMBOL(xor_regs_2_) + +ENTRY(xor_regs_3_) + vsetvli a4, a0, e8, m8, ta, ma + vle8.v v0, (a1) + vle8.v v8, (a2) + sub a0, a0, a4 + vxor.vv v0, v0, v8 + vle8.v v16, (a3) + add a2, a2, a4 + vxor.vv v16, v0, v16 + add a3, a3, a4 + vse8.v v16, (a1) + add a1, a1, a4 + bnez a0, xor_regs_3_ + ret +END(xor_regs_3_) +EXPORT_SYMBOL(xor_regs_3_) + +ENTRY(xor_regs_4_) + vsetvli a5, a0, e8, m8, ta, ma + vle8.v v0, (a1) + vle8.v v8, (a2) + sub a0, a0, a5 + vxor.vv v0, v0, v8 + vle8.v v16, (a3) + add a2, a2, a5 + vxor.vv v0, v0, v16 + vle8.v v24, (a4) + add a3, a3, a5 + vxor.vv v16, v0, v24 + add a4, a4, a5 + vse8.v v16, (a1) + add a1, a1, a5 + bnez a0, xor_regs_4_ + ret +END(xor_regs_4_) +EXPORT_SYMBOL(xor_regs_4_) + +ENTRY(xor_regs_5_) + vsetvli a6, a0, e8, m8, ta, ma + vle8.v v0, (a1) + vle8.v v8, (a2) + sub a0, a0, a6 + vxor.vv v0, v0, v8 + vle8.v v16, (a3) + add a2, a2, a6 + vxor.vv v0, v0, v16 + vle8.v v24, (a4) + add a3, a3, a6 + vxor.vv v0, v0, v24 + vle8.v v8, (a5) + add a4, a4, a6 + vxor.vv v16, v0, v8 + add a5, a5, a6 + vse8.v v16, (a1) + add a1, a1, a6 + bnez a0, xor_regs_5_ + ret +END(xor_regs_5_) +EXPORT_SYMBOL(xor_regs_5_) From patchwork Wed Sep 21 16:50:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12984002 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D5D63C6FA8E for ; Wed, 21 Sep 2022 16:50:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=JErx8qwIDxj7Lq5ES+oWaTgiHLRFJQJC777IsZdWZls=; b=jWnBzBcEq6Gokl eQ0Zai2tL7/nhNGsuQjUQ87RBwKb9g6NKTVo/jvqNAXHKCrFgJ2XzV4udND16X7dyPd6p++KpfdsS 4IQZzIM7Hh4noHHsuu3Sz6b5NExOp7a5JEbkGSMxAX+CIvj+Mo/TAEbrCX6Hw6Dl5Ir754bwn4a3O P5sZlZSTBhzDjcmhlTsI79iIel+5+Lb9paj/CgqHOsxHBeaIufJ3GXHV2fzblrCfU449pnaOtNRd5 b8HsTf/tVMmqAHYuN8IfIG14P1TJttaNch7UGmY0bublwyCLI1eeyHeeK3M9MThRSblYsKrJxXkiC pkSxm2iw6T8oz0aRj4SQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2vp-00C45L-KM; Wed, 21 Sep 2022 16:50:41 +0000 Received: from mail-wr1-x433.google.com ([2a00:1450:4864:20::433]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2vm-00C42s-NF for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:50:40 +0000 Received: by mail-wr1-x433.google.com with SMTP id n15so3965418wrq.5 for ; Wed, 21 Sep 2022 09:50:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=2gHgaIeXWsDaK0jT9I81izavyxaAOC2Z2MO2p3WE3C4=; b=nkeni2Jn+Kuku0Chnr478GEVpIWCpyu/jXkU1/ZsCUZaIVLYjEWnomRI0L3SSQooVs ZHKss5UCQvu4bnTjQDdZtJugLaF/ye6eqQq2lrx2Nz130K1g3Cl9I7RZqOpL92h7mQdE 7gwXp3OVqYed0YbMi0z+37Vta/gOf2fYNP+H1iS+spYIDXmwFiBrhkLEe7pJP/bfaNsc rT9NvoNF+F8yAIXq68FxrVwxq6fzaLbiW8jxsd7nKtS8TvCjw2qXlaORPpZVuXFgtbGe 6JAm3OricsYWlmUyIt11ZTDDLj+h8eDZLxv3vZuk8Db5m629AIjiLCV9HklIsKpqqPYC 4hZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=2gHgaIeXWsDaK0jT9I81izavyxaAOC2Z2MO2p3WE3C4=; b=tgubpHlIQzyxGasUJUSoNYWNFW03BlqOvb94Fa6gWe00NJ7mNRiKFqIdCegNETOBlM JQi661V6E3kX98gPcG1uhJ7A/mMRBout1q+rIIuDcXffin2f7QRonHX0xHlyKaVmurnB vIU/l70G8BTVpnpce02meySPS1cZl9aWkOQk7aJVMhdPw4Brotd1HuEJAHqs07VSnnO+ LMJGSnbNNRJKY0H2Euxu47tIOSZPzDALGpbJZtPDyQqEFVvQgmYmPKc3ZiC9f+Gn3jly xMtNP8ujHPdjMfXOQV2GZgWDIfTK9/TT39bgm54p6CZQzDMNueU10YaBuZ/Up8S5yRhd AM9Q== X-Gm-Message-State: ACrzQf31TYrXv/0hTMwSxAAp2q0oIzyH0kMp5Mg5lW8rLS77kCieuoqZ jFu8p8W0kQbN3XTkgX5kBI9TWnEQX8AWT3Oo8aKwL8xnoqDrDg== X-Google-Smtp-Source: AMsMyM5egBqt8G1bG4m7b5OUvFG+8mLc1ag2sSWFSoiQm2i9VOXx8QmLfjU+VdmTT8HCS5Set7wcZ/o6sm3rB+IANKI= X-Received: by 2002:adf:dcc8:0:b0:22a:b9e3:bab8 with SMTP id x8-20020adfdcc8000000b0022ab9e3bab8mr17956544wrm.341.1663779037122; Wed, 21 Sep 2022 09:50:37 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:50:01 -0700 Message-ID: Subject: [PATCH 14/17] riscv: Fix a kernel panic issue if $s2 is set to a specific value before entering Linux To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_095038_781004_105FD8D9 X-CRM114-Status: GOOD ( 15.02 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Panic log: [ 0.018707] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000 [ 0.023060] Oops [#1] [ 0.023214] Modules linked in: [ 0.023725] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.14.0 #33 [ 0.023955] Hardware name: SiFive,FU800 (DT) [ 0.024150] epc : __vstate_save+0x1c/0x48 [ 0.024654] ra : arch_dup_task_struct+0x70/0x108 [ 0.024815] epc : ffffffff80005ad8 ra : ffffffff800035a8 sp : ffffffff81203d50 [ 0.025020] gp : ffffffff812e8290 tp : ffffffff8120bdc0 t0 : 0000000000000000 [ 0.025216] t1 : 0000000000000000 t2 : 0000000000000000 s0 : ffffffff81203d80 [ 0.025424] s1 : ffffffff8120bdc0 a0 : ffffffff8120c820 a1 : 0000000000000000 [ 0.025659] a2 : 0000000000001000 a3 : 0000000000000000 a4 : 0000000000000600 [ 0.025869] a5 : ffffffff8120cdc0 a6 : ffffffe00160b400 a7 : ffffffff80a1fe60 [ 0.026069] s2 : ffffffe0016b8000 s3 : ffffffff81204000 s4 : 0000000000004000 [ 0.026267] s5 : 0000000000000000 s6 : ffffffe0016b8000 s7 : ffffffe0016b9000 [ 0.026475] s8 : ffffffff81203ee0 s9 : 0000000000800300 s10: ffffffff812e9088 [ 0.026689] s11: ffffffd004008000 t3 : 0000000000000000 t4 : 0000000000000100 [ 0.026900] t5 : 0000000000000600 t6 : ffffffe00167bcc4 [ 0.027057] status: 8000000000000720 badaddr: 0000000000000000 cause: 000000000000000f [ 0.027344] [] __vstate_save+0x1c/0x48 [ 0.027567] [] copy_process+0x266/0x11a0 [ 0.027739] [] kernel_clone+0x90/0x2aa [ 0.027915] [] kernel_thread+0x76/0x92 [ 0.028075] [] rest_init+0x26/0xfc [ 0.028242] [] arch_call_rest_init+0x10/0x18 [ 0.028423] [] start_kernel+0x5ce/0x5fe [ 0.029188] ---[ end trace 9a59af33f7ba3df4 ]--- [ 0.029479] Kernel panic - not syncing: Attempted to kill the idle task! [ 0.029907] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]--- The NULL pointer accessing caused the kernel panic. There is a NULL pointer is because in vstate_save() function it will check (regs->status & SR_VS) == SR_VS_DIRTY and this is true, but it shouldn't be true because vector is not used here. Since vector is not used, datap won't be allocated so it is NULL. The reason why regs->status is set to a wrong value is because pt_regs->status is put in stack and it is polluted after setup_vm() called. In prologue of setup_vm(), we can observe it will save s2 to stack however s2 is meaningless here because the caller is assembly code and s2 is just some value from previous stage. The compiler will base on calling convention to save the register to stack. Then 0x80008638 in s2 is saved to stack. It might be any value. In this failure case it is 0x80008638 and it will accidentally cause SR_VS_DIRTY to call the vstate_save() function. (gdb) info addr setup_vm Symbol "setup_vm" is a function at address 0xffffffff80802c8a. (gdb) va2pa 0xffffffff80802c8a $64 = 0x80a02c8a (gdb) x/10i 0x80a02c8a 0x80a02c8a: addi sp,sp,-48 0x80a02c8c: li a3,-1 0x80a02c8e: auipc a5,0xff7fd 0x80a02c92: addi a5,a5,882 0x80a02c96: sd s0,32(sp) 0x80a02c98: sd s2,16(sp) <-- store to stack After returning from setup_vm() (gdb) x/20i 0x0000000080201138 0x80201138: mv a0,s1 0x8020113a: auipc ra,0x802 0x8020113e: jalr -1200(ra) <-- jump to setup_vm() 0x80201142: auipc a0,0xa03 (gdb) p/x $sp $70 = 0x81404000 (gdb) p/x *(struct pt_regs*)($sp-0x120) $71 = { epc = 0x0, ra = 0x0, sp = 0x0, gp = 0x0, tp = 0x0, t0 = 0x0, t1 = 0x0, t2 = 0x0, s0 = 0x0, s1 = 0x0, a0 = 0x0, a1 = 0x0, a2 = 0x0, a3 = 0x81403f90, a4 = 0x80c04000, a5 = 0x1, a6 = 0xffffffff81337000, a7 = 0x81096700, s2 = 0x81400000, s3 = 0xffffffff81200000, s4 = 0x81403fd0, s5 = 0x80a02c6c, s6 = 0x8000000000006800, s7 = 0x0, s8 = 0xfffffffffffffff3, s9 = 0x80c01000, s10 = 0x81096700, s11 = 0x82200000, t3 = 0x81404000, t4 = 0x80a02dea, t5 = 0x0, t6 = 0x82200000, status = 0x80008638, <- Wrong value in stack!!! badaddr = 0x82200000, cause = 0x0, orig_a0 = 0x80201142 } (gdb) p/x $pc $72 = 0x80201142 (gdb) p/x sizeof(struct pt_regs) $73 = 0x120 Co-developed-by: ShihPo Hung Signed-off-by: ShihPo Hung Co-developed-by: Vincent Chen Signed-off-by: Vincent Chen Signed-off-by: Greentime Hu --- arch/riscv/kernel/head.S | 2 ++ 1 file changed, 2 insertions(+) -- 2.25.1 diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S index 2c81ca42ec4e..c7effef23f41 100644 --- a/arch/riscv/kernel/head.S +++ b/arch/riscv/kernel/head.S @@ -301,6 +301,7 @@ clear_bss_done: la tp, init_task la sp, init_thread_union + THREAD_SIZE XIP_FIXUP_OFFSET sp + addi sp, sp, -PT_SIZE #ifdef CONFIG_BUILTIN_DTB la a0, __dtb_start XIP_FIXUP_OFFSET a0 @@ -318,6 +319,7 @@ clear_bss_done: /* Restore C environment */ la tp, init_task la sp, init_thread_union + THREAD_SIZE + addi sp, sp, -PT_SIZE #ifdef CONFIG_KASAN call kasan_early_init From patchwork Wed Sep 21 16:50:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12984003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BEEBDC6FA82 for ; Wed, 21 Sep 2022 16:51:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=Etyg4MVWdM/+qzNoxDRi8TWIE66R25CfOhAttgBspWM=; b=vGPELZE0rXGRVc 9bnL4zSGTsHA4qCu02qCN0beBIy5k36HEq61IpihzwhlodZvn+EOLX/Co2JnYEPUZFcZycEnDVf8b yRnp0J32Op9XujR6HUHZuX16IjEQ3lv+PsqzVMrYmS2/b/Ns4cpDTPc2f6Dk+1Ijhqewrik17jq7D COpasY3fq3hTgKcEaYc4a6pV880KRczYS0I1f+a+DVvq8nUQwH1bxSc2djamaBfc2QMxGVpCgd+x0 15F7zYWPQ2cOfYGmQddN1A22kppPN8UvybfV6JSxh7se5Qs8UP1plyrdsD8LtDhBWZrIJVQJqhp+d eKrWP0Uu4jZboOobWWrw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2w7-00C4I8-Gp; Wed, 21 Sep 2022 16:50:59 +0000 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2w4-00C4Es-05 for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:50:58 +0000 Received: by mail-wm1-x332.google.com with SMTP id iv17so271689wmb.4 for ; Wed, 21 Sep 2022 09:50:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=wNUcq1etcZciXQCHJChvzfvxl28Joooge09sg7cQiwc=; b=Jzi1LryoMP5RPsOHf5u8iCZIRCNbfj8N4WxVj6q1rrnZQOIbReqgdttTe3EFwyqw2h QuIzqoH0ou10m7K44afWjxtgNCV/iyzWZ/lDLHBPJdJMp07FM3Co3dDBcdRur2Wy46yW fK1jaJVTxrBsGfdKncRYylT9CTK6QT24Ft2WvzM+1XhHBbfZW2Pg18F8FgOUIj6QFugM lqO6zE5M5ucQzvsVU6acfYO+gmPypUkK4mCI9yURBj34z/GpJu/bQ/iYmiKesbyDbhzt tcskxBom0rzCxQhjlFOlZy1UrLNAg1u0jpaNRMne+O1HyH1dxqFyOTjmQlRxxnbeB5Gv VCug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=wNUcq1etcZciXQCHJChvzfvxl28Joooge09sg7cQiwc=; b=nx9M0nOuItwTTGsbaBg8+VI434i9SjEj6ZAlZr7JiU6RsAtHcQkSc9snnMw7tKATZQ NRTdAzbkdHZe96sNKYYrP2icXaRgt82T+qEjCNR03/Ryn4MtM1/5sAO2ItF0z1AH/wE/ WBFRwuoi8oSQZFRza+jgDF7gM+YfG3wZR4ifgvN1CsiQyRoAYtPxttyvCvEG0bacs+Sm gjo15uLKYuUX7E6RJ18k4lkcMX6N0mlQ92TFQVFX3WUR2uH/wSzbfqBpqfhtvBEdZz/g VRrlfLAQSCSWTC9IwfsYpWpPgV1nqvCQSS6VADe3q6EVE9mYpaWNFtPYaRHI+yokycXD ZcWA== X-Gm-Message-State: ACrzQf1zEddWsMsaTAbvwVJSXTxIxar6EYWFfIczNWsQHDcOb/aoVia/ oTZUMI6KVEJRxZr8/z53IgtBydbpyK3eAho6k2LzXw0SJem2ww== X-Google-Smtp-Source: AMsMyM6s6C7h/XHI6eGYcrWSxpaHqFnbQN31rOQItXfDFfJ8W15eX4HsTzDspNqLXzj9dh/9m40LOuSLuxd+TU0Kv1c= X-Received: by 2002:a05:600c:410d:b0:3b4:9454:f894 with SMTP id j13-20020a05600c410d00b003b49454f894mr6442213wmi.111.1663779053539; Wed, 21 Sep 2022 09:50:53 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:50:17 -0700 Message-ID: Subject: [PATCH 15/17] riscv: Add V extension to KVM ISA allow list To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_095056_076021_B948364D X-CRM114-Status: UNSURE ( 7.83 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Add V extension to KVM_RISCV_ISA_ALLOWED list to enable VCPU to support V extension. Signed-off-by: Vincent Chen Signed-off-by: Greentime Hu --- arch/riscv/include/asm/hwcap.h | 1 + 1 file changed, 1 insertion(+) -- 2.25.1 diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index 6f59ec64175e..b242ed155262 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -35,6 +35,7 @@ extern unsigned long elf_hwcap; #define RISCV_ISA_EXT_m ('m' - 'a') #define RISCV_ISA_EXT_s ('s' - 'a') #define RISCV_ISA_EXT_u ('u' - 'a') +#define RISCV_ISA_EXT_v ('v' - 'a') /* * Increse this to higher value as kernel support more ISA extensions. From patchwork Wed Sep 21 16:50:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12984004 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CB14FC6FA82 for ; Wed, 21 Sep 2022 16:51:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=QA+ohmAN5AdquuXperRJOKVnK0OTSP/rD1fDpOi0dYQ=; b=ffJgtr3tkiG1f8 NW8/v0WNvfwCx4RBXe1K0xZTSX/2zk1dK04JdYdXzQCsd0HnH8ShZ084nAd9FrDlrhcvtcoVBK4pv NuDzOqeZknNFSNeoBUT6IDlOPAvipAkDO0Os0USTq1Th1SRsDxl/9D4LrTn/QFFcQYXLnKO/qq5zp PAOpx1cPoEL9ptHlNwI5hXudG8JBHmF30dbFnvE0IbAUeEwr9AbiahAuK5bRJ2VSjXCQT8gj7cO+F tyd/6Q4b+mSkvA7HGGWpd/EJmYcTP/gh8LDXswva6PH3uA9000oxi4Bw4JloNPrviGfbda8HizEN4 AMmYadvVSsBoXkqowTlA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2wP-00C4S7-Iu; Wed, 21 Sep 2022 16:51:17 +0000 Received: from mail-wm1-x336.google.com ([2a00:1450:4864:20::336]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2wL-00C4OI-En for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:51:15 +0000 Received: by mail-wm1-x336.google.com with SMTP id e10-20020a05600c4e4a00b003b4eff4ab2cso3835006wmq.4 for ; Wed, 21 Sep 2022 09:51:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=OLd7jomrti0MIXYfhUfWpHNmSafVVmXZj2Icqw//+Ag=; b=jgvRm5lZ9NBhQOytWRfNLF4bxMRzaAgadRwSTi01FSFiso7mk5A/7lNKRAyQEifJx/ O46qVIsephT9qgHV7ebrcckBhumL1uby6hPAOf6EXKVGv0DASG+J8k2cHMjHZOwrrfhV /RxZ8TizoEZZSLtJATgFEA9S8oeArDffeMLE/yNb/9yovXs3pbQWFywkZ3mW8uTnoOpX 1DRRdu48F+3zZnaSLUyT2S5J9CX6ZXA2el2ArVlSsqB6fU8evfdiP1a2EYadTffatb+0 XAfB4nPg0J2QHx6nIB3TDnfPtzQ9bfBHu+eT9bk0opNTHZPKPbEoZYq7ElcHTjv2FdMi xEYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=OLd7jomrti0MIXYfhUfWpHNmSafVVmXZj2Icqw//+Ag=; b=3iBghob9yn8HVlqt9CGPA7j6BC63pm67pSbXW/V75ylbj+ioLH7n0XWysQicG51zag vDN55FP03KL46VTEfML01KP+WPLVdarLZGDNHI7I6aWQN13WFSEq0AHBySGmzPxNs9bZ 01fixBam9BLeFbTDGRZGHep3mXQO2uDfSot28GcZaf982vjbXfHv+FPQTkXJSJLupnP0 6S6bH7W4j7TsdnvYbHfeVylOcG9HerA/Zd63TOmoRXB9ItR0VRHvvMrk4MB2V/FjoXGe hatmkAgYY7I9yqfI/4/k/oAnSnYClfdK/zhtJgjogdfr5lnJ2dvAoFOZu2NdOoiU2uGO jZuw== X-Gm-Message-State: ACrzQf0Uko95Y/dgzjnm+q25//SRvzrLppPu3q7W28aa8IW1r9oyGb3r y8NIWlFE9F34np8mE8kMn9EEIgvrsE/nWVEYUcIqjFWRk5RKZA== X-Google-Smtp-Source: AMsMyM4JCc8to3TR1MDAxwPtUVBI1rmbTSoWuR80r4J+eRNRcrgpfit+pbA+zu5LSsuv1g+qcl+SMiStgf8pb68FGV0= X-Received: by 2002:a05:600c:5118:b0:3b4:fd72:7a8 with SMTP id o24-20020a05600c511800b003b4fd7207a8mr1256540wms.80.1663779070092; Wed, 21 Sep 2022 09:51:10 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:50:34 -0700 Message-ID: Subject: [PATCH 16/17] riscv: KVM: Add vector lazy save/restore support To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_095113_527398_8C8364B2 X-CRM114-Status: GOOD ( 21.00 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This patch adds vector context save/restore for guest VCPUs. To reduce the impact on KVM performance, the implementation imitates the FP context switch mechanism to lazily store and restore the vector context only when the kernel enters/exits the in-kernel run loop and not during the KVM world switch. Signed-off-by: Vincent Chen Signed-off-by: Greentime Hu --- arch/riscv/include/asm/kvm_host.h | 2 + arch/riscv/include/asm/kvm_vcpu_vector.h | 65 +++++++++ arch/riscv/include/uapi/asm/kvm.h | 7 + arch/riscv/kernel/asm-offsets.c | 7 + arch/riscv/kvm/Makefile | 1 + arch/riscv/kvm/vcpu.c | 32 +++++ arch/riscv/kvm/vcpu_switch.S | 69 +++++++++ arch/riscv/kvm/vcpu_vector.c | 173 +++++++++++++++++++++++ 8 files changed, 356 insertions(+) create mode 100644 arch/riscv/include/asm/kvm_vcpu_vector.h create mode 100644 arch/riscv/kvm/vcpu_vector.c -- 2.25.1 diff --git a/arch/riscv/include/asm/kvm_host.h b/arch/riscv/include/asm/kvm_host.h index 60c517e4d576..665ddb4cec62 100644 --- a/arch/riscv/include/asm/kvm_host.h +++ b/arch/riscv/include/asm/kvm_host.h @@ -17,6 +17,7 @@ #include #include #include +#include #include #define KVM_MAX_VCPUS 1024 @@ -143,6 +144,7 @@ struct kvm_cpu_context { unsigned long sstatus; unsigned long hstatus; union __riscv_fp_state fp; + struct __riscv_v_state vector; }; struct kvm_vcpu_csr { diff --git a/arch/riscv/include/asm/kvm_vcpu_vector.h b/arch/riscv/include/asm/kvm_vcpu_vector.h new file mode 100644 index 000000000000..1dcc1b2e05bb --- /dev/null +++ b/arch/riscv/include/asm/kvm_vcpu_vector.h @@ -0,0 +1,65 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2021 Western Digital Corporation or its affiliates. + * Copyright (C) 2022 SiFive + * + * Authors: + * Atish Patra + * Anup Patel + * Vincent Chen + * Greentime Hu + */ + +#ifndef __KVM_VCPU_RISCV_VECTOR_H +#define __KVM_VCPU_RISCV_VECTOR_H + +#include + +struct kvm_cpu_context; + +#ifdef CONFIG_VECTOR +void __kvm_riscv_vector_save(struct kvm_cpu_context *context); +void __kvm_riscv_vector_restore(struct kvm_cpu_context *context); +void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu); +void kvm_riscv_vcpu_guest_vector_save(struct kvm_cpu_context *cntx, + unsigned long isa); +void kvm_riscv_vcpu_guest_vector_restore(struct kvm_cpu_context *cntx, + unsigned long isa); +void kvm_riscv_vcpu_host_vector_save(struct kvm_cpu_context *cntx); +void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx); +void kvm_riscv_vcpu_free_vector_context(struct kvm_vcpu *vcpu); +#else +static inline void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu) +{ +} + +static inline void kvm_riscv_vcpu_guest_vector_save(struct kvm_cpu_context *cntx, + unsigned long isa) +{ +} + +static inline void kvm_riscv_vcpu_guest_vector_restore(struct kvm_cpu_context *cntx, + unsigned long isa) +{ +} + +static inline void kvm_riscv_vcpu_host_vector_save(struct kvm_cpu_context *cntx) +{ +} + +static inline void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx) +{ +} + +static inline void kvm_riscv_vcpu_free_vector_context(struct kvm_vcpu *vcpu) +{ +} +#endif + +int kvm_riscv_vcpu_get_reg_vector(struct kvm_vcpu *vcpu, + const struct kvm_one_reg *reg, + unsigned long rtype); +int kvm_riscv_vcpu_set_reg_vector(struct kvm_vcpu *vcpu, + const struct kvm_one_reg *reg, + unsigned long rtype); +#endif diff --git a/arch/riscv/include/uapi/asm/kvm.h b/arch/riscv/include/uapi/asm/kvm.h index 7351417afd62..f4ba57b235a3 100644 --- a/arch/riscv/include/uapi/asm/kvm.h +++ b/arch/riscv/include/uapi/asm/kvm.h @@ -96,6 +96,7 @@ enum KVM_RISCV_ISA_EXT_ID { KVM_RISCV_ISA_EXT_H, KVM_RISCV_ISA_EXT_I, KVM_RISCV_ISA_EXT_M, + KVM_RISCV_ISA_EXT_V, KVM_RISCV_ISA_EXT_SVPBMT, KVM_RISCV_ISA_EXT_SSTC, KVM_RISCV_ISA_EXT_MAX, @@ -145,6 +146,12 @@ enum KVM_RISCV_ISA_EXT_ID { /* ISA Extension registers are mapped as type 7 */ #define KVM_REG_RISCV_ISA_EXT (0x07 << KVM_REG_RISCV_TYPE_SHIFT) +/* V extension registers are mapped as type 7 */ +#define KVM_REG_RISCV_VECTOR (0x07 << KVM_REG_RISCV_TYPE_SHIFT) +#define KVM_REG_RISCV_VECTOR_CSR_REG(name) \ + (offsetof(struct __riscv_v_state, name) / sizeof(unsigned long)) +#define KVM_REG_RISCV_VECTOR_REG(n) \ + ((n) + sizeof(struct __riscv_v_state) / sizeof(unsigned long)) #endif #endif /* __LINUX_KVM_RISCV_H */ diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c index 80316ef7bb78..2540b9146072 100644 --- a/arch/riscv/kernel/asm-offsets.c +++ b/arch/riscv/kernel/asm-offsets.c @@ -278,6 +278,13 @@ void asm_offsets(void) OFFSET(KVM_ARCH_FP_D_F31, kvm_cpu_context, fp.d.f[31]); OFFSET(KVM_ARCH_FP_D_FCSR, kvm_cpu_context, fp.d.fcsr); + /* V extension */ + + OFFSET(KVM_ARCH_VECTOR_VSTART, kvm_cpu_context, vector.vstart); + OFFSET(KVM_ARCH_VECTOR_VL, kvm_cpu_context, vector.vl); + OFFSET(KVM_ARCH_VECTOR_VTYPE, kvm_cpu_context, vector.vtype); + OFFSET(KVM_ARCH_VECTOR_VCSR, kvm_cpu_context, vector.vcsr); + OFFSET(KVM_ARCH_VECTOR_DATAP, kvm_cpu_context, vector.datap); /* * THREAD_{F,X}* might be larger than a S-type offset can handle, but * these are used in performance-sensitive assembly so we can't resort diff --git a/arch/riscv/kvm/Makefile b/arch/riscv/kvm/Makefile index 019df9208bdd..b26bc605a267 100644 --- a/arch/riscv/kvm/Makefile +++ b/arch/riscv/kvm/Makefile @@ -17,6 +17,7 @@ kvm-y += mmu.o kvm-y += vcpu.o kvm-y += vcpu_exit.o kvm-y += vcpu_fp.o +kvm-y += vcpu_vector.o kvm-y += vcpu_insn.o kvm-y += vcpu_switch.o kvm-y += vcpu_sbi.o diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c index d0f08d5b4282..76941937e745 100644 --- a/arch/riscv/kvm/vcpu.c +++ b/arch/riscv/kvm/vcpu.c @@ -19,6 +19,7 @@ #include #include #include +#include const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { KVM_GENERIC_VCPU_STATS(), @@ -51,6 +52,7 @@ static const unsigned long kvm_isa_ext_arr[] = { RISCV_ISA_EXT_h, RISCV_ISA_EXT_i, RISCV_ISA_EXT_m, + RISCV_ISA_EXT_v, RISCV_ISA_EXT_SVPBMT, RISCV_ISA_EXT_SSTC, }; @@ -79,6 +81,7 @@ static bool kvm_riscv_vcpu_isa_enable_allowed(unsigned long ext) return true; } +//CMS FIXME static bool kvm_riscv_vcpu_isa_disable_allowed(unsigned long ext) { switch (ext) { @@ -121,6 +124,8 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu *vcpu) kvm_riscv_vcpu_fp_reset(vcpu); + kvm_riscv_vcpu_vector_reset(vcpu); + kvm_riscv_vcpu_timer_reset(vcpu); WRITE_ONCE(vcpu->arch.irqs_pending, 0); @@ -171,6 +176,15 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) cntx->hstatus |= HSTATUS_SPVP; cntx->hstatus |= HSTATUS_SPV; + if (has_vector()) { + cntx->vector.datap = kmalloc(riscv_vsize, GFP_KERNEL); + if (!cntx->vector.datap) + return -ENOMEM; + vcpu->arch.host_context.vector.datap = kzalloc(riscv_vsize, GFP_KERNEL); + if (!vcpu->arch.host_context.vector.datap) + return -ENOMEM; + } + /* By default, make CY, TM, and IR counters accessible in VU mode */ reset_csr->scounteren = 0x7; @@ -201,6 +215,9 @@ void kvm_arch_vcpu_destroy(struct kvm_vcpu *vcpu) /* Free unused pages pre-allocated for G-stage page table mappings */ kvm_mmu_free_memory_cache(&vcpu->arch.mmu_page_cache); + + /* Free vector context space for host and guest kernel */ + kvm_riscv_vcpu_free_vector_context(vcpu); } int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu) @@ -539,6 +556,9 @@ static int kvm_riscv_vcpu_set_reg(struct kvm_vcpu *vcpu, KVM_REG_RISCV_FP_D); else if ((reg->id & KVM_REG_RISCV_TYPE_MASK) == KVM_REG_RISCV_ISA_EXT) return kvm_riscv_vcpu_set_reg_isa_ext(vcpu, reg); + else if ((reg->id & KVM_REG_RISCV_TYPE_MASK) == KVM_REG_RISCV_VECTOR) + return kvm_riscv_vcpu_set_reg_vector(vcpu, reg, + KVM_REG_RISCV_VECTOR); return -EINVAL; } @@ -562,6 +582,9 @@ static int kvm_riscv_vcpu_get_reg(struct kvm_vcpu *vcpu, KVM_REG_RISCV_FP_D); else if ((reg->id & KVM_REG_RISCV_TYPE_MASK) == KVM_REG_RISCV_ISA_EXT) return kvm_riscv_vcpu_get_reg_isa_ext(vcpu, reg); + else if ((reg->id & KVM_REG_RISCV_TYPE_MASK) == KVM_REG_RISCV_VECTOR) + return kvm_riscv_vcpu_get_reg_vector(vcpu, reg, + KVM_REG_RISCV_VECTOR); return -EINVAL; } @@ -818,6 +841,9 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) kvm_riscv_vcpu_host_fp_save(&vcpu->arch.host_context); kvm_riscv_vcpu_guest_fp_restore(&vcpu->arch.guest_context, vcpu->arch.isa); + kvm_riscv_vcpu_host_vector_save(&vcpu->arch.host_context); + kvm_riscv_vcpu_guest_vector_restore(&vcpu->arch.guest_context, + vcpu->arch.isa); vcpu->cpu = cpu; } @@ -834,6 +860,12 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) kvm_riscv_vcpu_timer_save(vcpu); + kvm_riscv_vcpu_guest_vector_save(&vcpu->arch.guest_context, + vcpu->arch.isa); + kvm_riscv_vcpu_host_vector_restore(&vcpu->arch.host_context); + + csr_write(CSR_HGATP, 0); + csr->vsstatus = csr_read(CSR_VSSTATUS); csr->vsie = csr_read(CSR_VSIE); csr->vstvec = csr_read(CSR_VSTVEC); diff --git a/arch/riscv/kvm/vcpu_switch.S b/arch/riscv/kvm/vcpu_switch.S index d74df8eb4d71..730dc9b8c644 100644 --- a/arch/riscv/kvm/vcpu_switch.S +++ b/arch/riscv/kvm/vcpu_switch.S @@ -406,3 +406,72 @@ __kvm_riscv_fp_d_restore: csrw CSR_SSTATUS, t2 ret #endif + +#ifdef CONFIG_VECTOR + +#define vstatep a0 +#define datap a1 +#define x_vstart t0 +#define x_vtype t1 +#define x_vl t2 +#define x_vcsr t3 +#define incr t4 +#define status t5 + +ENTRY(__kvm_riscv_vector_save) + li status, SR_VS + csrs CSR_STATUS, status + + li a2, KVM_ARCH_VECTOR_DATAP + add datap, a0, a2 + ld datap, (datap) + csrr x_vstart, CSR_VSTART + csrr x_vtype, CSR_VTYPE + csrr x_vl, CSR_VL + csrr x_vcsr, CSR_VCSR + vsetvli incr, x0, e8, m8, ta, ma + vse8.v v0, (datap) + add datap, datap, incr + vse8.v v8, (datap) + add datap, datap, incr + vse8.v v16, (datap) + add datap, datap, incr + vse8.v v24, (datap) + + REG_S x_vstart, KVM_ARCH_VECTOR_VSTART(vstatep) + REG_S x_vtype, KVM_ARCH_VECTOR_VTYPE(vstatep) + REG_S x_vl, KVM_ARCH_VECTOR_VL(vstatep) + REG_S x_vcsr, KVM_ARCH_VECTOR_VCSR(vstatep) + + csrc CSR_STATUS, status + ret +ENDPROC(__kvm_riscv_vector_save) + +ENTRY(__kvm_riscv_vector_restore) + li status, SR_VS + csrs CSR_STATUS, status + + li a2, KVM_ARCH_VECTOR_DATAP + add datap, a0, a2 + ld datap, (datap) + vsetvli incr, x0, e8, m8, ta, ma + vle8.v v0, (datap) + add datap, datap, incr + vle8.v v8, (datap) + add datap, datap, incr + vle8.v v16, (datap) + add datap, datap, incr + vle8.v v24, (datap) + + REG_L x_vstart, KVM_ARCH_VECTOR_VSTART(vstatep) + REG_L x_vtype, KVM_ARCH_VECTOR_VTYPE(vstatep) + REG_L x_vl, KVM_ARCH_VECTOR_VL(vstatep) + REG_L x_vcsr, KVM_ARCH_VECTOR_VCSR(vstatep) + vsetvl x0, x_vl, x_vtype + csrw CSR_VSTART, x_vstart + csrw CSR_VCSR, x_vcsr + + csrc CSR_STATUS, status + ret +ENDPROC(__kvm_riscv_vector_restore) +#endif diff --git a/arch/riscv/kvm/vcpu_vector.c b/arch/riscv/kvm/vcpu_vector.c new file mode 100644 index 000000000000..37bf4ffd47dd --- /dev/null +++ b/arch/riscv/kvm/vcpu_vector.c @@ -0,0 +1,173 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2021 Western Digital Corporation or its affiliates. + * Copyright (C) 2022 SiFive + * + * Authors: + * Atish Patra + * Anup Patel + * Vincent Chen + * Greentime Hu + */ + +#include +#include +#include +#include +#include + +#ifdef CONFIG_VECTOR +extern unsigned long riscv_vsize; +void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu) +{ + unsigned long isa = vcpu->arch.isa; + struct kvm_cpu_context *cntx = &vcpu->arch.guest_context; + + cntx->sstatus &= ~SR_VS; + if (riscv_isa_extension_available(&isa, v)) + cntx->sstatus |= SR_VS_INITIAL; + else + cntx->sstatus |= SR_VS_OFF; + + memset(cntx->vector.datap, 0, riscv_vsize); +} + +static void kvm_riscv_vcpu_vector_clean(struct kvm_cpu_context *cntx) +{ + cntx->sstatus &= ~SR_VS; + cntx->sstatus |= SR_VS_CLEAN; +} + +void kvm_riscv_vcpu_guest_vector_save(struct kvm_cpu_context *cntx, + unsigned long isa) +{ + if ((cntx->sstatus & SR_VS) == SR_VS_DIRTY) { + if (riscv_isa_extension_available(&isa, v)) + __kvm_riscv_vector_save(cntx); + kvm_riscv_vcpu_vector_clean(cntx); + } +} + +void kvm_riscv_vcpu_guest_vector_restore(struct kvm_cpu_context *cntx, + unsigned long isa) +{ + if ((cntx->sstatus & SR_VS) != SR_VS_OFF) { + if (riscv_isa_extension_available(&isa, v)) + __kvm_riscv_vector_restore(cntx); + kvm_riscv_vcpu_vector_clean(cntx); + } +} + +void kvm_riscv_vcpu_host_vector_save(struct kvm_cpu_context *cntx) +{ + /* No need to check host sstatus as it can be modified outside */ + __kvm_riscv_vector_save(cntx); +} + +void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx) +{ + __kvm_riscv_vector_restore(cntx); +} + +void kvm_riscv_vcpu_free_vector_context(struct kvm_vcpu *vcpu) +{ + kfree(vcpu->arch.guest_reset_context.vector.datap); + kfree(vcpu->arch.host_context.vector.datap); +} +#else +#define riscv_vsize (0) +#endif + +static void *kvm_riscv_vcpu_vreg_addr(struct kvm_vcpu *vcpu, + unsigned long reg_num, + size_t reg_size) +{ + struct kvm_cpu_context *cntx = &vcpu->arch.guest_context; + void *reg_val; + size_t vlenb = riscv_vsize / 32; + + if (reg_num < KVM_REG_RISCV_VECTOR_REG(0)) { + if (reg_size != sizeof(unsigned long)) + return NULL; + switch (reg_num) { + case KVM_REG_RISCV_VECTOR_CSR_REG(vstart): + reg_val = &cntx->vector.vstart; + break; + case KVM_REG_RISCV_VECTOR_CSR_REG(vl): + reg_val = &cntx->vector.vl; + break; + case KVM_REG_RISCV_VECTOR_CSR_REG(vtype): + reg_val = &cntx->vector.vtype; + break; + case KVM_REG_RISCV_VECTOR_CSR_REG(vcsr): + reg_val = &cntx->vector.vcsr; + break; + case KVM_REG_RISCV_VECTOR_CSR_REG(datap): + default: + return NULL; + } + } else if (reg_num <= KVM_REG_RISCV_VECTOR_REG(31)) { + if (reg_size != vlenb) + return NULL; + reg_val = cntx->vector.datap + + (reg_num - KVM_REG_RISCV_VECTOR_REG(0)) * vlenb; + } else { + return NULL; + } + + return reg_val; +} + +int kvm_riscv_vcpu_get_reg_vector(struct kvm_vcpu *vcpu, + const struct kvm_one_reg *reg, + unsigned long rtype) +{ + unsigned long isa = vcpu->arch.isa; + unsigned long __user *uaddr = + (unsigned long __user *)(unsigned long)reg->addr; + unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK | + KVM_REG_SIZE_MASK | + rtype); + void *reg_val; + size_t reg_size = KVM_REG_SIZE(reg->id); + + if ((rtype == KVM_REG_RISCV_VECTOR) && + riscv_isa_extension_available(&isa, v)) { + reg_val = kvm_riscv_vcpu_vreg_addr(vcpu, reg_num, reg_size); + } + + if (!reg_val) + return -EINVAL; + + if (copy_to_user(uaddr, reg_val, reg_size)) + return -EFAULT; + + return 0; +} + +int kvm_riscv_vcpu_set_reg_vector(struct kvm_vcpu *vcpu, + const struct kvm_one_reg *reg, + unsigned long rtype) +{ + unsigned long isa = vcpu->arch.isa; + unsigned long __user *uaddr = + (unsigned long __user *)(unsigned long)reg->addr; + unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK | + KVM_REG_SIZE_MASK | + rtype); + void *reg_val = NULL; + size_t reg_size = KVM_REG_SIZE(reg->id); + + if ((rtype == KVM_REG_RISCV_VECTOR) && + riscv_isa_extension_available(&isa, v)) { + reg_val = kvm_riscv_vcpu_vreg_addr(vcpu, reg_num, reg_size); + } + + if (!reg_val) + return -EINVAL; + + if (copy_from_user(reg_val, uaddr, reg_size)) + return -EFAULT; + + return 0; +} From patchwork Wed Sep 21 16:50:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chris Stillson X-Patchwork-Id: 12984005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 739DAC6FA82 for ; Wed, 21 Sep 2022 16:51:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=3RW7m+jcJhJrzKvd5/FVLlp3SkqlvprQRmy0N4iWQ6g=; b=c22oKzX6vExFAJ U4qX71HUHWFhhvVjuHzRriEPh3q85M8CCNhdl61ldSOacgOzZZabsCdjDv5tUYwBl+11qDiYmnxJ6 S3zi8sDe/EBg/S4d5TQWxv1jfsDPemU6j20cEWmIEZ41gIB87fqPbR/J8f4SdC8DZzYPYch9EDiQA SL9l1e9hH/6Sa85GVTL59Z6mm8+PMCk3Jt2tUycsfoT0+7MB8GAGqaVvqKUM6W/OIkkSKz4pH4WQd wek8ILyPpEQaqEC42ToQduz/N/HbQIDN89/n7TGvJ8OcIy1sXXcu+a+s6UzsZQpq6ZMWcd7joDlFR GKTlz58Y9xTUtqDyatkw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2wf-00C4bu-3d; Wed, 21 Sep 2022 16:51:33 +0000 Received: from mail-wm1-x336.google.com ([2a00:1450:4864:20::336]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ob2wc-00C4a2-Cp for linux-riscv@lists.infradead.org; Wed, 21 Sep 2022 16:51:32 +0000 Received: by mail-wm1-x336.google.com with SMTP id l8so4895752wmi.2 for ; Wed, 21 Sep 2022 09:51:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date; bh=iiNWrVYExZtyCTmhZMPXv3Z+V5/s9eeZmuS1v1hmbZc=; b=l6iPTEdQXyy7Lmzuah2UTCk2eXp7jdvPcOf9lidb8uCcqODJpCvm8+/gAdt0o7yiZN HRnR1FOjvBdbMO/6ITLdCfCHUQ1VZeMQXoauUveDDC4EWG2kZAeMVTCbu294FEZgJeHH QIDHy8w4jQ0skUbChO/JqNZrEDjoBTQwa68M4xhx/Cj8RGXdIarqlp6WHq+NPvxRXe/p MoaGAMPZJ/6eJSJO6JN+8nCpus3ttKhbZnOYcksg616foYbxISZ0UW/5nMrrXPwwtWyG kGOVU1jynRaws7ctHAwY4gQYHMpd8gizajzRWTBsRyCSsr+njvnUV1Xxa9HNs9hVTt9b LY5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date; bh=iiNWrVYExZtyCTmhZMPXv3Z+V5/s9eeZmuS1v1hmbZc=; b=wmd5ssl3ECfp7ZOajJerLa0Olr9lO1Vg9Gm5vpVzv6EdNqA6x8EHnOYCoGauZ9ug6t uWcEGfEQo5urk/CS8ICCh0v4wyG+znDv5zFIAcwvnPi+3erBRQG6RDURSZtILI7HtH40 RLpX55AYuDD5qqkW8dZaRMORuzFhwgn2LppG0uUQHYsdkts0Er4dXoKQhXB/UfI2ZvCB t4kxFY+q66gBzmMeBCH1t9YZDB8poees3pzGQA2mABlu5pKZmFsw26SHIbFlu+WGRSAP nB95dFW+RxBGeKsq38cHPeFbdaoM/QHxMgJkbpL/LvZA5v4O6sc532Nq1FpNpJ519NsA fX2Q== X-Gm-Message-State: ACrzQf3VsO+tFMgOQs+3rxRUcDbCf9zZMbIpcqkh7kuWfdngkgeLCd4v xMO3w7TPfFVzgE0TW2OHzPC5j/HXX+n4Y9lOHpqiCtmt7GKjwg== X-Google-Smtp-Source: AMsMyM6aor/vHTbQFZim6OWwSlN7TYikJGkCiKEB/N8x0zyjbwFR0fUFOS8gaw68B0aYsramY9VI5SlyeTK6zgjnsDw= X-Received: by 2002:a1c:4b05:0:b0:3b4:90c1:e249 with SMTP id y5-20020a1c4b05000000b003b490c1e249mr6690440wma.201.1663779088748; Wed, 21 Sep 2022 09:51:28 -0700 (PDT) MIME-Version: 1.0 From: Chris Stillson Date: Wed, 21 Sep 2022 09:50:52 -0700 Message-ID: Subject: [PATCH 17/17] riscv: prctl to enable vector commands To: linux-riscv@lists.infradead.org Cc: palmer@dabbelt.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220921_095130_473965_E1B7228E X-CRM114-Status: GOOD ( 18.11 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This code makes enabling the vector extension on a riscv manchine optional by adding an option to prctl() to allow a process to enable, disable or query its vector context state. -added prctl to enable/disable/query current vector state -added actual function in riscv specific code to change/query the process state - Fixed problem with initial set of patches (missing some EXPORT_SYMBOL() macro calls) - rebased to 6.0-rc1 --- arch/riscv/configs/defconfig | 6 ++++++ arch/riscv/include/asm/kvm_vcpu_vector.h | 8 ++++---- arch/riscv/include/asm/processor.h | 6 ++++++ arch/riscv/include/asm/switch_to.h | 11 +++++++++++ arch/riscv/kernel/cpufeature.c | 3 ++- arch/riscv/kernel/process.c | 20 +++++++++++++++++++- arch/riscv/kvm/vcpu_vector.c | 14 +++++++------- include/uapi/linux/prctl.h | 6 ++++++ kernel/sys.c | 7 +++++++ 9 files changed, 68 insertions(+), 13 deletions(-) break; -- 2.25.1 diff --git a/arch/riscv/configs/defconfig b/arch/riscv/configs/defconfig index aed332a9d4ea..fce054286b1f 100644 --- a/arch/riscv/configs/defconfig +++ b/arch/riscv/configs/defconfig @@ -209,3 +209,9 @@ CONFIG_RCU_EQS_DEBUG=y # CONFIG_FTRACE is not set # CONFIG_RUNTIME_TESTING_MENU is not set CONFIG_MEMTEST=y +CONFIG_ARCH_RV64I=y +CONFIG_64BIT=y +CONFIG_VECTOR=y +CONFIG_ARCH_RV64I=y +CONFIG_64BIT=y +CONFIG_VECTOR=y diff --git a/arch/riscv/include/asm/kvm_vcpu_vector.h b/arch/riscv/include/asm/kvm_vcpu_vector.h index 1dcc1b2e05bb..c7101ff943a0 100644 --- a/arch/riscv/include/asm/kvm_vcpu_vector.h +++ b/arch/riscv/include/asm/kvm_vcpu_vector.h @@ -22,9 +22,9 @@ void __kvm_riscv_vector_save(struct kvm_cpu_context *context); void __kvm_riscv_vector_restore(struct kvm_cpu_context *context); void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu); void kvm_riscv_vcpu_guest_vector_save(struct kvm_cpu_context *cntx, - unsigned long isa); + unsigned long *isa); void kvm_riscv_vcpu_guest_vector_restore(struct kvm_cpu_context *cntx, - unsigned long isa); + unsigned long *isa); void kvm_riscv_vcpu_host_vector_save(struct kvm_cpu_context *cntx); void kvm_riscv_vcpu_host_vector_restore(struct kvm_cpu_context *cntx); void kvm_riscv_vcpu_free_vector_context(struct kvm_vcpu *vcpu); @@ -34,12 +34,12 @@ static inline void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu) } static inline void kvm_riscv_vcpu_guest_vector_save(struct kvm_cpu_context *cntx, - unsigned long isa) + unsigned long *isa) { } static inline void kvm_riscv_vcpu_guest_vector_restore(struct kvm_cpu_context *cntx, - unsigned long isa) + unsigned long *isa) { } diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h index a09141ecf6aa..f2d0a91ce174 100644 --- a/arch/riscv/include/asm/processor.h +++ b/arch/riscv/include/asm/processor.h @@ -88,6 +88,12 @@ extern void riscv_fill_hwcap(void); extern int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src); extern unsigned long signal_minsigstksz __ro_after_init; + +#ifdef CONFIG_VECTOR +extern int rvv_proc_enable(unsigned long x); +#define RVV_PROC_ENABLE(x) rvv_proc_enable(x) +#endif + #endif /* __ASSEMBLY__ */ #endif /* _ASM_RISCV_PROCESSOR_H */ diff --git a/arch/riscv/include/asm/switch_to.h b/arch/riscv/include/asm/switch_to.h index 527951c033d4..d9747450311c 100644 --- a/arch/riscv/include/asm/switch_to.h +++ b/arch/riscv/include/asm/switch_to.h @@ -80,6 +80,17 @@ extern unsigned long riscv_vsize; extern void __vstate_save(struct __riscv_v_state *save_to, void *datap); extern void __vstate_restore(struct __riscv_v_state *restore_from, void *datap); +static inline bool vstate_query(struct pt_regs *regs) +{ + return (regs->status & SR_VS) != 0; +} + +static inline void vstate_on(struct task_struct *task, + struct pt_regs *regs) +{ + regs->status = (regs->status & ~(SR_VS)) | SR_VS_INITIAL; +} + static inline void __vstate_clean(struct pt_regs *regs) { regs->status = (regs->status & ~(SR_VS)) | SR_VS_CLEAN; diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 0487ab19b234..3be469cb9266 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -37,6 +37,8 @@ __ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_fpu); #include __ro_after_init DEFINE_STATIC_KEY_FALSE(cpu_hwcap_vector); unsigned long riscv_vsize __read_mostly; +EXPORT_SYMBOL(cpu_hwcap_vector); +EXPORT_SYMBOL(riscv_vsize); #endif /** @@ -346,4 +348,3 @@ void __init_or_module riscv_cpufeature_patch_func(struct alt_entry *begin, } } #endif -} diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c index e88a37fc77ed..a5a76d1374ec 100644 --- a/arch/riscv/kernel/process.c +++ b/arch/riscv/kernel/process.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include @@ -134,7 +135,6 @@ void start_thread(struct pt_regs *regs, unsigned long pc, if (WARN_ON(!vstate->datap)) return; } - regs->status |= SR_VS_INITIAL; /* * Restore the initial value to the vector register @@ -230,3 +230,21 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) p->thread.sp = (unsigned long)childregs; /* kernel sp */ return 0; } + +#ifdef CONFIG_VECTOR +int rvv_proc_enable(unsigned long x) { + switch (x) { + case PR_RVV_DISABLE: + vstate_off(current, task_pt_regs(current)); + return 0; + case PR_RVV_ENABLE: + vstate_on(current, task_pt_regs(current)); + return 0; + case PR_RVV_QUERY: + return vstate_query(task_pt_regs(current)); + default: + return -(EINVAL); + + } +} +#endif diff --git a/arch/riscv/kvm/vcpu_vector.c b/arch/riscv/kvm/vcpu_vector.c index 37bf4ffd47dd..9d1613da561a 100644 --- a/arch/riscv/kvm/vcpu_vector.c +++ b/arch/riscv/kvm/vcpu_vector.c @@ -20,7 +20,7 @@ extern unsigned long riscv_vsize; void kvm_riscv_vcpu_vector_reset(struct kvm_vcpu *vcpu) { - unsigned long isa = vcpu->arch.isa; + unsigned long isa = *vcpu->arch.isa; struct kvm_cpu_context *cntx = &vcpu->arch.guest_context; cntx->sstatus &= ~SR_VS; @@ -39,20 +39,20 @@ static void kvm_riscv_vcpu_vector_clean(struct kvm_cpu_context *cntx) } void kvm_riscv_vcpu_guest_vector_save(struct kvm_cpu_context *cntx, - unsigned long isa) + unsigned long *isa) { if ((cntx->sstatus & SR_VS) == SR_VS_DIRTY) { - if (riscv_isa_extension_available(&isa, v)) + if (riscv_isa_extension_available(isa, v)) __kvm_riscv_vector_save(cntx); kvm_riscv_vcpu_vector_clean(cntx); } } void kvm_riscv_vcpu_guest_vector_restore(struct kvm_cpu_context *cntx, - unsigned long isa) + unsigned long *isa) { if ((cntx->sstatus & SR_VS) != SR_VS_OFF) { - if (riscv_isa_extension_available(&isa, v)) + if (riscv_isa_extension_available(isa, v)) __kvm_riscv_vector_restore(cntx); kvm_riscv_vcpu_vector_clean(cntx); } @@ -122,7 +122,7 @@ int kvm_riscv_vcpu_get_reg_vector(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg, unsigned long rtype) { - unsigned long isa = vcpu->arch.isa; + unsigned long isa = *vcpu->arch.isa; unsigned long __user *uaddr = (unsigned long __user *)(unsigned long)reg->addr; unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK | @@ -149,7 +149,7 @@ int kvm_riscv_vcpu_set_reg_vector(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg, unsigned long rtype) { - unsigned long isa = vcpu->arch.isa; + unsigned long isa = *vcpu->arch.isa; unsigned long __user *uaddr = (unsigned long __user *)(unsigned long)reg->addr; unsigned long reg_num = reg->id & ~(KVM_REG_ARCH_MASK | diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index a5e06dcbba13..8ea56e4c48f8 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -281,6 +281,12 @@ struct prctl_mm_map { # define PR_SME_VL_LEN_MASK 0xffff # define PR_SME_VL_INHERIT (1 << 17) /* inherit across exec */ +/* RISC-V V vector extension */ +#define PR_RVV_STATE 65 +# define PR_RVV_DISABLE 0 +# define PR_RVV_ENABLE 1 +# define PR_RVV_QUERY 2 + #define PR_SET_VMA 0x53564d41 # define PR_SET_VMA_ANON_NAME 0 diff --git a/kernel/sys.c b/kernel/sys.c index b911fa6d81ab..3049b1823273 100644 --- a/kernel/sys.c +++ b/kernel/sys.c @@ -138,6 +138,9 @@ #ifndef GET_TAGGED_ADDR_CTRL # define GET_TAGGED_ADDR_CTRL() (-EINVAL) #endif +#ifndef RVV_PROC_ENABLE +# define RVV_PROC_ENABLE(x) (-EINVAL) +#endif /* * this is where the system-wide overflow UID and GID are defined, for @@ -2620,6 +2623,10 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3, error = sched_core_share_pid(arg2, arg3, arg4, arg5); break; #endif + case PR_RVV_STATE: + error = RVV_PROC_ENABLE(arg2); + break; + case PR_SET_VMA: error = prctl_set_vma(arg2, arg3, arg4, arg5);