From patchwork Tue Dec 19 17:50:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jisheng Zhang X-Patchwork-Id: 13498800 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B221AC46CA2 for ; Tue, 19 Dec 2023 18:03:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 516C48D0008; Tue, 19 Dec 2023 13:03:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 41C7B8D0001; Tue, 19 Dec 2023 13:03:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2701E8D0008; Tue, 19 Dec 2023 13:03:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 0E06E8D0001 for ; Tue, 19 Dec 2023 13:03:38 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D36BD160A48 for ; Tue, 19 Dec 2023 18:03:37 +0000 (UTC) X-FDA: 81584340474.20.7115247 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf07.hostedemail.com (Postfix) with ESMTP id D425340019 for ; Tue, 19 Dec 2023 18:03:35 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="BvEJaO/t"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf07.hostedemail.com: domain of jszhang@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=jszhang@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703009016; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZOfv5mqPVoNjWH7q2gBw3BsdWC6ZGwg86MfP6BHLzRA=; b=1/oTLbu3ly6l+MVTzIR5zNAAs0iEg6N6B9JoYdBiBmQ5aaCgvAQ5MSWL7u53MoVLUnx9qI 1KGA1eoAc7Aco87c+JDeEinCCudCpcdt9e8qYe/6Yk5/GdX2xXEdINMmAi9uDj6AesLaEW 1LfmenlNTz0MitbA3K37EoRJR85w5Iw= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="BvEJaO/t"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf07.hostedemail.com: domain of jszhang@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=jszhang@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1703009016; a=rsa-sha256; cv=none; b=BwBLqj//o1RkMElqpasvnkE/i75DGx/bRiLo3telCA4k6J18LjhU2QoImtjVpNlQiI9D+x +bNihkPX+Ckoe7jJDtKQNmtsMEC/29SA1iDopHYd+fMsyRN+jVmoMDQoWrA8fNKznBn7HX htFrkakoB3V3HUi+KTDzaA/1LcsXG7I= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id E5A88B81A3A; Tue, 19 Dec 2023 18:03:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 01013C433CC; Tue, 19 Dec 2023 18:03:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1703009013; bh=jOzh15UGY8/GeKyvzomEjUoOApjigEX2YDPvhINsBhY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BvEJaO/t7fLRI9ME+g6Nf2uKvBFzhDxeGVkHtu0GNM/cuRLsGbH1Dn5794sKOGhX7 t2neLuWSxp8mcJ6f0iqpeVhqi4qYZ8ahPfVqN2Da62LUO4i/t7gL4bmvdBqH64Llyn QkRpi8dmIk/b/s2jltgElLVpM3vEGAMoladx4fbNv3xbFj5qlDKciiWQ+lStLf8ju1 pub/Wz86s+HQWQ3Ve18yw7Zyy0EcvOkhR8zgpWpEnigIMCNNdI7HcIC4rkQ9U2GhTK L7bquPxY+IC1hy+VduUcpRaujnplUMl8dZNy79gV5MHb1aIEYQXoOGfUkVwIpLYaME gcTZZsxBtN4Pg== From: Jisheng Zhang To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 4/4] riscv: enable HAVE_FAST_GUP if MMU Date: Wed, 20 Dec 2023 01:50:46 +0800 Message-Id: <20231219175046.2496-5-jszhang@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20231219175046.2496-1-jszhang@kernel.org> References: <20231219175046.2496-1-jszhang@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: D425340019 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: y8b4axp4qb5ox7enkfj1fxp3xnqp3m9e X-HE-Tag: 1703009015-536020 X-HE-Meta: U2FsdGVkX1+ze6A+i/LG+2RzlBHj9xvBEqNBYJA9MtVJou4y/3TzPKW05s6vKD/i4/giigvVBImQ5UlG6MxxLo9hGIwYLlpJ54l3G8pC8p5li8jfxYeA5nVP+DAtWH2fZG3bwZ+ULcPgMulJme1Mv6X7wdyA/5atahB8sWnENtwl9PX68XZdXAb7GI319SpCnZGxlbd0hCxH30b+M1F9jkxyH3jDc5dKzHwxfhW6JnLTmtFopbi7T4wLt81FblfSTTxFJ/0fGXZv+aazAi2a3cJ30ajzIbdpTKUbo2DFw5Hnq4oTZN5cUI8FXs3c7L0omTY7SaB1SMBqBOc3w107Po35Zqj4YtDURniS6zmg//9/1mPkFICIrB4XpLE7hZwReFrrrTNwXDlxnJrKrky361fXFe0qjXcZ53RSVCRrcKw+I0jI4LL+PJQmww4/gsYnJRFtO50hni7d1SCn0HR7EuJoWM7vaa6qtGoHDHobCZHQY1UR8vr5qcpOObZNH1t6icTyM/HXLxkhaV5HCD0CKxF3M9DFsI7o0UPwMr0TF+rdsnmFVWVPYky+X3NkzD2m5pRJUGDa/f2wVd5adsH5n7ra4z0gRkvdTlEWqVongj4IBC/30ILhlO9ato4dR0ShZxWgcxcYgCPxQjMn8HLWvKNpj6ZndXjDUuNS/Dl0W6+iGnzQY6djt0SdF5nt+P5AKTxA30UO0UIJ75NUtGfWZRUftIn7of+FWlQLlPMBz/qP376x9jedTOiQtCeLlfk213dL82vATgOka4b5yTs2+vHBBn8/7VeVzYQHki9FJfMmy3I5+02X+DJoWvZIIDijXljLACh2DgBRnVwu8WmcIsMuo5vaZcFzCMkKFXznEf7SA96Scyqld2JuFL7DDOAE4IdL0Mhcdx/3+vApkUSSQECxbOvh728cpGfLH+zwD30EA2D9aQQXF0+TMTYRu8pgVYUujuuR+zRKQkT2Gps 8bZscLMF rPGo1dY3AfWlv4eebSD2OKr3rocL1oqrMMjjuNKOzvyBGkbTZ/1yjdRKhwj/8FJ1HhiIO37J/uzYJ7qPU76e4qC6Ewo3P18+w56knBCyBfzB7kTtod5JoUfXeozLeYDYbiMbeocC/5N6rt8u89O2oEgTungVngoWjCtcSHBeua64s1cbfTLFTrRfqvhHrg/Qs7j4cB/tI5l5aDXlTYr2dhUbqb3b4Io1dNa2Wvk9jNM6rgKrLhzFlavhYj8KdoonbT3xNkFbMgcPeFdgyDUs+50Q0Gwo8Ooy9nuta X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Activate the fast gup for riscv mmu platforms. Here are some GUP_FAST_BENCHMARK performance numbers: Before the patch: GUP_FAST_BENCHMARK: Time: get:53203 put:5085 us After the patch: GUP_FAST_BENCHMARK: Time: get:17711 put:5060 us The get time is reduced by 66.7%! IOW, 3x get speed! Signed-off-by: Jisheng Zhang Reviewed-by: Alexandre Ghiti --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/pgtable.h | 6 ++++++ 2 files changed, 7 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index d3555173d9f4..04df9920282d 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -119,6 +119,7 @@ config RISCV select HAVE_FUNCTION_GRAPH_RETVAL if HAVE_FUNCTION_GRAPH_TRACER select HAVE_FUNCTION_TRACER if !XIP_KERNEL && !PREEMPTION select HAVE_EBPF_JIT if MMU + select HAVE_FAST_GUP if MMU select HAVE_FUNCTION_ARG_ACCESS_API select HAVE_FUNCTION_ERROR_INJECTION select HAVE_GCC_PLUGINS diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index ab00235b018f..c6eb214139e6 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -673,6 +673,12 @@ static inline int pmd_write(pmd_t pmd) return pte_write(pmd_pte(pmd)); } +#define pud_write pud_write +static inline int pud_write(pud_t pud) +{ + return pte_write(pud_pte(pud)); +} + static inline int pmd_dirty(pmd_t pmd) { return pte_dirty(pmd_pte(pmd));