From patchwork Mon Jan 22 19:41:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13526133 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93D8BC47DAF for ; Mon, 22 Jan 2024 19:42:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1563A6B0098; Mon, 22 Jan 2024 14:42:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 107F46B009A; Mon, 22 Jan 2024 14:42:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EE9186B009B; Mon, 22 Jan 2024 14:42:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id DA3AD6B0098 for ; Mon, 22 Jan 2024 14:42:23 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5F8351A0698 for ; Mon, 22 Jan 2024 19:42:23 +0000 (UTC) X-FDA: 81707968566.30.9C60817 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf11.hostedemail.com (Postfix) with ESMTP id 8E5CE40006 for ; Mon, 22 Jan 2024 19:42:21 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="dgWT/Dnt"; spf=pass (imf11.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705952541; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vFTvJ98fhkS9/MTVBvrs88F9yaB/zvK8UHNYcTujcyU=; b=upL0sVzR9ZILoKd+Qj7hTt/dfrtk9iDuViIWA6Gq2BF0YP3BCpELDEGu8CIQ0y8nrDJ9// MCFwRc4KMz2TFcrqqhn9aDk5zasl6xVsh0hBJTdsFIXASJfMiD3rflCIumsgaLcACFsapb Qjww/Iws9sN7YyoC6XwLXaMgT9yQ5fA= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="dgWT/Dnt"; spf=pass (imf11.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705952541; a=rsa-sha256; cv=none; b=V3CbxENFNM4UsfbK5fS3UKHyN81g+YBFnnxmXIEikzgvrh1n9R4cAg11IsxcOo5KyLlaA0 E6wxiU71XOy+m6o8k61q9GcA2dBNiP1bIoF6YaOtvnpcPX7k+YrkoWLUyhLeYh9x3dgQQl KAUSjsP93K7mGe797M2+GYHsmDI8+qo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705952540; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vFTvJ98fhkS9/MTVBvrs88F9yaB/zvK8UHNYcTujcyU=; b=dgWT/DntW2iOcDXreeUYMJ5dQ+wcg76413atckZl5NERbxOdIxGBkcet+fovEqh8jDcrK1 uCRisibyWdEh7k/3lWYq70gOsFD+2T7Fb3/6KnPzduYUBAxnVB6V6azYIOqqkW0ylbOB4M UcNz3C4/LLA09rA6+KK3fL2wpj8LWx8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-167-RMuEdrELNYKNm7WJWua3og-1; Mon, 22 Jan 2024 14:42:17 -0500 X-MC-Unique: RMuEdrELNYKNm7WJWua3og-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6EBB6185A782; Mon, 22 Jan 2024 19:42:15 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.195.114]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1AAFB3C2E; Mon, 22 Jan 2024 19:42:09 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v1 01/11] arm/pgtable: define PFN_PTE_SHIFT on arm and arm64 Date: Mon, 22 Jan 2024 20:41:50 +0100 Message-ID: <20240122194200.381241-2-david@redhat.com> In-Reply-To: <20240122194200.381241-1-david@redhat.com> References: <20240122194200.381241-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Rspamd-Queue-Id: 8E5CE40006 X-Rspam-User: X-Stat-Signature: xrm6aoackd3t94wjhbze6xrhjtgub5tr X-Rspamd-Server: rspam01 X-HE-Tag: 1705952541-327003 X-HE-Meta: U2FsdGVkX18MiLE8J87cSzjBaeqJiOjUkPC1Rkzv1n2RgE+y5SosSjiSWL4+6iqorxVqfPXaImLvOVn7OGCHdhQo8gM9IEDHe4BXylfZFBmkn6AUORqkSo9cFds1z7KbM3IENIpWwMdZW0eu1FcIXQVmII0L1JnekNGtmF19nqhYF1n6tc1q1tdfcoBiTSKOBX69Av/blws+QFQQKohfEE/gcauSJxG/uTEM+oCQ+nrfx/UeughU0RcQMZJ1PUuvAvv/pMAN4n8BMYkkBD4jrTCyGST4PIF/6Bgh47WOoxRXNwDlmHYwz8TXcBF6nA/xnBXLlr/iW+112V/zBKszN4OXwXQj7XNq53I9+IWs1JGBhY6xjlyaWWQSmPm97ZmkrKsBu06/6Lb/NqqTN7oX0EejnGhcjq1lAecs+z5uypxY+YnD5m61kMv93t25hCYB9m98wYWTd8JW26k0xXnB0vWZaSDx3dOCzvfuEt/qytdeIsyEzwSDot7bbTWbfhraEamwwr5na0Ro8wXlWAaL0+CyYjSRKxvTiBSXdUAIWK7rHtZbCzdWXsLrhuW517G8ZBCl5+eeFMhYx5B4mDAvGG7Dc+Ifyno4BesHEglN/C/z8PqZE9+pawol4cOt8UOlE08hD0KAoyd2UwMeXYomfXqboBWXGwLzRyjDkZjCSOpWdAli9baYPz+iQw++MiLS9yyv/eft+gRJoKWfiUHgn7DL40oSs9aLR5NxCIAUhynRMukn2Yt1+WnfgrIirmm/wBIy+q0KX2AstPmh/4vmuEAGj8cZ/l+zYGIrzWGO2cv+okYzj2SJmKI0ip6vrJhjKDCUTNd1IOamrTgvoVkpUpEDdyT7Zpcudmgx4eG7G46trX/EYCfcwoU49g93Vb8QSQsQbP7iQUgm1XFkjVgkf9g9SxceLOx5SINPD91l4VXk0EMRb/Ny99rPhgpVfgh9PJO20/02qAm/aCejROw 6SJq4Jrb gHRNB8YYAJ9ra1RKhmiCQb2rVjMRkI6AasP55JBkxvzLxyXBG1B4gpI7QbOv89zcKxhTeiSPvfP6798xej0dQ3Z0lv2cV/pXyN8lSfemVKamR7yadX90gp550GAMjSLR+lfFKD76B2naDz3MhxV0JzTHhGqbYULjl6weMDjt85hdlNiC/8+nWaospsKFQdVxAM6S9LtIwhfFYOEiFjEs1PGLXduxL/CZ65YsS X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We want to make use of pte_next_pfn() outside of set_ptes(). Let's simpliy define PFN_PTE_SHIFT, required by pte_next_pfn(). Signed-off-by: David Hildenbrand --- arch/arm/include/asm/pgtable.h | 2 ++ arch/arm64/include/asm/pgtable.h | 2 ++ 2 files changed, 4 insertions(+) diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h index d657b84b6bf70..be91e376df79e 100644 --- a/arch/arm/include/asm/pgtable.h +++ b/arch/arm/include/asm/pgtable.h @@ -209,6 +209,8 @@ static inline void __sync_icache_dcache(pte_t pteval) extern void __sync_icache_dcache(pte_t pteval); #endif +#define PFN_PTE_SHIFT PAGE_SHIFT + void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pteval, unsigned int nr); #define set_ptes set_ptes diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 79ce70fbb751c..d4b3bd96e3304 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -341,6 +341,8 @@ static inline void __sync_cache_and_tags(pte_t pte, unsigned int nr_pages) mte_sync_tags(pte, nr_pages); } +#define PFN_PTE_SHIFT PAGE_SHIFT + static inline void set_ptes(struct mm_struct *mm, unsigned long __always_unused addr, pte_t *ptep, pte_t pte, unsigned int nr) From patchwork Mon Jan 22 19:41:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13526134 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0C2BC47DD3 for ; Mon, 22 Jan 2024 19:42:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6D3E96B009B; Mon, 22 Jan 2024 14:42:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 685256B009C; Mon, 22 Jan 2024 14:42:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4FFEE6B009D; Mon, 22 Jan 2024 14:42:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3B7CC6B009B for ; Mon, 22 Jan 2024 14:42:30 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id DD39BA08C3 for ; Mon, 22 Jan 2024 19:42:29 +0000 (UTC) X-FDA: 81707968818.06.07A537E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf21.hostedemail.com (Postfix) with ESMTP id 26B6C1C001C for ; Mon, 22 Jan 2024 19:42:27 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Tjm747rz; spf=pass (imf21.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705952548; a=rsa-sha256; cv=none; b=GdE2Zw/5YrSPtxJsFseql2owjTzNhb6iBBATXD6Czva6siesUA8amMdYCeXSh2Yp0U+6XJ yD/b0SXkXMYfhndw3U/qgOSIkwpSXayq36h6jduoCohXVKXfwTdkplZXshZ7euroiFEur3 hhXL3XFpogHkCieo11wctFXFFNxrskc= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=Tjm747rz; spf=pass (imf21.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705952548; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OA1ZFxm0ba3AcXcmS+8/ecOKI3aFHalZlUfREzi3hZE=; b=1Td19w6lrJWCyKutnqD5jxxQRL24yCcuUDgtF6jpPHrTiJZHxj0hzNJH1gkV7ddbBe+smO uwW4BZwynrwHVALMeDkEndzyWaN7Z7kGgN6gqRiWgxH40JJQMBd/52STqPfeukS2pPKq7+ gzdBShx3WMQ9EzsvOpBcVSznT12RH90= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705952547; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OA1ZFxm0ba3AcXcmS+8/ecOKI3aFHalZlUfREzi3hZE=; b=Tjm747rzCbeJ9Nvjzpncp3ayvHRQEAB7MowtNvha+nBWZ166WxLfnlopbUNKwOf/NGghkP WPfr76RouZcxrzUuAZUtVMwqqhOpDTtMBQXejOvFdb3a9cgObVIf2q/xUoRPn04emo3cZN Vqtlw4TmtKLbsMEh0xwoPh2sDu+AeSU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-66-LD2D6IcIPfyJ8rm49qezLQ-1; Mon, 22 Jan 2024 14:42:23 -0500 X-MC-Unique: LD2D6IcIPfyJ8rm49qezLQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 72494185A781; Mon, 22 Jan 2024 19:42:20 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.195.114]) by smtp.corp.redhat.com (Postfix) with ESMTP id B2CA03C2E; Mon, 22 Jan 2024 19:42:15 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v1 02/11] nios2/pgtable: define PFN_PTE_SHIFT Date: Mon, 22 Jan 2024 20:41:51 +0100 Message-ID: <20240122194200.381241-3-david@redhat.com> In-Reply-To: <20240122194200.381241-1-david@redhat.com> References: <20240122194200.381241-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 26B6C1C001C X-Stat-Signature: hghm49b1bcx5xp5e5qkssmb6jhrxejjx X-Rspam-User: X-HE-Tag: 1705952547-573411 X-HE-Meta: U2FsdGVkX1+/xWa3RakOMi26oeyCH9XsEKgefmXVd30+58A/ONjseoHqG2f3vXBBiJDNNvOjQ/omRCug133EgrowwKvxvWsajwx8ZA+G2DC7JGhttfbQ/IX5eVoSDI/YiAuPtqaPWuNN995greaEA2LPsp1yuWfQwYSm42eAXRGFU37/VSSlvsubPltKL9d40k9IpLh74xR9HON0vEMVXeKWoTD4C81QaUazM8DZK6+LIZirAid7Pb6l/4NgRuq/ZO1TEqtUr+HGhbd4e1asJNwUV2y1Bb41F0AkAZQMny2c7rRtGRME7gZeKlLFZG5AG3glfPURgkjQR/yDHh8zzo7cTJuhoM5DZEPeTiT+CU75g8dsi7N4ax/6ZX3GkEx3BpvsnHAd2c0ff1XuH0j6UBnx5MmlIOB68dA4Hl0tw8hWSaeEZFKr+H7OzsX+snUOV1VbrBz1P2gvbW1MzvlTjFJh5TZBOFfKXt3tzBsVsQhWSOb7uHcnIxnGIitctkAuaZ85tovRFd+D1IKMp6wHVgff7txvQcOUBZ/Key49k4WFJXdkGkMbXT3Hnj/VIUz3NaiCwwVAMkUqj+FlanuSY8H3xZQ1p/3809PGB5X6q3f2IQTjbLG6/EZTPNy6buhGVUBs3/AztuscX/DaiXXg8qWYctijYXRFZRbzZvOPYMe0dJIdZSX2b+Vzu5UNllm3xm+bOXfAvwIHu6ePYF0ywKEL/LN0mF20ZVl+uC94QQ/7bwstvge1ATF6AdEANF6GhT8Z6TfuOqpJAk7PW3OZHzaZ/xlpLqDR8YoGG1Qz6sF6CY+5NYdbNiZ3Z5JRZvOMeCBPKvU1OxGPD/rQDtULWnXQpwiXVSPSZWM6E8g75ciPbbC8AHFu/K8VnnAontsFxYLwD1PusBOQWk3ibAoAYNIv27q3FXZe7a67Vq0JxHVBaZoLLtzFbRpxeHIoYEhKAJbdKOwZiBsp/YdV3D5 lPEb1yGd Y1Xc4Oja06tLYNz1iXH1cpLRbMPPPqfVhYaP40E8ymg0Q9g8kpoYzMU5WQP6TyuFuE3bpCif6Pgi7YZUz6XUe0saM+Qm4sOTRYzzI1RTdsB457xS76KrxzQqDzQflym2i7yQRFYVw2SJP8lO1QY0hmag3jPETgyrlvxSqIk9XvT6OU5evuF/B9qZBVwIfOy3LOf9OuaPTfoVkw6x6jjTUHJinTjn5IS1PvhLc X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We want to make use of pte_next_pfn() outside of set_ptes(). Let's simpliy define PFN_PTE_SHIFT, required by pte_next_pfn(). Signed-off-by: David Hildenbrand --- arch/nios2/include/asm/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/nios2/include/asm/pgtable.h b/arch/nios2/include/asm/pgtable.h index 5144506dfa693..d052dfcbe8d3a 100644 --- a/arch/nios2/include/asm/pgtable.h +++ b/arch/nios2/include/asm/pgtable.h @@ -178,6 +178,8 @@ static inline void set_pte(pte_t *ptep, pte_t pteval) *ptep = pteval; } +#define PFN_PTE_SHIFT 0 + static inline void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, unsigned int nr) { From patchwork Mon Jan 22 19:41:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13526135 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF1D2C47DD3 for ; Mon, 22 Jan 2024 19:42:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 379EB6B009D; Mon, 22 Jan 2024 14:42:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 32A3F6B009E; Mon, 22 Jan 2024 14:42:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1CC056B009F; Mon, 22 Jan 2024 14:42:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 0461A6B009D for ; Mon, 22 Jan 2024 14:42:35 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 9F2A41C13F2 for ; Mon, 22 Jan 2024 19:42:34 +0000 (UTC) X-FDA: 81707969028.22.DDE15F9 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf08.hostedemail.com (Postfix) with ESMTP id D8D02160002 for ; Mon, 22 Jan 2024 19:42:32 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=SJk1ZGFR; spf=pass (imf08.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705952552; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7CvT91MHkJr0ioIf++HrmdTETO7/LeECE+2pAFvObkA=; b=YZkAYRs7P7WuhbC5e0Gf4wImpPy9tXfL94DRRes0vaBWPCIIWeOz8Iwx85q5P8jkhXXqYx s2OXymif8HcmD+PFxO1zwLe1NSpneEeEe3LHjnVqoaJzEamMMCb2YB4GibEctNQcR/6u9z h6SS6OTmxQgMEhegyGTwO3AmgvH5RuU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705952552; a=rsa-sha256; cv=none; b=WIs5RJ4qUJOSuZ0YgviPapocnrwUWsaUoLTNhBjRtsdzryWICWLSbgOmUxh+VsqQHpY65v rCfwqLgHJ9pf22AHonF1+1C/jb/YGm54qmHGCYvoenoOTWG48xf7ywTG2a1a707Biu3Ju5 7v0kKKCI/Kg5/sepUEIDshd6YPhRw7Q= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=SJk1ZGFR; spf=pass (imf08.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705952552; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7CvT91MHkJr0ioIf++HrmdTETO7/LeECE+2pAFvObkA=; b=SJk1ZGFRhcimqto1w9cvlaH/iewCDCszBtBCP4ayn0HEVvCQ8/j5LKqLXhcTiMn8oGL/jm uxXZuVuWgM8tbzqVpfO6bAtpdcKsjgv8YIBOu5XQiTWLhDAr5tNuGQAHY3Q2z4zHfQ4Wwu eQkhcTkSd+xrYXfrLnSfRMW59IcrLgw= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-79-mRFmOKJDPVaGmmWPnrJHKA-1; Mon, 22 Jan 2024 14:42:28 -0500 X-MC-Unique: mRFmOKJDPVaGmmWPnrJHKA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 634743C00084; Mon, 22 Jan 2024 19:42:26 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.195.114]) by smtp.corp.redhat.com (Postfix) with ESMTP id ADD273C2E; Mon, 22 Jan 2024 19:42:20 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v1 03/11] powerpc/pgtable: define PFN_PTE_SHIFT Date: Mon, 22 Jan 2024 20:41:52 +0100 Message-ID: <20240122194200.381241-4-david@redhat.com> In-Reply-To: <20240122194200.381241-1-david@redhat.com> References: <20240122194200.381241-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Rspamd-Queue-Id: D8D02160002 X-Rspam-User: X-Stat-Signature: j7spjp9rgfk39idrj5ubidpg6ez8u7ds X-Rspamd-Server: rspam03 X-HE-Tag: 1705952552-458302 X-HE-Meta: U2FsdGVkX1/5nGktqKEOswJ7BKIY2l8CkWocQnGcpekBtWP02+oHlaNNdsYUmNwYBr/R9zZB6M/Bac74ItBAE8HQvR4c/02F/+8mwaA1A/Biv5E1kMca2B2TYpO7aoHfMwLIMxZKQJfG9Y6S+I11b548JtuS/BEaKoGj4rSuolCa6oqw6vPImh4k7c865bWoit+NLECkMI+LcOrkuPqxaMGehMbGcaFzItAUvE6j9qIuDekoLSy0gnFHekbz7u7QHliB5KlneRlrPeCnEuskSWTDI52ER2PiO4GoFOuJcf8JLTeeh2B03FqesS4T7NaMESv8IypM3caD2NPXUIW/6/II3GUfPuG62sLMtlXzc560a91yqjqxo5q6SuNxAbEGKU9bttsQ9npj2j5/ANqDBRDqQRhOFwes0bIZnfQnvcaEApA2NlPJAEFDxMz4iQRJJPrNZeqbR9eSayz3YcC/pgD9s/r/3h2H8wzFsAyeifVgXqFjjCHzy7006ey8MsLajYLE7b5IS+u0LF//1YWKtmNdBhZPI/9IMRqy3s8+f8zAhoiJsugZ3JXJDz2RW8jMUqqwtJ8ZzcVYQu5ItREnkWw/SNlmwSaVHdQzHb7rKeNkRwDgRxoxWgjnW9YvKUgNmJVmJVHJ3d36g7dDdcfYr2is/5SZUzVkKSDnOb598jLcZFQRJ5kMkN5zyuhZk5y/vsgtVlniMMF+Kw8Y24yPgaRyVdfaioUDg6RIHFVAu4BXylBV8+u9xEGgSsIm8hIbJDYXmZNJQY3muy8z9vXVZcUkJdzso1kEtTwIwuWzFU1SGycoyDIQ5F1tdx56Upri30v+wl3hK0/wSF97o5JXO14HykT1Co87IXUJ8R+jcdSlaRNSVMOTVNTAErOcGgk+yyet9CiH8fqpVBlV8iWll462PlCxlK3Zg/4RFFcR3WCkRNszhfT3SBK7LGggRFjuv4Qd3KjalV/eSKBUn+S bMqr112n 2vY0bKHL4jxBXEvz9xV/yLoHWsR3GH1F3V+fK6uyONT28sJ/4Mux1dgCy0SFfs+dcbQ5haaEL/uWzqOMq5JoOXMJYuFq05lfoPomnYpKAbIaLIuZSbD5JnA8Pvc+ugtYmQy7awNS8mvwSfWJ+GytqzdjOpbfvG74t7kkBfHwHjJBxvhUAiq+/Wy5MeRYIWX9oR75apvdpNL8c6dZGscZB939tT5zBy5HBfz0C X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We want to make use of pte_next_pfn() outside of set_ptes(). Let's simpliy define PFN_PTE_SHIFT, required by pte_next_pfn(). Signed-off-by: David Hildenbrand --- arch/powerpc/include/asm/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/powerpc/include/asm/pgtable.h b/arch/powerpc/include/asm/pgtable.h index 9224f23065fff..7a1ba8889aeae 100644 --- a/arch/powerpc/include/asm/pgtable.h +++ b/arch/powerpc/include/asm/pgtable.h @@ -41,6 +41,8 @@ struct mm_struct; #ifndef __ASSEMBLY__ +#define PFN_PTE_SHIFT PTE_RPN_SHIFT + void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, unsigned int nr); #define set_ptes set_ptes From patchwork Mon Jan 22 19:41:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13526136 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38C18C47DAF for ; Mon, 22 Jan 2024 19:42:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C04D18D0003; Mon, 22 Jan 2024 14:42:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BB5788D0001; Mon, 22 Jan 2024 14:42:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A555D8D0003; Mon, 22 Jan 2024 14:42:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 90E978D0001 for ; Mon, 22 Jan 2024 14:42:44 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5AD09406B1 for ; Mon, 22 Jan 2024 19:42:44 +0000 (UTC) X-FDA: 81707969448.17.175E81E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf27.hostedemail.com (Postfix) with ESMTP id 648BC40008 for ; Mon, 22 Jan 2024 19:42:42 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EzEIuWdZ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf27.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705952562; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tOYdVx3Vo+Ft7G1OTsqYnhs+sWPYKb0IUhtnLA3JLTY=; b=wblw6uJlM42TOnIon8uY8lLzDbO6EhUNMQA7tjvBErJ1o/UqRJRU5UYa2hUIn8U/g/QrRO U59LpaPceAIdZKpJ5lrtg2naI+EF38tlfr+kZx15DlXWLbiAtf13wacmbHfFLQlpb6U3NG tUCq/txJaWONxgQiBVSJYiwD4bck1+s= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EzEIuWdZ; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf27.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705952562; a=rsa-sha256; cv=none; b=yiLPqN7QQbk7hwcu3a+eFKcHR13qO7tJJjW6ELgIpMNFFobNBT76kkepaHJhjfp41mhxRb 1ADbslwOlOAWAJs7XrHTMlOSCQzmCqmtLdkzD1Skev3UGaJbdHIBQWC4CMpeV0W1ODNxnR uQWQyVWsr+XYAmQV6/kGKK6FIu20/fM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705952559; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tOYdVx3Vo+Ft7G1OTsqYnhs+sWPYKb0IUhtnLA3JLTY=; b=EzEIuWdZEUgQpYuk2yTInH7diqBcgugG9x8UVXha9iEwgo9mPLqIx84NjvJPd0fk5y35rv QwhFR3IQSy2WJDt22BHz8wiN21dmIPkSohAD8W1wrVRw4lAjXG9+sG0KeOjfOkJUFzOM1U tJ27QPpmtfs4rRuxGA8jyXn0kFFPpBw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-283-U65XcrjPPBKjRfwTqL42NA-1; Mon, 22 Jan 2024 14:42:33 -0500 X-MC-Unique: U65XcrjPPBKjRfwTqL42NA-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9A01F85A597; Mon, 22 Jan 2024 19:42:31 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.195.114]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9F97B3C2E; Mon, 22 Jan 2024 19:42:26 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v1 04/11] risc: pgtable: define PFN_PTE_SHIFT Date: Mon, 22 Jan 2024 20:41:53 +0100 Message-ID: <20240122194200.381241-5-david@redhat.com> In-Reply-To: <20240122194200.381241-1-david@redhat.com> References: <20240122194200.381241-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 648BC40008 X-Stat-Signature: 7xze14x1pari9gxhyrqc999qmr3tpnj4 X-HE-Tag: 1705952562-601446 X-HE-Meta: U2FsdGVkX19s1QjVTwlbgBHeG8I2qsCeBwHu4L1ljPle1qj8q4a6WX6dAJLU7+EgKxM5HFdVvayh1s87TmIdGAw9Fe3kuWQ2lhR3Z3cwvQ4rHQ3+TwsolQAJzc7kQLKcHX7l0SsYIdoCuYSS15xFVFpJUXuanI55l7UTZhY020070Q/W2IuOGTTYSlHZF6dRmj2BmuqyNel6F7Sc+KCdXWwtDlgKtzy7Tfl/hegbOiZSUxbdFdL5HrWgWpIfftOfQ5CLiNsAZlTfFInLDDmwzQtAl1qh0lAaWJdGmXeeyHlmTflaz2dMeotIW6F6oi8+i8rHFFDm3QDs3ld3iaS2CYl5RZlsFXTYJZWKUYaY5Yd7DpMrCAq15IvvKMM6VySR+UT+UMdn+y7l/7ng/t5AB9xwLH+B8/LZX5jGpiWD8dgzNbc5snIzAATCdQ9plZ7YNbvPXmYF6TYSN8EaJZIIMrzjNOzz9X8Xbaz+1ywcPJJEWRAZJYrZGuh13akyDRsX/If9GaKV2UKvxpUAGObw1mLApcje01CpFmxnlJAA/0mSdOd2EPcBmwoAYK1gbt2Q7MbQcXWLn1jjZFt6dpRQZ885GpUO6sB9UA1bV+sn6+rsuNYNGPR5e1Mv0T/bOdfIz++0CK4rSOh5fn0Dc1t641VtSPeT23/lp1uL+eBUbjTRgwHUo5h9MZSXD1jkWBC1MJs5dZh0LJozFTa1pPSjtgJah56lseX18D/Mg8j381RLWk3bS8APPdHuic3LzrdKGJWOBRC+w3rJlIWgk2KYJ48+h0vVlfCRAd1huwKAgc623TWlO/igfGznPqOcukuc6GG37sKSqdqXblx0vXkOo2Ai0BP3C9iMQdaFGDpF7HuP9m1nFJXZpTpL1xFQu+4OlMIgTHkikQ4p8rleQr6JFHPCgD3m6sV9CKd5KAaTbU3U7P+xuKrbJjveebmQ9vjm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We want to make use of pte_next_pfn() outside of set_ptes(). Let's simpliy define PFN_PTE_SHIFT, required by pte_next_pfn(). Signed-off-by: David Hildenbrand Reviewed-by: Alexandre Ghiti --- arch/riscv/include/asm/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 0c94260b5d0c1..add5cd30ab34d 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -523,6 +523,8 @@ static inline void __set_pte_at(pte_t *ptep, pte_t pteval) set_pte(ptep, pteval); } +#define PFN_PTE_SHIFT _PAGE_PFN_SHIFT + static inline void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pteval, unsigned int nr) { From patchwork Mon Jan 22 19:41:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13526137 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B15B6C47DD9 for ; Mon, 22 Jan 2024 19:42:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 43DA58D0008; Mon, 22 Jan 2024 14:42:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3ED288D0001; Mon, 22 Jan 2024 14:42:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2B4928D0008; Mon, 22 Jan 2024 14:42:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 17D488D0001 for ; Mon, 22 Jan 2024 14:42:46 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id EF15F120681 for ; Mon, 22 Jan 2024 19:42:45 +0000 (UTC) X-FDA: 81707969490.02.B60D76F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf25.hostedemail.com (Postfix) with ESMTP id 41FC4A0014 for ; Mon, 22 Jan 2024 19:42:44 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MLNcA1SE; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf25.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705952564; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vjWUYiHAoGZrNfW9Y2JrmbsG1L8Q45EKDO2/2fqZPoM=; b=v7564LiBNTtx9bRBmUueKjBiSGq3EpMygHP3eJx3UxIPa+MUTsZhdD3GCnrDj/iFtgV9Yu 4V787I66ktxze2lrnuABNZIGAQgRmUDq3Hdf8AnIlSSbrm2FW9PCGZG/alV1XWmQ6zk5vh HF4Z3q/bTAYNtoj3V+ws7/vdGUOqxRQ= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MLNcA1SE; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf25.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705952564; a=rsa-sha256; cv=none; b=DiUsrDwAGcFaG5mQ8KhS6NwSBjjgb99Ur8cJgnzRN7IUPEmuLXC07mYL3f7OnwmZ+3cONb wqCY1mY1buc7g8vm0H3qwu8e2le37AGmfvznEkjHOWzd2njXd9YnmTRVohr3w9GCgUE1Ok zE4YC7rMnG59Dhv+BzIKjprhAYklk84= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705952563; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vjWUYiHAoGZrNfW9Y2JrmbsG1L8Q45EKDO2/2fqZPoM=; b=MLNcA1SE28TAlewxXojpcRQRVAShXCy+NSKKghUYLADHWwtTFLWhlysrFF00j3nHfAN1kH 3l18u+1uorYSEay/hCUJBBSj4GHdOGqnAZzMJhcide5HzCSJFz9N+WrpDIV5SzvK+UAtse BYtnh5HvcpzywAQLHB40Oq+oCIypMhc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-424-qOBFwd40M36O2u6x9q2y0A-1; Mon, 22 Jan 2024 14:42:38 -0500 X-MC-Unique: qOBFwd40M36O2u6x9q2y0A-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B76F3106D1A8; Mon, 22 Jan 2024 19:42:36 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.195.114]) by smtp.corp.redhat.com (Postfix) with ESMTP id D5C393C2E; Mon, 22 Jan 2024 19:42:31 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v1 05/11] s390/pgtable: define PFN_PTE_SHIFT Date: Mon, 22 Jan 2024 20:41:54 +0100 Message-ID: <20240122194200.381241-6-david@redhat.com> In-Reply-To: <20240122194200.381241-1-david@redhat.com> References: <20240122194200.381241-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Rspamd-Queue-Id: 41FC4A0014 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: qw9kcd9bm9tra8uhb6c4eqe3z957uf3g X-HE-Tag: 1705952564-144241 X-HE-Meta: U2FsdGVkX1+F7mwLR/mcZLUQIKj+yRS4OUdFFCy7CicnaxSIAHlMbdT/gHRNHbDgmvboAVJcDvYaZURDWhCtarQrVhu75kl2F/IJyB/DXo9wzWIzuSaCDxOnGMa8sJcLPAuAis4vXlkRTl6Tj/6QQe/gjYyubeWEz07Hf+Cs4+ykWf3Hn1lsy6MP+gcL8sY4BKIoVAjtPEo2Q5qqlWwQP2/65dRmTqzdas+jr0DvMCOI4A48t/FknKpcfSsZnp8Wirau+IZmdO3PhXCRb5j9MeD+6VLOor7vXWyZ/JG9Re3sseeZ/n50gFz3gx+9avWpLnBJ8JB7m4DPlXbKHBu7q9JsD9TPgfcJ7okOMgBTW3zrwSeR17hOqtf24+2u/Zjoc/2yj05DizaAjm11SDc6j2+wBrJ/S5Mtt+WeO4nCxIyoNUMK4nmrRHdE37nTVE8aUuMv8deEj+JtlG5m/b1nrAXDIYPTgzVVfmw6nCaYBDcamOuV+FTZ7FdWFfQIO1VSoxcCkmHB92OHqh/KGeqawo6wTmPei0me3F3exhtNpj8KPu320O7S8R1yipx0cdPzo+MLnVXcfJnDiLYiifyd6/rzRNdDS0RGL+//S3rQpJiYb0NB5cLovGMRNKOyb2pKc6vhIsa2N6HWPjreuuDc+ywkzQmEN1H9YBeQVcXIQse3+Ig21xktlQ+x8+5VunBGevVJSgBRI+EBWFnPqAqLkA/3nJdLqECqcgB2bRUIyXKn8RNr4O1QJ5STJOW5eFJWClSc/ln7emTl0B+Xbc378j1MzdJsaqtJYxbfX34ShoAaGL6Msv0f8Gm2RqrQnTodMo8ww39UOuoR2cWIggkYy/mhCr8g6HtBVZZbdGIgl8Mu/rViaJNQcoH5K6ZkQUeqst0HSQtYiuKtmZCw4DhURdkDH0CTHcNDYxCzDo+aitGEtyXdD5WTSRWBluLAUZaDX4ge3bYoUl9NmaIPKQQ jiyOUMyq 2DmK5uKiMtENxjuX2zUgiiigy6xvpUYcnrVjm+xYLZuq/7rnSoxs+8Qbwkgn8Wl6RDKDQ7Bb5/X2DrTdOaoqQZr7SK+7QmZxzIUtY5tcYuzJajQCXGsWtLbX69QHP8aZGBpxwj8T0/94RBjl96GRZgPlX9shcjSGu8fx5khcaJj7kk8mmUzW1RreKhVpuIMxaMxd/Oqe6tjshwhuMtQld1S64G3DlDJ0uHhxh X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We want to make use of pte_next_pfn() outside of set_ptes(). Let's simpliy define PFN_PTE_SHIFT, required by pte_next_pfn(). Signed-off-by: David Hildenbrand --- arch/s390/include/asm/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h index 1299b56e43f6f..4b91e65c85d97 100644 --- a/arch/s390/include/asm/pgtable.h +++ b/arch/s390/include/asm/pgtable.h @@ -1316,6 +1316,8 @@ pgprot_t pgprot_writecombine(pgprot_t prot); #define pgprot_writethrough pgprot_writethrough pgprot_t pgprot_writethrough(pgprot_t prot); +#define PFN_PTE_SHIFT PAGE_SHIFT + /* * Set multiple PTEs to consecutive pages with a single call. All PTEs * are within the same folio, PMD and VMA. From patchwork Mon Jan 22 19:41:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13526138 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A122C47DD3 for ; Mon, 22 Jan 2024 19:42:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DDAD08D0009; Mon, 22 Jan 2024 14:42:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D63418D0001; Mon, 22 Jan 2024 14:42:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BDC918D0009; Mon, 22 Jan 2024 14:42:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A9A448D0001 for ; Mon, 22 Jan 2024 14:42:49 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 7F30C140627 for ; Mon, 22 Jan 2024 19:42:49 +0000 (UTC) X-FDA: 81707969658.27.24AED3F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf03.hostedemail.com (Postfix) with ESMTP id 80E2820008 for ; Mon, 22 Jan 2024 19:42:47 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=QLq3fNNr; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf03.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705952567; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=i8imtZo6vz5POOPfjS1eojctB3SqsK1V0htYi8JXsE8=; b=Qm6NpmeJIfyMiGVHb/b2bIdyRRGFVTmUcNsLCKBZqhZlPj+zka8M9bdVeLOLBItAzN1keh rddNLE2ONoCrmOG8TJGc98q4734glxU2Lsvxbs8Hm6XXppLar1QuEQxtRECAqhZBLVOcry CejVHm7ppQCdjS3N8r+UiDCHLZc06m0= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=QLq3fNNr; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf03.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705952567; a=rsa-sha256; cv=none; b=iMk95xk+zMmmlG7T31I0EV6r/1ohu1A+D2Roe9zCSYwUAMF0E892PDFEIf80T75MJRTX2u A4TeQKO6At0pLwekI8yvGlcUxfjwx5oCfniI44/pjR06C5NDKibuyA0eumapFD7fSQKkaT 599cNRTmLA3RkSIZKEFyRIJocXm7AX8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705952566; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i8imtZo6vz5POOPfjS1eojctB3SqsK1V0htYi8JXsE8=; b=QLq3fNNrs23wtoTAxMg2qvuT6gFwZ/3SearF/qbUqiZWhT6Q/RS3lAKbqzVfQqoALqU9FA OSwKqkLBiZVkfNEMe5mCC1n47WAUNb7b/XEHxTvMf6XSY3xlB5MOv4jklQM0teZSsV/QPL JK80f8taiwWU3C3BV24+0c1r7mbqxn0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-451-DdD_mImzNUSPGQ0oBD5LBQ-1; Mon, 22 Jan 2024 14:42:43 -0500 X-MC-Unique: DdD_mImzNUSPGQ0oBD5LBQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7920E879844; Mon, 22 Jan 2024 19:42:41 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.195.114]) by smtp.corp.redhat.com (Postfix) with ESMTP id F3EE13C2E; Mon, 22 Jan 2024 19:42:36 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v1 06/11] sparc/pgtable: define PFN_PTE_SHIFT Date: Mon, 22 Jan 2024 20:41:55 +0100 Message-ID: <20240122194200.381241-7-david@redhat.com> In-Reply-To: <20240122194200.381241-1-david@redhat.com> References: <20240122194200.381241-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 80E2820008 X-Stat-Signature: kadrko1hboid8gtsw5axsd3zp1hjd8b6 X-HE-Tag: 1705952567-604871 X-HE-Meta: U2FsdGVkX1/5rN7bsFwS/TnZuZFbj+wcm40PO6gdFftJ9fDpCOqhmR5JqOXkJbnXIAQkNyEDJ1a7oYCA2CUeI3X3txfNJbh9vxKt++6FbTjzXroT3Adw7oG/RL/RGA2yCO0mES09rxtnuWHuyIQ5x/V1q4XwTLYQOFIb9qBMqsOUdZrhsjyiT+2yq72v18BhaKZijHNjcr1ehbIbRW9TCdVTi6wYTAE3BNGwSlfZrdRy5rS7DlNpyk6GgBxzCMr0kCgfadDtuxtE+SFhkuNVhH5qSfaR3aBZbJx5lverR5oehpkZUGzADOTn3XrKXNuKdcNL4VIJVLsGWbWOXf5z923tqWZrjRrQDk+lDHI2PZhmQUcozPMiW8uvgRrTJTMI45r3Mo9t46pc8EU7ZJaUTgY0G5YtanyB7GrM+x1r/BkcBHaAW+/t03oO58dZ0pIMU2nhg+pWgtJ22UtSYwCq4D4i6jQtutQ4mhNgUfYwaXPopcSZYQWJvq/SAxjfhyj2nzT2bmLMmuK6+dkQz5jgl9dkXq+oTiW4UDAVFzzJzQwR5BW4ZOmf6WcJVrArzwrHpvIPQY3B3E8wbm4otSPIm1h/hKY05RJU5iCdPXaqa1fRYHL9isg+jRxwq51W3SA91ZKXWiJvTURgwoKr07BZPG8XzuMaOV8YSKv8dYurgRVEd2kKQeypLA5v05vDjShA1XNeqE3u2ZmEZvkCHOg1yxDOf1ogyAgkBJWKE2PZWEWatvGU+Ou16ZUxyt6IqNluhgeMcmMXnge2uT4NkF7IE3QkZD/+O4ShI0o3GssrUzlwdsJCFqYlS+jwOXF9RjEWDd/JyycITejOpgJd0EID/O7na+iJml/N6D2tPZd6oo0dpRHT8qTNqssSPCZEQMDebAnEENsUIqHQ3oRY9gT3r7KJ5KRyOVXAuyHJ3mlwdMbJGdOf1s9xBL0WcvtS4rv+oFf1lxt8ioS7unn3Mpo Y49JHPhn +G15Qk92A0DIpaOp/sOhmKAJxdWL2jxyEEtS7IrY5FTlZoM2Hi76y/UG1PtlIUrtHoKg5wVRl7kmSbJcERCtOyLeo0lxO0wYtYPuo2Zk6qILgs+0Kq4BVWuVDxZFO7iLxDqfjjDAj9QeIRD/A5b1B4q+coIQOrysDV8brf++O9ucIXJLjw3LKX2aj9wJpn+0DFDR3W0BjvVZC6tU5cYABCwInobmjIeYuCFgj X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We want to make use of pte_next_pfn() outside of set_ptes(). Let's simpliy define PFN_PTE_SHIFT, required by pte_next_pfn(). Signed-off-by: David Hildenbrand --- arch/sparc/include/asm/pgtable_64.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/sparc/include/asm/pgtable_64.h b/arch/sparc/include/asm/pgtable_64.h index a8c871b7d7860..652af9d63fa29 100644 --- a/arch/sparc/include/asm/pgtable_64.h +++ b/arch/sparc/include/asm/pgtable_64.h @@ -929,6 +929,8 @@ static inline void __set_pte_at(struct mm_struct *mm, unsigned long addr, maybe_tlb_batch_add(mm, addr, ptep, orig, fullmm, PAGE_SHIFT); } +#define PFN_PTE_SHIFT PAGE_SHIFT + static inline void set_ptes(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, unsigned int nr) { From patchwork Mon Jan 22 19:41:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13526139 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 060D2C47DD9 for ; Mon, 22 Jan 2024 19:42:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8ABE88D000A; Mon, 22 Jan 2024 14:42:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 85B218D0001; Mon, 22 Jan 2024 14:42:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D3ED8D000A; Mon, 22 Jan 2024 14:42:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 599748D0001 for ; Mon, 22 Jan 2024 14:42:56 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 1DAB1C05CE for ; Mon, 22 Jan 2024 19:42:56 +0000 (UTC) X-FDA: 81707969952.26.4A3EB82 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf28.hostedemail.com (Postfix) with ESMTP id 4EBF6C0017 for ; Mon, 22 Jan 2024 19:42:54 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=dGxnKNQD; spf=pass (imf28.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705952574; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aezdbNXr3jeNg/oq4Ir7c4pnUJMPrBPlV8tZyKwBKMo=; b=NUboHf+nmtbSDNjezcwRIra84kNSnzUVH1Maxl4ff5aERosm7czwg3/Bkv1cLHcaXlwxyK ncRSCApDbwx7oMPHwBoHScMEnaaaO30v9qmqT76XhVUaCfL8zJRpMJ9GBsAjyamaGnMV85 mk9A20wNcOMe8RTe3ztfZ2eG4jKtxBU= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=dGxnKNQD; spf=pass (imf28.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705952574; a=rsa-sha256; cv=none; b=uDj9l9mWVuEWgt4KOhoIhWdkizWxfHclDAcW89tLp5DmGKFkIQDp/5t9lmMQjS57smSSto 96mT0M70yc+WIbclmXcPIjImfzgIB4CLgrFboQ68SKP/5kvQsvjBj2jm0f20pTipQ0bU0q peWyBMO2KWXMwUJwiHcR7bskk8JOo0Q= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705952573; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aezdbNXr3jeNg/oq4Ir7c4pnUJMPrBPlV8tZyKwBKMo=; b=dGxnKNQDZ1ziB7IgcohH5Y6vLcNBnB/+OSCVmMuFVa7fj/Cy1tomKhCcymXtKpxWDTgg6A PnI2pbaZ0cvqqTPnGdirV5uWU+cK+epwg6c73icpb3IQsGSAaYXPydu44xQhBdk+CsoAsC OlkU3HvCZBOFE2fk3PVlojpZOXH3KUM= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-284-dLhPkRjVM62_BryuXLzoyw-1; Mon, 22 Jan 2024 14:42:48 -0500 X-MC-Unique: dLhPkRjVM62_BryuXLzoyw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 175DA3806063; Mon, 22 Jan 2024 19:42:47 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.195.114]) by smtp.corp.redhat.com (Postfix) with ESMTP id B54363C2E; Mon, 22 Jan 2024 19:42:41 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v1 07/11] mm/memory: factor out copying the actual PTE in copy_present_pte() Date: Mon, 22 Jan 2024 20:41:56 +0100 Message-ID: <20240122194200.381241-8-david@redhat.com> In-Reply-To: <20240122194200.381241-1-david@redhat.com> References: <20240122194200.381241-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Rspamd-Queue-Id: 4EBF6C0017 X-Rspam-User: X-Stat-Signature: s5e4ke6i7bgg5wsom4iw4ae44kgcdmb3 X-Rspamd-Server: rspam01 X-HE-Tag: 1705952574-194439 X-HE-Meta: U2FsdGVkX1/6vJFiheIMp0C3fa5nLXAYjTzAJL/ZNytIyhVje7B6QH9K9w7KmnJrAYv/cOQrEKI0XfUIV6UkFjDpKf7KFa8a5jNv54IU+gRYG25/CGwNtliwwYJb9OctRadC8E/FZz+Aad5lxsyPe4HUyn7LHPGjf5R/U/cMLOyN95g8aisgbe+soeP9hDZ2PzAFFPb09qfzuKw1UOJxUh+hrXEd0MwKIr6cGnDV+t9MjNi42ulGOLfJf4K7Ibd5Gxvm5apWBcT9gNpBcNwzLoCiFVAxVrJFn8Di0IhUkyfcYXAgWsGqWDi4NIBhqEXPnHdCbFTNiObstwf8XJ5ayavUEMIRZCCrM0VIGmQKy2SSDFcBxzqIzfd3rth2qxAMS5HtknLS8s5ycMGqIs/4Pf48yHOQKAm7uXPFY5niCXnIBR1OajLW0OmSmExv4cg2wzwWMTq3xs6cIdzLnZT8x+c/72el6qm7DU3EYtfwjoxOk4rr3dXQz8CG4RjJdHezr2NZL8pBGjbki0LB8EfQAiOUx/xP4TJKEkwsjtO7ngfzDqEVWNBzNiL39N0IzzWwrlZwVt7IpwQbsWt2qvm+AriT38Hyz3+ZUSZ8Eeu1bi562UGZUYdlQjpTOc4walRx+NFOqiu7nrdsNvJTnFSXN7r2/mpYPes+gDXum6vdJgfZLyonPP43vWE7JKpA1PPegjxJ9DXZ8iNQr2C4mLzGvHx5j4rpjvp4VdZW/fow8CVKJ8FBFvWc+ORqVp1yaG5WKAa0UojGODMURvRSVneEb/v9f8AhzGFiID9rSQvFO+laHFkXV6Ng+NeS3EO7Qp2OiU2g1YJLgPumwi0+dYjCZrHh7eMDYSLP50Jj7uzeLE6EFFf75BAfSI/v+z4gWH8s34+Q0wdwD8OIbg6Vf5TBqSdS9S9xfb8l+byDitulwkaVIqhbzfRLCXViXbL7cDvSqp2NRPHVgS/az5EeIVi ktG6xeOH L1I9GKQoefyoLzERNYA49gc4rTz7iADFa+oY8mIV2GvAJ/nv+OgV5VULop+lrMqKUu2jeyTIWA9b2tldBTvmRapHgkmoikLoanfSZFo3Nxht7zpZ+yAT8w+6tgBacWZFwI6LfcNMUIFuYp+N24YG4rVMdjz3yy+xMcbokgryamomHiWsxpY5rhXAi/sqEh5+q6TEH+C7JGpf9WPEIdZMgShQLsKfwmOrYv58V X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Let's prepare for further changes. Signed-off-by: David Hildenbrand Reviewed-by: Ryan Roberts --- mm/memory.c | 60 ++++++++++++++++++++++++++++------------------------- 1 file changed, 32 insertions(+), 28 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 7e1f4849463aa..2aa2051ee51d3 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -930,6 +930,29 @@ copy_present_page(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma return 0; } +static inline void __copy_present_pte(struct vm_area_struct *dst_vma, + struct vm_area_struct *src_vma, pte_t *dst_pte, pte_t *src_pte, + pte_t pte, unsigned long addr) +{ + struct mm_struct *src_mm = src_vma->vm_mm; + + /* If it's a COW mapping, write protect it both processes. */ + if (is_cow_mapping(src_vma->vm_flags) && pte_write(pte)) { + ptep_set_wrprotect(src_mm, addr, src_pte); + pte = pte_wrprotect(pte); + } + + /* If it's a shared mapping, mark it clean in the child. */ + if (src_vma->vm_flags & VM_SHARED) + pte = pte_mkclean(pte); + pte = pte_mkold(pte); + + if (!userfaultfd_wp(dst_vma)) + pte = pte_clear_uffd_wp(pte); + + set_pte_at(dst_vma->vm_mm, addr, dst_pte, pte); +} + /* * Copy one pte. Returns 0 if succeeded, or -EAGAIN if one preallocated page * is required to copy this pte. @@ -939,16 +962,16 @@ copy_present_pte(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, pte_t *dst_pte, pte_t *src_pte, unsigned long addr, int *rss, struct folio **prealloc) { - struct mm_struct *src_mm = src_vma->vm_mm; - unsigned long vm_flags = src_vma->vm_flags; pte_t pte = ptep_get(src_pte); struct page *page; struct folio *folio; page = vm_normal_page(src_vma, addr, pte); - if (page) - folio = page_folio(page); - if (page && folio_test_anon(folio)) { + if (unlikely(!page)) + goto copy_pte; + + folio = page_folio(page); + if (folio_test_anon(folio)) { /* * If this page may have been pinned by the parent process, * copy the page immediately for the child so that we'll always @@ -963,34 +986,15 @@ copy_present_pte(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, addr, rss, prealloc, page); } rss[MM_ANONPAGES]++; - } else if (page) { + VM_WARN_ON_FOLIO(PageAnonExclusive(page), folio); + } else { folio_get(folio); folio_dup_file_rmap_pte(folio, page); rss[mm_counter_file(page)]++; } - /* - * If it's a COW mapping, write protect it both - * in the parent and the child - */ - if (is_cow_mapping(vm_flags) && pte_write(pte)) { - ptep_set_wrprotect(src_mm, addr, src_pte); - pte = pte_wrprotect(pte); - } - VM_BUG_ON(page && folio_test_anon(folio) && PageAnonExclusive(page)); - - /* - * If it's a shared mapping, mark it clean in - * the child - */ - if (vm_flags & VM_SHARED) - pte = pte_mkclean(pte); - pte = pte_mkold(pte); - - if (!userfaultfd_wp(dst_vma)) - pte = pte_clear_uffd_wp(pte); - - set_pte_at(dst_vma->vm_mm, addr, dst_pte, pte); +copy_pte: + __copy_present_pte(dst_vma, src_vma, dst_pte, src_pte, pte, addr); return 0; } From patchwork Mon Jan 22 19:41:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13526140 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 072D1C47DAF for ; Mon, 22 Jan 2024 19:43:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8AA148D000B; Mon, 22 Jan 2024 14:42:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 85B078D0001; Mon, 22 Jan 2024 14:42:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D4FD8D000B; Mon, 22 Jan 2024 14:42:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5A92D8D0001 for ; Mon, 22 Jan 2024 14:42:59 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 38C43160A74 for ; Mon, 22 Jan 2024 19:42:59 +0000 (UTC) X-FDA: 81707970078.24.AED35AC Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf11.hostedemail.com (Postfix) with ESMTP id 8BCC540007 for ; Mon, 22 Jan 2024 19:42:57 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=XgXhB8ub; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705952577; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uF2bxfDUAnNambtI6tdyCXd/UbNqDVLKUcWAFsWTSNU=; b=dmivyvnZw76CgR8EdhibFddaanFtG8ouWWYXaw8hhdWOHLhWCg0vcDaPoijDSP1y6sLwqn uN6PadiAKZkd4I5YAHarEfhCc7C4Ii3bX3GoobtrsmALnoQ9ZdGWl3Asg7xU7a9m9qqmE3 TTvQTniBkHEyvPVUdIiUC1iI3OMumPY= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=XgXhB8ub; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705952577; a=rsa-sha256; cv=none; b=oTT6aAWYxcYHCAB/VOUrebcw/ERKvLZp08vWvcbEy3U6zXqtlvo1BldmgemDvWL0dTSszC j0RXl9nfKeUR1iXw19GctG6J8MbZWxAe7FMVqqVnLYd3+syP+LKfR31aHrCHVulsEu8yBy l6Jsw+VE1w+oGHGg+Rb0kh5/EPAvprk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705952576; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uF2bxfDUAnNambtI6tdyCXd/UbNqDVLKUcWAFsWTSNU=; b=XgXhB8ubseusFXcQsSveJa6ROP1ah1KI3wc9TlwMgae+3DLknbPQDB5YY0qGnD+oj+EoDq p1fjdrv6o2Si1sUeBFXy/XDwj+TvJTS9whqASSm/1h7/UxMSiczq1MkFFnyoK2Y1R9ir6R jmE3OnxsMDLglDu69/N879JdWaAodw8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-668-81vkS9EYMCG6II-n3hayug-1; Mon, 22 Jan 2024 14:42:53 -0500 X-MC-Unique: 81vkS9EYMCG6II-n3hayug-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 055AF106D1A6; Mon, 22 Jan 2024 19:42:52 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.195.114]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7D396488; Mon, 22 Jan 2024 19:42:47 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v1 08/11] mm/memory: pass PTE to copy_present_pte() Date: Mon, 22 Jan 2024 20:41:57 +0100 Message-ID: <20240122194200.381241-9-david@redhat.com> In-Reply-To: <20240122194200.381241-1-david@redhat.com> References: <20240122194200.381241-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 8BCC540007 X-Stat-Signature: u3nmesot5ngp6uscoy56gaftsgiibdp3 X-HE-Tag: 1705952577-140170 X-HE-Meta: U2FsdGVkX1+/nL6F8TYv1aGfIXOIFrJootFp1aZRPrmTkltN9vNkKrjtGshs7jUQRgBEMlyknaRfKqO5Vh0lzFt0vdrb+Ltk/f7V/r+BiQy1rPrkBK9UoXpVYeXts5RbaWtk9g4da8wIfBvtxeZ+bjk2H+rPDqo0pf7+R5ZDJWBXPFiiHiZeGBN4V2OXfrAZN80l9aYYSSyQSBnyv4OVDMz/D07OlOzHKsMo+677eL7qSUYH01c5mJXpE7TkNEYUVvKcBPtdetTHibKvMezqK8Q+ktVLG74HTqge3zEyn5pI0oBXGizmHheri4dpm3TYdHg5vUwYzpmTcX0tYWxr9I71xjyhcNrPscZR8naQWNcHSl2KyTcSXLpsJicPlgvcRMGrsFf9J27pCnIfMvLSnQLh+De75w2tSAzrCyeXasdlckx0Uqv7LgykovmNUdAvPiznOug4s4BAiovPwZK/zZmuI8bLiw339QfFTCX81aTk2QsMr1AYRv2k5JVOGr5j7JiRTRuqhv8y8dpoEQBO5YXmzj/eV03zgYE/DMU2nPuglwIwAYXmZy4+z+ovaQfae2O7U2EoxlYNaDx96F43doxSaXR6fdaOQt8BNl+Lll4aqji/0K8UJ1U5YnCCf0VgrdGu0yzA8x4BTYanUcIwaqDRbYJhTb4hxGJdMUPbna9s2C4eww2iGjxcfqWfw06LgMaF2z5EPz1RdItJ6/IaJBrJ8tcl4IrcyS1chUcdN7/8H7JX0uiQfvMlBhu+jhlaxZvGTSL8f0DosEhjM4EPYuhkojVW26tqd2OZqVBD2HcEsyRi6EwzhZe0+ucyn+QRY/OzARyHele4d6va7qW9DmvdIh8cYAUeBZzD4lkoFbJaJVWFoiPQBBTejybDfs7keUbETVAMFtxQyoOXJ09iTJ2m5KcI3T8XkFlOjSk84FKFcp4jRZ0xDB7XgAIE5ybWOeVVjyap6iUrYksk2QY 4/qESGNq fn9Xh3+KbbM4WXjqGFsRh22YGB0AGYtXCdwncJODHFFmgFbt06VJp54rQIXXMB2JLEIsYGLoA6WNIzTGY0WURAaeD6anHLv9SiIsib+C9ryGQdLrK4t7k+zEvMdWf3QXCQkgXeEywSDsWGcK85spdUiDqZ8V1LrFfaMdbx/cF6MFZWPTZbx8hviqLrRjqdouI4RE072dKsbbpEcvpzn4ZDz2YTyAab3sREIuv X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We already read it, let's just forward it. This patch is based on work by Ryan Roberts. Signed-off-by: David Hildenbrand Reviewed-by: Ryan Roberts --- mm/memory.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 2aa2051ee51d3..185b4aff13d62 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -959,10 +959,9 @@ static inline void __copy_present_pte(struct vm_area_struct *dst_vma, */ static inline int copy_present_pte(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, - pte_t *dst_pte, pte_t *src_pte, unsigned long addr, int *rss, - struct folio **prealloc) + pte_t *dst_pte, pte_t *src_pte, pte_t pte, unsigned long addr, + int *rss, struct folio **prealloc) { - pte_t pte = ptep_get(src_pte); struct page *page; struct folio *folio; @@ -1104,7 +1103,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, } /* copy_present_pte() will clear `*prealloc' if consumed */ ret = copy_present_pte(dst_vma, src_vma, dst_pte, src_pte, - addr, rss, &prealloc); + ptent, addr, rss, &prealloc); /* * If we need a pre-allocated page for this pte, drop the * locks, allocate, and try again. From patchwork Mon Jan 22 19:41:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13526141 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83228C47DD3 for ; Mon, 22 Jan 2024 19:43:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F3F28D000C; Mon, 22 Jan 2024 14:43:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A3A28D0001; Mon, 22 Jan 2024 14:43:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0447C8D000C; Mon, 22 Jan 2024 14:43:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id E4CB78D0001 for ; Mon, 22 Jan 2024 14:43:07 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B06B4160A74 for ; Mon, 22 Jan 2024 19:43:07 +0000 (UTC) X-FDA: 81707970414.30.7303AED Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf20.hostedemail.com (Postfix) with ESMTP id CE74F1C0005 for ; Mon, 22 Jan 2024 19:43:05 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="G/hBpqwX"; spf=pass (imf20.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705952586; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zE4dQFotU52w5fdTyhT0mbQikQr5RVENH6sHZzse5Ds=; b=labWHD2eZrDV6pz52hhdlMv4lMHXh52mowtMo9pVJTfgpz+OddK8RLqLrMbNO62WASee3j 1gepE4iHgePIR6zNOCqOvc3mjybRlVOGgP76mirUy0VX0P7YIUhFk2jLzDgqx+T4mY3t+f la9c6yUW3nLeScXx822bC9Zhmcf3W70= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="G/hBpqwX"; spf=pass (imf20.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705952586; a=rsa-sha256; cv=none; b=jMDjQ0cU7jLDE3VjTBNvSHb4YfpiO+H90j8MkZHCGpHr4euTbDArX9catK12ggoW1/GT/q f5f8LeOVLKMCrDkNHZ0ushf2XFhKqhtviUmOFC7T4xXxKXaHFMLjudPpcGcOPDGV5aIQFq G/btEFDWEj66qX+xOEvqNhiG8Tn3dyU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705952585; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zE4dQFotU52w5fdTyhT0mbQikQr5RVENH6sHZzse5Ds=; b=G/hBpqwXyT04TPg6hab+88o9y5Wo8YWEsEWQjEMKzT+4J97MY0vy9KuiGufVRZgBCbmfY7 IwB11X4W8sM+t6+bRruokTAE9Vo08HbdW5sIhS0LRg+AfbANpaSFT6HiTdP9ao9vO3DnG3 uTc1voSJLpqNLAv51NdGZ7eXKYh+DC0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-515-qUji4n2QNQWDdxOalZIvKw-1; Mon, 22 Jan 2024 14:42:59 -0500 X-MC-Unique: qUji4n2QNQWDdxOalZIvKw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 24D858871C8; Mon, 22 Jan 2024 19:42:58 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.195.114]) by smtp.corp.redhat.com (Postfix) with ESMTP id 492083C2E; Mon, 22 Jan 2024 19:42:52 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v1 09/11] mm/memory: optimize fork() with PTE-mapped THP Date: Mon, 22 Jan 2024 20:41:58 +0100 Message-ID: <20240122194200.381241-10-david@redhat.com> In-Reply-To: <20240122194200.381241-1-david@redhat.com> References: <20240122194200.381241-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Rspamd-Queue-Id: CE74F1C0005 X-Rspam-User: X-Stat-Signature: zsddtotsfcn9gcpyuu3dwr3nua8ngbjh X-Rspamd-Server: rspam01 X-HE-Tag: 1705952585-279268 X-HE-Meta: U2FsdGVkX1+kPijHCDb9kpDzOrbQsNOF1rAOkERGR6tyApKiHmtcsiRn4q6uiBQq48W/TyKM495P9BtjensWtYxSXv5DF4Z1MQDZxIyUZgvyH3t5weAWOgiBk5zUmoRZuy7rgIxbgDpwXUpC5Tu+B+Zesb7fOYqB+L88cbLwOovfmP1lydti2wS3X386yPQQ+1uYfemmiksV/RtQtWHur51icWdq9L8Wf5qZeLnbBAV+6B0CzvgzaDhuHDcZfQfZ0vZDz24N7j8IBs354U2g0zfl+5KbTuzoy7QF3maTrO6fBxg3nCQ5RpMT+h3Q8WyqV0sJWAtxGS4g2FDaUMDYlgojs3qUBnJP733o+HLm2/8JGk4Exg19cmKvY6LJho5bzm/svPQB2o0tKpvb3DLx0gFUh/znpv7TagC5nIaDxnj5pLA5+QIJp582ZxpmSK8y48NUUDgfS7Z/dP1w8TDmMnFura2952fQEUmp1Y3rFYSuELGNDQmqHXVq514IJEHFJ+rgBu5TShOz3qfMAx+yVtK1R5uwaEv9QbM0WteiVhd12wHzJOEm2ilyxPUqncVlfzvUevyvDIEK+dc5GmzMEji7A/+UCUSKigWJXbH97hm0+tPvFOF0kkypCUvdYz4xqKmDS/1Vp+tWpnolvOLFgji5yaw8lOqmWSfg1wkI3ofhcihGTMKbrsGYkT13WFWr/6YrdJhZD0hmcdOd5BGGB50O5/AHMV1HSNOdF5OOdu7mHAF2ZEMacKBMZGQqa9Ob64GD4J6mv4zTAi1t2SQETgsu0kaDshIYVN2o0jN1I8dlD5MeZDCRw6OejNljjmfz2qmG2JnZKJzEzwaozU85BrdSc2zAT9PLq53yjCNuZwGSS6skHVmnv4So8xOdTTWjolFXavUcKeyLxdX5SsBVx/8YL+X2Lx0/r/klOw7XzIf2/1IiHiOzqW5g6jAVImn8fmOLpw9IbBxB0uFEzzO ukVU9MPY 7TAOWlq6wjHv/LddN3zw5bW15/FI84ikJBjj/9ESOlV0JBKQIx2WxvACNgWs/d+VUFR89Mzi4UpCLKbdyzgbVE4eQRxVhvOH0YKREgGTPMaSP6CivEofNf1oxz8P0rImi8iOvx7ZTwtMBu51R+3GhFJj1d5ktERPmu3SGirgfJ7mTSLQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Let's implement PTE batching when consecutive (present) PTEs map consecutive pages of the same large folio, and all other PTE bits besides the PFNs are equal. We will optimize folio_pte_batch() separately, to ignore some other PTE bits. This patch is based on work by Ryan Roberts. Use __always_inline for __copy_present_ptes() and keep the handling for single PTEs completely separate from the multi-PTE case: we really want the compiler to optimize for the single-PTE case with small folios, to not degrade performance. Note that PTE batching will never exceed a single page table and will always stay within VMA boundaries. Signed-off-by: David Hildenbrand --- include/linux/pgtable.h | 17 +++++- mm/memory.c | 113 +++++++++++++++++++++++++++++++++------- 2 files changed, 109 insertions(+), 21 deletions(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index f6d0e3513948a..d32cedf6936ba 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -212,8 +212,6 @@ static inline int pmd_dirty(pmd_t pmd) #define arch_flush_lazy_mmu_mode() do {} while (0) #endif -#ifndef set_ptes - #ifndef pte_next_pfn static inline pte_t pte_next_pfn(pte_t pte) { @@ -221,6 +219,7 @@ static inline pte_t pte_next_pfn(pte_t pte) } #endif +#ifndef set_ptes /** * set_ptes - Map consecutive pages to a contiguous range of addresses. * @mm: Address space to map the pages into. @@ -650,6 +649,20 @@ static inline void ptep_set_wrprotect(struct mm_struct *mm, unsigned long addres } #endif +#ifndef wrprotect_ptes +static inline void wrprotect_ptes(struct mm_struct *mm, unsigned long addr, + pte_t *ptep, unsigned int nr) +{ + for (;;) { + ptep_set_wrprotect(mm, addr, ptep); + if (--nr == 0) + break; + ptep++; + addr += PAGE_SIZE; + } +} +#endif + /* * On some architectures hardware does not set page access bit when accessing * memory page, it is responsibility of software setting this bit. It brings diff --git a/mm/memory.c b/mm/memory.c index 185b4aff13d62..f563aec85b2a8 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -930,15 +930,15 @@ copy_present_page(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma return 0; } -static inline void __copy_present_pte(struct vm_area_struct *dst_vma, +static __always_inline void __copy_present_ptes(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, pte_t *dst_pte, pte_t *src_pte, - pte_t pte, unsigned long addr) + pte_t pte, unsigned long addr, int nr) { struct mm_struct *src_mm = src_vma->vm_mm; /* If it's a COW mapping, write protect it both processes. */ if (is_cow_mapping(src_vma->vm_flags) && pte_write(pte)) { - ptep_set_wrprotect(src_mm, addr, src_pte); + wrprotect_ptes(src_mm, addr, src_pte, nr); pte = pte_wrprotect(pte); } @@ -950,26 +950,94 @@ static inline void __copy_present_pte(struct vm_area_struct *dst_vma, if (!userfaultfd_wp(dst_vma)) pte = pte_clear_uffd_wp(pte); - set_pte_at(dst_vma->vm_mm, addr, dst_pte, pte); + set_ptes(dst_vma->vm_mm, addr, dst_pte, pte, nr); +} + +/* + * Detect a PTE batch: consecutive (present) PTEs that map consecutive + * pages of the same folio. + * + * All PTEs inside a PTE batch have the same PTE bits set, excluding the PFN. + */ +static inline int folio_pte_batch(struct folio *folio, unsigned long addr, + pte_t *start_ptep, pte_t pte, int max_nr) +{ + unsigned long folio_end_pfn = folio_pfn(folio) + folio_nr_pages(folio); + const pte_t *end_ptep = start_ptep + max_nr; + pte_t expected_pte = pte_next_pfn(pte); + pte_t *ptep = start_ptep + 1; + + VM_WARN_ON_FOLIO(!pte_present(pte), folio); + + while (ptep != end_ptep) { + pte = ptep_get(ptep); + + if (!pte_same(pte, expected_pte)) + break; + + /* + * Stop immediately once we reached the end of the folio. In + * corner cases the next PFN might fall into a different + * folio. + */ + if (pte_pfn(pte) == folio_end_pfn) + break; + + expected_pte = pte_next_pfn(expected_pte); + ptep++; + } + + return ptep - start_ptep; } /* - * Copy one pte. Returns 0 if succeeded, or -EAGAIN if one preallocated page - * is required to copy this pte. + * Copy one present PTE, trying to batch-process subsequent PTEs that map + * consecutive pages of the same folio by copying them as well. + * + * Returns -EAGAIN if one preallocated page is required to copy the next PTE. + * Otherwise, returns the number of copied PTEs (at least 1). */ static inline int -copy_present_pte(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, +copy_present_ptes(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, pte_t *dst_pte, pte_t *src_pte, pte_t pte, unsigned long addr, - int *rss, struct folio **prealloc) + int max_nr, int *rss, struct folio **prealloc) { struct page *page; struct folio *folio; + int err, nr; page = vm_normal_page(src_vma, addr, pte); if (unlikely(!page)) goto copy_pte; folio = page_folio(page); + + /* + * If we likely have to copy, just don't bother with batching. Make + * sure that the common "small folio" case stays as fast as possible + * by keeping the batching logic separate. + */ + if (unlikely(!*prealloc && folio_test_large(folio) && max_nr != 1)) { + nr = folio_pte_batch(folio, addr, src_pte, pte, max_nr); + if (folio_test_anon(folio)) { + folio_ref_add(folio, nr); + if (unlikely(folio_try_dup_anon_rmap_ptes(folio, page, + nr, src_vma))) { + folio_ref_sub(folio, nr); + return -EAGAIN; + } + rss[MM_ANONPAGES] += nr; + VM_WARN_ON_FOLIO(PageAnonExclusive(page), folio); + } else { + folio_ref_add(folio, nr); + folio_dup_file_rmap_ptes(folio, page, nr); + rss[mm_counter_file(page)] += nr; + } + __copy_present_ptes(dst_vma, src_vma, dst_pte, src_pte, pte, + addr, nr); + return nr; + } + if (folio_test_anon(folio)) { /* * If this page may have been pinned by the parent process, @@ -981,8 +1049,9 @@ copy_present_pte(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, if (unlikely(folio_try_dup_anon_rmap_pte(folio, page, src_vma))) { /* Page may be pinned, we have to copy. */ folio_put(folio); - return copy_present_page(dst_vma, src_vma, dst_pte, src_pte, - addr, rss, prealloc, page); + err = copy_present_page(dst_vma, src_vma, dst_pte, src_pte, + addr, rss, prealloc, page); + return err ? err : 1; } rss[MM_ANONPAGES]++; VM_WARN_ON_FOLIO(PageAnonExclusive(page), folio); @@ -993,8 +1062,8 @@ copy_present_pte(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, } copy_pte: - __copy_present_pte(dst_vma, src_vma, dst_pte, src_pte, pte, addr); - return 0; + __copy_present_ptes(dst_vma, src_vma, dst_pte, src_pte, pte, addr, 1); + return 1; } static inline struct folio *folio_prealloc(struct mm_struct *src_mm, @@ -1031,10 +1100,11 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, pte_t *src_pte, *dst_pte; pte_t ptent; spinlock_t *src_ptl, *dst_ptl; - int progress, ret = 0; + int progress, max_nr, ret = 0; int rss[NR_MM_COUNTERS]; swp_entry_t entry = (swp_entry_t){0}; struct folio *prealloc = NULL; + int nr; again: progress = 0; @@ -1065,6 +1135,8 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, arch_enter_lazy_mmu_mode(); do { + nr = 1; + /* * We are holding two locks at this point - either of them * could generate latencies in another task on another CPU. @@ -1101,9 +1173,10 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, */ WARN_ON_ONCE(ret != -ENOENT); } - /* copy_present_pte() will clear `*prealloc' if consumed */ - ret = copy_present_pte(dst_vma, src_vma, dst_pte, src_pte, - ptent, addr, rss, &prealloc); + /* copy_present_ptes() will clear `*prealloc' if consumed */ + max_nr = (end - addr) / PAGE_SIZE; + ret = copy_present_ptes(dst_vma, src_vma, dst_pte, src_pte, + ptent, addr, max_nr, rss, &prealloc); /* * If we need a pre-allocated page for this pte, drop the * locks, allocate, and try again. @@ -1120,8 +1193,10 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, folio_put(prealloc); prealloc = NULL; } - progress += 8; - } while (dst_pte++, src_pte++, addr += PAGE_SIZE, addr != end); + nr = ret; + progress += 8 * nr; + } while (dst_pte += nr, src_pte += nr, addr += PAGE_SIZE * nr, + addr != end); arch_leave_lazy_mmu_mode(); pte_unmap_unlock(orig_src_pte, src_ptl); @@ -1142,7 +1217,7 @@ copy_pte_range(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma, prealloc = folio_prealloc(src_mm, src_vma, addr, false); if (!prealloc) return -ENOMEM; - } else if (ret) { + } else if (ret < 0) { VM_WARN_ON_ONCE(1); } From patchwork Mon Jan 22 19:41:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13526142 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA064C47DAF for ; Mon, 22 Jan 2024 19:43:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7E6CC6B007E; Mon, 22 Jan 2024 14:43:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7976B8D000D; Mon, 22 Jan 2024 14:43:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 638598D0001; Mon, 22 Jan 2024 14:43:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 4FF286B007E for ; Mon, 22 Jan 2024 14:43:15 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 2BF59A0684 for ; Mon, 22 Jan 2024 19:43:15 +0000 (UTC) X-FDA: 81707970750.21.A7933F4 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf28.hostedemail.com (Postfix) with ESMTP id A219AC0017 for ; Mon, 22 Jan 2024 19:43:13 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=iB5TJCMr; spf=pass (imf28.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705952593; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OpHohfrBZPpQ+Ag8soc6zzIQIq/5TLvJxKBi0yYAutY=; b=I770wBoDY8jLKb8p0Dcg7Yg3QRwyygca8oZBjXU5aSvzV7wiQHgUSsDZFWLxeNXVyqYI6o x3tf+L6BU97bJwk3kGJY7I5Non/Mx1CszWKdPul6vHKdiKZmMQL2fw+em2yWN2I+X85AM3 VVUJRn7psMFmnBVn1odeLhZCDfQ11Y0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705952593; a=rsa-sha256; cv=none; b=tc2Hz+sussc57pB4oH2aJfeyP49hOuO5+yqsFNkm7gwLKSj9ep5pTm1fPaJhwI6QIEodfg XSdEQobMkNfKCPLgNex7VLBGaOuhuYP740DoVXKaOjqE0FWeZABInp08ON/J2JnQ7IdNo7 nf+x8sB6JyfxhzkY9NOlxuUyVShOpPs= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=iB5TJCMr; spf=pass (imf28.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705952593; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OpHohfrBZPpQ+Ag8soc6zzIQIq/5TLvJxKBi0yYAutY=; b=iB5TJCMruJyOUVEIUpxlfDQIgb9LypypkStctisTPuRNxG2RQOMmRPaBNdUtCeASJZyPen W40mRoQTTW8N5hgREGsWlDVWWpL3z0ICYbFMWsx8Hmw3/FCM4cHXgwqVbgds2lfvhN5dif GZtg4pfY7o53gHkbXLRAb6svCv7xRs0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-499-dgbcNuPJMAq9R1a_XNgOmg-1; Mon, 22 Jan 2024 14:43:10 -0500 X-MC-Unique: dgbcNuPJMAq9R1a_XNgOmg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AB3D888D145; Mon, 22 Jan 2024 19:43:08 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.195.114]) by smtp.corp.redhat.com (Postfix) with ESMTP id 851B33C2E; Mon, 22 Jan 2024 19:42:58 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v1 10/11] mm/memory: ignore dirty/accessed/soft-dirty bits in folio_pte_batch() Date: Mon, 22 Jan 2024 20:41:59 +0100 Message-ID: <20240122194200.381241-11-david@redhat.com> In-Reply-To: <20240122194200.381241-1-david@redhat.com> References: <20240122194200.381241-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Stat-Signature: 8b9n7sxycxyz1mpc9ixwha3s45u8jsrw X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: A219AC0017 X-Rspam-User: X-HE-Tag: 1705952593-61231 X-HE-Meta: U2FsdGVkX1+r6w7N8rkxavRMQryafWW+fPtqNjiJgHxTdWC4cKk1dItIt73xDJ0js7IwEffy/6yPzXDOt7mWgdwrpWM/bkAv8XbKcTVWoHp4vHkTG2IbSGCMSnoOMGr+urxKcwMFlugm+F38RwKB57eBq4CwXXahCcQGbkn2jWfCiND3+RLMsl/fvHUs2DtaKhIERPHH56D1/hENkPwGTfc2ssuOahbnMed05ZYtLcC5E8BJ/hHPqMtH9WjxXAjI35Gl1XRt8S56lzzB1hDOUpWRaRBLmFLOWarRwrvKf9AWqCfFQofqEGWwJ2Ol1iZVvinXg5Ge0gi9SY9Vt98k8MDp9QxwiViO4AHuZIrwBAys/IEQcjeh3wXB6rSSNHOJFhFiWqFt18sdh/PP6BOPYAA8BdrbEdXYFoAch8uIXs9CGWwdF4r7f5TjmhRWYgKhFT8jGC4ow3/LGJe0dPYozeAo4njLagTHuFcz8r7TIyPDwuYqMi9RyzQeEjaWjd2CkxzyRYgDboFOzq+ssCcCq8f0DtvkEEhQMheQNqXPztrjZ4nSMD3lefKzjA4Cz63XFqG7ImCxy3mR/QcRVFyP3aEC2I5rH6QuBn8FnJ4uVUAATFul9uEVMa1eHjYXH1No65lCai+kiuFhG239fkAEPPADUUfRK+MfCDC0BPbJhvZV/JZrtD7XfZvq7fSHvpriP6TkYjIpSo0+U4AGFlHLxeIO7qneq2Y06XU2ybb6vc6LpZbzb9R3OERu5LixE5gp111S2PpbtjA/WA+jhRNQHheRbVROiA7wDoqhOVmDLKcrfmoWBaNvwPI9dRTRjv1/D+6tcQKh4dDYI5XgVe/FjMWLoSMSs8wU4UUs9GAcz1MN2l7CMh69dMLMi2NnpZJxAsYyuNC8Kl//cb/ayy+YjxGap/jqlOnpS/fDXY1uEtstK4agJXEhbAvMowCWEvxD X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Let's ignore these bits: they are irrelevant for fork, and will likely be irrelevant for upcoming users such as page unmapping. Signed-off-by: David Hildenbrand Signed-off-by: David Hildenbrand --- mm/memory.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index f563aec85b2a8..341b2be845b6e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -953,24 +953,30 @@ static __always_inline void __copy_present_ptes(struct vm_area_struct *dst_vma, set_ptes(dst_vma->vm_mm, addr, dst_pte, pte, nr); } +static inline pte_t __pte_batch_clear_ignored(pte_t pte) +{ + return pte_clear_soft_dirty(pte_mkclean(pte_mkold(pte))); +} + /* * Detect a PTE batch: consecutive (present) PTEs that map consecutive * pages of the same folio. * * All PTEs inside a PTE batch have the same PTE bits set, excluding the PFN. + * the accessed bit, dirty bit and soft-dirty bit. */ static inline int folio_pte_batch(struct folio *folio, unsigned long addr, pte_t *start_ptep, pte_t pte, int max_nr) { unsigned long folio_end_pfn = folio_pfn(folio) + folio_nr_pages(folio); const pte_t *end_ptep = start_ptep + max_nr; - pte_t expected_pte = pte_next_pfn(pte); + pte_t expected_pte = __pte_batch_clear_ignored(pte_next_pfn(pte)); pte_t *ptep = start_ptep + 1; VM_WARN_ON_FOLIO(!pte_present(pte), folio); while (ptep != end_ptep) { - pte = ptep_get(ptep); + pte = __pte_batch_clear_ignored(ptep_get(ptep)); if (!pte_same(pte, expected_pte)) break; From patchwork Mon Jan 22 19:42:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13526143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89207C47DD9 for ; Mon, 22 Jan 2024 19:43:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EEC3C6B009F; Mon, 22 Jan 2024 14:43:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E9B578D000D; Mon, 22 Jan 2024 14:43:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D15448D0001; Mon, 22 Jan 2024 14:43:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id BFFF16B009F for ; Mon, 22 Jan 2024 14:43:26 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6807CC05CE for ; Mon, 22 Jan 2024 19:43:26 +0000 (UTC) X-FDA: 81707971212.01.0B4D00C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf29.hostedemail.com (Postfix) with ESMTP id B432C12001F for ; Mon, 22 Jan 2024 19:43:24 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=epISbfNM; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf29.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705952604; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tPsvEkVgYnC21FVdKrk6yuEGSmSqLr856lIR4lg3IVs=; b=uJCEWTZ7JoQo4a1rtT8X9gXW/lvZhM1Kz5oxfBHOHUg4sSAI7o67pRPt5zNEuiIiJYS3cC gOKpuLbhJjhNMjcT+FhfcVnh0MSHrS57OLqj2mP2BlSC8EbrVOBFURpL5Z+mSsSQUk45hS Qsh3HccaMNf+bdJkiSBU9Pz390MjvTM= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=epISbfNM; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf29.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705952604; a=rsa-sha256; cv=none; b=mI0YJCwrtWlTYXSdk8mJij6L9MJJXwn9SqbjXT7/mpoFne8rQl1K/YoGgw15PCitqay36h GRWciGNLBOeda5Bqdrj2Qf2Ywx7L7EQTWyD4KP3yGzzKcr7ooWUBe4Td3jgnNp+rRuJ7ve h/qcNayQvz+e4dmZLaA7qWxKB9j7CeY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1705952604; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tPsvEkVgYnC21FVdKrk6yuEGSmSqLr856lIR4lg3IVs=; b=epISbfNMsFymD8bEd6Gm/XAxGGp/qvluvDYWJZKLuaRZlKTUbkJc9plqovX5P5OAsFL5bP VNAdgmM9xuyQRlzHWEhXmStfBWVItleAfwUyeUgIPZRTTKG5SoFrVJ0BYCdxeOiFfCIdmR F8bHg61fXVmOLKZxn7f1KTHPJio/pfI= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-678-YpW0JN5UPxiQRu3cMsFdBw-1; Mon, 22 Jan 2024 14:43:18 -0500 X-MC-Unique: YpW0JN5UPxiQRu3cMsFdBw-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2508C87A38D; Mon, 22 Jan 2024 19:43:15 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.195.114]) by smtp.corp.redhat.com (Postfix) with ESMTP id 192C124ED; Mon, 22 Jan 2024 19:43:08 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Matthew Wilcox , Ryan Roberts , Russell King , Catalin Marinas , Will Deacon , Dinh Nguyen , Michael Ellerman , Nicholas Piggin , Christophe Leroy , "Aneesh Kumar K.V" , "Naveen N. Rao" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Gordeev , Gerald Schaefer , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , "David S. Miller" , linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, sparclinux@vger.kernel.org Subject: [PATCH v1 11/11] mm/memory: ignore writable bit in folio_pte_batch() Date: Mon, 22 Jan 2024 20:42:00 +0100 Message-ID: <20240122194200.381241-12-david@redhat.com> In-Reply-To: <20240122194200.381241-1-david@redhat.com> References: <20240122194200.381241-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.1 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: B432C12001F X-Stat-Signature: m6i7e4n88tqa5ypt5gaeodr9ckczfim9 X-HE-Tag: 1705952604-44604 X-HE-Meta: U2FsdGVkX1/j/8PuveOARv9h8BiSXVknmcpT95fiVcA7BqxeL43Eoh+Jp9TwglzlP55OwDbG7o4wQZpCOu25eihAFY9Zm2Xkpl7oJAuWjZ+naogHoPA9XbJFQ414ipWxbn/bsxh5kl/UwlQl4XJ1gSK6lxhlKQMeQ8qvhuwuqYjrZ1vbeHi2zFNLGdolLFEB9ERX2SMCSNPfrwft4UKeZ4lSYKKDFnLxcNGmqOaYBw1cMJVDa8FGldYLFP/t/AG9Hh+UrkiNzoGNJb+ZiH9FhivKzQv1HzF379q6/W71cAaVQFW/SB6L5f15Sxr3ayqsCKmc+lmI3oVNBZvX1yvzhZau/GnBUTqr6DkSPxsb06iZStoevlmsLjJslj4tNo12S0sKRodaD84AO6OZwVseFbUinGPERTZiAt5MIw+zaFBvpQc6BIg+wN5vWrxFwKhMzOQG0oOdRigTajQB+YHVGyDvra1cUt4EAfkaeWC+r/OIrQvH3oOj9F12ZyzGSXhFWzQUTRhIBDLTG8jgu3pNonh/bDesgVJ4DeSRkjLnHpq+rOh8CeIq3caDbTwL60wrg1c2Gn4nhDCltBx8DvuzORV5avDulXDaxvJ5FGPc2tQiFlEq8aP/Cb4IpNxKLBIe48+XOCI9Ndx/RYzMvt8JpxtUteAg82XjyyM+iBXQr1j84Y3r4kCJXB92V0lvUpeIEdZUJ3uM0o2F+Fxfb5TJYipsWLVEUKv+ADc7P9i02TIb7INdJTSuzjXRPiaBYh4Mp2VztJAykVFKKORKlS9ccNm9N8mPHnxg5U1KsQXoamqdFq+AENd4LaoFhg/52aBDtbhRlb9KUHxKMmezeqRrHKRxkdWwLZBVWrAEfmaDqIt65P1xLLjzZQbaez0e4tekS2897syxZkxFEdPscbCvLYQcOsoZoAzpd2B5L0etAMu5I314asNGFqD5KJ+9uwhZ2zc6DYmK6f9uOb612fV A+ztQ6xP u2qfUNJQ5UgV3JGvSrSsjx4kqRFLiDxM40OKxa9W2pGsNK7c16o76GmzXXreiOGVE9rmKk3SXeBmy2FFq6GcUNcgStUaPyNSXlqsZWVETeyIrbpVp83eMQU24N46w3dD7TnT4Oxh0m7CErK439lLp0OwX97ksIB1ILUurJZthA0IYxRF26MLNfW2AdGM88qsICZaU76VchwPPn1KqvDvk3jE6nJFBOmSbPYNx X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: ... and conditionally return to the caller if any pte except the first one is writable. fork() has to make sure to properly write-protect in case any PTE is writable. Other users (e.g., page unmaping) won't care. Signed-off-by: David Hildenbrand Reviewed-by: Ryan Roberts --- mm/memory.c | 26 +++++++++++++++++++++----- 1 file changed, 21 insertions(+), 5 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 341b2be845b6e..a26fd0669016b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -955,7 +955,7 @@ static __always_inline void __copy_present_ptes(struct vm_area_struct *dst_vma, static inline pte_t __pte_batch_clear_ignored(pte_t pte) { - return pte_clear_soft_dirty(pte_mkclean(pte_mkold(pte))); + return pte_wrprotect(pte_clear_soft_dirty(pte_mkclean(pte_mkold(pte)))); } /* @@ -963,20 +963,29 @@ static inline pte_t __pte_batch_clear_ignored(pte_t pte) * pages of the same folio. * * All PTEs inside a PTE batch have the same PTE bits set, excluding the PFN. - * the accessed bit, dirty bit and soft-dirty bit. + * the accessed bit, dirty bit, soft-dirty bit and writable bit. + . If "any_writable" is set, it will indicate if any other PTE besides the + * first (given) PTE is writable. */ static inline int folio_pte_batch(struct folio *folio, unsigned long addr, - pte_t *start_ptep, pte_t pte, int max_nr) + pte_t *start_ptep, pte_t pte, int max_nr, bool *any_writable) { unsigned long folio_end_pfn = folio_pfn(folio) + folio_nr_pages(folio); const pte_t *end_ptep = start_ptep + max_nr; pte_t expected_pte = __pte_batch_clear_ignored(pte_next_pfn(pte)); pte_t *ptep = start_ptep + 1; + bool writable; + + if (any_writable) + *any_writable = false; VM_WARN_ON_FOLIO(!pte_present(pte), folio); while (ptep != end_ptep) { - pte = __pte_batch_clear_ignored(ptep_get(ptep)); + pte = ptep_get(ptep); + if (any_writable) + writable = !!pte_write(pte); + pte = __pte_batch_clear_ignored(pte); if (!pte_same(pte, expected_pte)) break; @@ -989,6 +998,9 @@ static inline int folio_pte_batch(struct folio *folio, unsigned long addr, if (pte_pfn(pte) == folio_end_pfn) break; + if (any_writable) + *any_writable |= writable; + expected_pte = pte_next_pfn(expected_pte); ptep++; } @@ -1010,6 +1022,7 @@ copy_present_ptes(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma { struct page *page; struct folio *folio; + bool any_writable; int err, nr; page = vm_normal_page(src_vma, addr, pte); @@ -1024,7 +1037,8 @@ copy_present_ptes(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma * by keeping the batching logic separate. */ if (unlikely(!*prealloc && folio_test_large(folio) && max_nr != 1)) { - nr = folio_pte_batch(folio, addr, src_pte, pte, max_nr); + nr = folio_pte_batch(folio, addr, src_pte, pte, max_nr, + &any_writable); if (folio_test_anon(folio)) { folio_ref_add(folio, nr); if (unlikely(folio_try_dup_anon_rmap_ptes(folio, page, @@ -1039,6 +1053,8 @@ copy_present_ptes(struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma folio_dup_file_rmap_ptes(folio, page, nr); rss[mm_counter_file(page)] += nr; } + if (any_writable) + pte = pte_mkwrite(pte, src_vma); __copy_present_ptes(dst_vma, src_vma, dst_pte, src_pte, pte, addr, nr); return nr;