From patchwork Mon Jul 29 09:10:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13744625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC639C3DA4A for ; Mon, 29 Jul 2024 09:10:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0A5AE6B0085; Mon, 29 Jul 2024 05:10:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0618F6B008C; Mon, 29 Jul 2024 05:10:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D36BD6B0085; Mon, 29 Jul 2024 05:10:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id B2A846B0085 for ; Mon, 29 Jul 2024 05:10:38 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 4391216020D for ; Mon, 29 Jul 2024 09:10:38 +0000 (UTC) X-FDA: 82392219756.14.7F0B45D Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf18.hostedemail.com (Postfix) with ESMTP id 3E4971C000E for ; Mon, 29 Jul 2024 09:10:35 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf18.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722244183; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BfkSOz2Y9ZrBcjfIMnhUE9RAoBJM1JlNNfTgf+rznQs=; b=nYPKyAKi8S1NF1hHjNwn6aVBDBsgSamXv4HJz9tvKTjNsRXO9+6xcXqkUj4DdLPY9unSnV voc5b4pAbaoHyD4FWuAwyt44UFt4T6wzTsV4zRaPG3GIUQ/90t0+bIwKZpb6w5VMgwWNzS fWNFAgm/3CCr9N97B2BbWq41Ga3UzWk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722244183; a=rsa-sha256; cv=none; b=PHsBa/JcW8Awo6GxFZTC3HqOyGrwvfg6WhnUmUCAO6L4be8F/dhagC4CWZqGH5/tuDGObN zwG9kAhT5Npdw3/DxbtNMGgU6MoxzA5U6D5l93jiPZgN49mJu/pBD2qpMdb+yaDY0Q5/sW 9mkbWUhG5KW8qOKuJw1zYlcQqCKi3Vs= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf18.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id DAFA61F445; Mon, 29 Jul 2024 09:10:33 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 96A1C1397F; Mon, 29 Jul 2024 09:10:32 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id ADa6Hohcp2Z4BgAAD6G6ig (envelope-from ); Mon, 29 Jul 2024 09:10:32 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , Donet Tom , Matthew Wilcox , Vlastimil Babka , Michal Hocko , Oscar Salvador Subject: [PATCH v2 1/9] mm/mmap: Teach generic_get_unmapped_area{_topdown} to handle hugetlb mappings Date: Mon, 29 Jul 2024 11:10:10 +0200 Message-ID: <20240729091018.2152-2-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240729091018.2152-1-osalvador@suse.de> References: <20240729091018.2152-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 3E4971C000E X-Stat-Signature: 45fnnphm4ymd1sr4qguoqcitqu7z8oqi X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1722244235-783555 X-HE-Meta: U2FsdGVkX1+BSjhKsEx0oxhvjG/xH2yGMLyLc5OKwGCryMMjpAxzdEfmQ3PGDlCPPE9YSwLjJbUR1Z+DUPVstrzjViUPgIESoKZZ372sgWNVFwVhomaJrHMg0qyuaf3vu1LAHOu6WxE+7teCzWftqJvv2IVS1a51wYFFzXZ/K0KJO3GvEdX4NCT8eXznjiqydaIX93EO5ssrcBBDC9sb1FYJCrEtZFpK7GFxXjfRQVIWAE5YeHSINZV0uAjTYA1ArvIZbUONt1echPo0gfEQ9asUZ9tb2S1Pydy+8ham9QjQNZDFeb2mFmLS4Td5l+vkDSGp7U8pt25pVO2eVvHBjJeW5CwttH2EDOtTbIaGqwNKXPT/0a87Cv1dceTQfi9cuF+CxJ9VKuOeFj+KiPhRR9Ss8kaE5VMDM3QmGfFsjGMiFevwkqJCEbSElfI5dUcJtJSRJ3ZUAwrh3nZr1siL2DJiab71O5vlcQla1xnyatkOg3+p/1kxzEZ5qgBKqHIajVtjq44+BQ2DjJLzATd1rs7VNTdV5uBjC3Y4eEoWq/gNMvFs0tjm/G9efUH6iToYz6jF9KeNOHmHaZ1IHkJy38dywr5YR+Pdli2g6gErJhNAeTGvKKb2yEtuHxfnOSuYd3wvBii6yWSXU/s++noP9XTIXCmGK0NlWMeRq+IlzJrS/cTwezYmB12CRaFvtsHIRZ6kfN0JCPxqaLg93JWi85pcId+Q/f2Snd5wrNaYwqKALRfN/GBITtdtkY+UgAnNgFXxesLi7RuA3zc0K8o9mY15U6dTOvd4B39aS3xeZEL6iCJT5RvHK3cbMSNTP1zlctnkSwjA6iKd9Q+szYbWAkQ5Q1E+F4RRhy7CQu7Lak2U0FHwF4X2K3poVsLDLdb+lm/C0JiZoyBicRfUUF52TqmGOBJ3CChdwPf2W4ZpXhuqasNUt4Xu1bYmzn85gniGjx5p0tqKHO8pMrO0GY/ VrQ1qEki 6hZ61UH3HXIh128yISKHCCVTBgYybzoNZCa3I2D1yQTUN053fowLk8XtpG2RvtYkZDMwx5BlEu7Khr3HUyQhqZSPoTHeyQzQItcZMNNY31zxjmL5i4SjYEI4t8ht5VhoJUagkKPXmRHYqTjHq+EoAVkOmihCPRPFKxthje5Jocyh6UfA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We want to stop special casing hugetlb mappings and make them go through generic channels, so teach generic_get_unmapped_area{_topdown} to handle those. The main difference is that we set info.align_mask for huge mappings. Signed-off-by: Oscar Salvador --- include/linux/hugetlb.h | 10 ++++++++++ mm/mmap.c | 4 ++++ 2 files changed, 14 insertions(+) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index c9bf68c239a0..0ec14e5e0890 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1007,9 +1007,19 @@ void hugetlb_unregister_node(struct node *node); */ bool is_raw_hwpoison_page_in_hugepage(struct page *page); +static inline unsigned long huge_page_mask_align(struct file *file) +{ + return PAGE_MASK & ~huge_page_mask(hstate_file(file)); +} + #else /* CONFIG_HUGETLB_PAGE */ struct hstate {}; +static inline unsigned long huge_page_mask_align(struct file *file) +{ + return 0; +} + static inline struct hugepage_subpool *hugetlb_folio_subpool(struct folio *folio) { return NULL; diff --git a/mm/mmap.c b/mm/mmap.c index b74d2967cfc0..7b623811d82a 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -732,6 +732,8 @@ generic_get_unmapped_area(struct file *filp, unsigned long addr, info.length = len; info.low_limit = mm->mmap_base; info.high_limit = mmap_end; + if (filp && is_file_hugepages(filp)) + info.align_mask = huge_page_mask_align(filp); return vm_unmapped_area(&info); } @@ -780,6 +782,8 @@ generic_get_unmapped_area_topdown(struct file *filp, unsigned long addr, info.length = len; info.low_limit = PAGE_SIZE; info.high_limit = arch_get_mmap_base(addr, mm->mmap_base); + if (filp && is_file_hugepages(filp)) + info.align_mask = huge_page_mask_align(filp); addr = vm_unmapped_area(&info); /* From patchwork Mon Jul 29 09:10:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13744626 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83704C3DA4A for ; Mon, 29 Jul 2024 09:10:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4284C6B0088; Mon, 29 Jul 2024 05:10:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 386E36B008C; Mon, 29 Jul 2024 05:10:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0ED426B0088; Mon, 29 Jul 2024 05:10:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id CF8B46B008A for ; Mon, 29 Jul 2024 05:10:38 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 91A0840205 for ; Mon, 29 Jul 2024 09:10:38 +0000 (UTC) X-FDA: 82392219756.26.9FD118B Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf13.hostedemail.com (Postfix) with ESMTP id 955C420005 for ; Mon, 29 Jul 2024 09:10:36 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf13.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722244197; a=rsa-sha256; cv=none; b=y2AjKCbb5nqMJc1054bgMgcAqrTBTwQ3InscbvLgIqhKemY7B+5sCy0ep9xYQJy0s+Weko RRWBYCqc7ePIXff1Rrq/wBY5ONxO8/wuoBHV/BM0vNtlVV6p4EeEyJlU7uYVMv+NiHitx9 m77XvVem1is1LLdrWmJvyXp6SCIc3UA= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf13.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722244197; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/YtBEq+futfpDK6VnLTGAsLUUeg4b//ccUCfgdjh03Q=; b=qZpK+0dzSyL5Ja8IRhSKEo8XyOQnXNLx5qoJulgMRFgNX92pUfXPVjTCgoXSi6BQASwCTs fUNw1VuOvOfaiVZN0y6kTarcC/wJ+5sUptuP6puIu0UQaf/s0gX89N8j8LYFTsurPfyybX T+8V+qmqwxoC7wARJSySMgsTCS6+ons= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 56E361F78D; Mon, 29 Jul 2024 09:10:35 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1030C1368A; Mon, 29 Jul 2024 09:10:33 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id GAkEOolcp2Z4BgAAD6G6ig (envelope-from ); Mon, 29 Jul 2024 09:10:33 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , Donet Tom , Matthew Wilcox , Vlastimil Babka , Michal Hocko , Oscar Salvador Subject: [PATCH v2 2/9] arch/s390: Teach arch_get_unmapped_area{_topdown} to handle hugetlb mappings Date: Mon, 29 Jul 2024 11:10:11 +0200 Message-ID: <20240729091018.2152-3-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240729091018.2152-1-osalvador@suse.de> References: <20240729091018.2152-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Queue-Id: 955C420005 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: ieubk1qj9ogtyrkwkb3ry8whxe9djheu X-HE-Tag: 1722244236-453920 X-HE-Meta: U2FsdGVkX1+q2zBU/vzSjeBzQl45+gRTKllKkRtDHt0/g8PXyt876ASQfcGAoqYQYz3MT3PgVaeB+T+YzoMYeoN1ChLzkXoiN1TdB/P5XcSc6rtuwJ/8DFmRjiJSrHQSOaKit6dzrmyjwdiaFj9hf5h0JTsnr8Tw2L+UVMq2wjGdT7zl6hjc17EiHX/kwgWtCnNRQamB8RfYdDMuI1p9HBsowuXHzIureIqQUL+BhwFuJJTfMXSec2UYboffMlLSnZ9+2KxVmLIwJgz+6uiuz3MUEp85LytYy/rqC96hLLbl0lg+1SJSbyj0/nAM1SwRPznX/xtzDGEw6vuM9JGS2r/D0pRFLp1pO7ncm3FQIW4biZ/vD2D1EQSAp75oKeOjxqNrUaM6gEcnzMq61u6+nnJAoa5wNyBzK0pTop5HtAgu7UNJWkxblvXppuRiTStjtsAo43QOC9Hk16LolPwQ8N1qVWoF3z5B9pk66qZ9gvuG/WtHVDkCQWW0zDAg7gOTSJ1WNLhiIngFMlZohvLUkzvyMgb20lJ0abdEUr1robN5SY2GZv7St+dKAe3KpOhVcBcqCMbkCLHwFdw6eEfXWLHgHFNNyO6AyTCeDQv/AenzlbwB4tEAMwozEyvW3Y98DdyEjKnlmWH9vjAIHwIByfpxa+n5LARwFo8CiS0V2yFY+R5KOclauxr7xaYLleCPELJe7U9lAGDEu6x5QVgSp8UglhqHRZyHftzlAm3lq7BoU1w8ZwDI5vOgA1V/2umPjZ18oGi147WIqiHE5z7TsbiawAhT3R5RYdQcj6q6hjIk5h1ar+O/P0hA0gOWz30hbrhrBx9SY4KC3a0Z4PYtSTHP5GAozu3EZ7953LUFqzPT1AtkJPOAZ8eFJkSk058TlMVgdOaOTZhDG+xzHmFj6PiZJ1dKlUp+Xwm69a1hM85F/N7Tmw48mwx4f85WWSEGXk7bX3+h4ZV8eETYw/3 k30VPs7W 8xFK96QVy46cmWfcFWArFuCbcEGmid1+0HOq5lGf2erxMSUGqOqHwkldSLyA9CxX9wScxVLAZalobyTz/LkRrRSJlUZqGDjvVSoON9jd8B6NW60T7LsbfQsehPqfTARLv9oNuFV/TVJG6nCJo07MbAkeuNNMLxCd2bqol5oTYHGXVTPg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We want to stop special casing hugetlb mappings and make them go through generic channels, so teach arch_get_unmapped_area{_topdown} to handle those. s390 specific hugetlb function does not set info.align_offset, so do the same here for compability. Signed-off-by: Oscar Salvador --- arch/s390/mm/mmap.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/arch/s390/mm/mmap.c b/arch/s390/mm/mmap.c index 206756946589..408b5a541a28 100644 --- a/arch/s390/mm/mmap.c +++ b/arch/s390/mm/mmap.c @@ -17,6 +17,7 @@ #include #include #include +#include #include static unsigned long stack_maxrandom_size(void) @@ -73,6 +74,8 @@ static inline unsigned long mmap_base(unsigned long rnd, static int get_align_mask(struct file *filp, unsigned long flags) { + if (filp && is_file_hugepages(filp)) + return huge_page_mask_align(filp); if (!(current->flags & PF_RANDOMIZE)) return 0; if (filp || (flags & MAP_SHARED)) @@ -106,7 +109,8 @@ unsigned long arch_get_unmapped_area(struct file *filp, unsigned long addr, info.low_limit = mm->mmap_base; info.high_limit = TASK_SIZE; info.align_mask = get_align_mask(filp, flags); - info.align_offset = pgoff << PAGE_SHIFT; + if (!(filp && is_file_hugepages(filp))) + info.align_offset = pgoff << PAGE_SHIFT; addr = vm_unmapped_area(&info); if (offset_in_page(addr)) return addr; @@ -144,7 +148,8 @@ unsigned long arch_get_unmapped_area_topdown(struct file *filp, unsigned long ad info.low_limit = PAGE_SIZE; info.high_limit = mm->mmap_base; info.align_mask = get_align_mask(filp, flags); - info.align_offset = pgoff << PAGE_SHIFT; + if (!(filp && is_file_hugepages(filp))) + info.align_offset = pgoff << PAGE_SHIFT; addr = vm_unmapped_area(&info); /* From patchwork Mon Jul 29 09:10:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13744627 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FB2EC3DA61 for ; Mon, 29 Jul 2024 09:10:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 119356B008A; Mon, 29 Jul 2024 05:10:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C9536B0092; Mon, 29 Jul 2024 05:10:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E5F966B0093; Mon, 29 Jul 2024 05:10:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BE2686B008A for ; Mon, 29 Jul 2024 05:10:40 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 6E44312020B for ; Mon, 29 Jul 2024 09:10:40 +0000 (UTC) X-FDA: 82392219840.18.6C43411 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf02.hostedemail.com (Postfix) with ESMTP id 5126180018 for ; Mon, 29 Jul 2024 09:10:38 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf02.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722244235; a=rsa-sha256; cv=none; b=JiamAPPkZXc1Piju4lQoPhTj49h1GJrtGtBUTSsP/CiSYhKPbJl0VIb/ABOFAX4e9bBait 0jmjEU0lb3+f/WCljzCMAanZJKY2wvIjWFVQJrqlcpFkLPjIlSlS2LBKUHl8hamRYw4nZW Z7oXn3CjEe68TSCuOGfE1U57XJmgQGU= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf02.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722244235; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BHN99Otp08yvKHunYIWd3zL4cHzZBNMKDmIpLbdX684=; b=Jk6PXq3a/QboVudUF4qp/r3n2HsfMJSz0Cf1tEFEE1fZEXbnWV/Ci24sew6cEOjYkig3Hw F+ZM18ZYLdxODkOml2z76hQ2nQ2XN5eM3N0qCNonTOUm8JF6PFZKy3jj42lz+ZGbLm9auz 8X6wrsDVRKEgplW3ywQdyc23zecCZ2E= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id CD03C1F790; Mon, 29 Jul 2024 09:10:36 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 82F171368A; Mon, 29 Jul 2024 09:10:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id UOzvGYtcp2Z4BgAAD6G6ig (envelope-from ); Mon, 29 Jul 2024 09:10:35 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , Donet Tom , Matthew Wilcox , Vlastimil Babka , Michal Hocko , Oscar Salvador Subject: [PATCH v2 3/9] arch/x86: Teach arch_get_unmapped_area_vmflags to handle hugetlb mappings Date: Mon, 29 Jul 2024 11:10:12 +0200 Message-ID: <20240729091018.2152-4-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240729091018.2152-1-osalvador@suse.de> References: <20240729091018.2152-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspam-User: X-Rspamd-Queue-Id: 5126180018 X-Rspamd-Server: rspam01 X-Stat-Signature: 99zh33gqfpisfmch5fck1ef4aecbf54a X-HE-Tag: 1722244238-785395 X-HE-Meta: U2FsdGVkX19p8Rx2uKSvtAjY4HCwvCbwHBSkchsXFA50siR+AUMNoC1aOGb3lT4rxf4ZKKFzc6vTeKx3QN3rCt7GNLcNM3PvEEFKF8L6xgnP52/nwLOgnra5rzTjmIGwkUxT9pOa0SEaveHH8XwaUJ7hf/X9uPOjitOZ2+TsE8p2Nbp8knslwm0+bhG7DcWiie2HkPbAoFZb1OT/0FHbaj3nCjZhFurHny9w6dfulP3K+P80H37gx+J75cgNulKDG6qYTa/h2bcsGu6cScoagwdDHu+YkA3zivOrjmYXmXiM0H968IdT9llqK5AIQLTZn3/5ykJCAKJjo+VGd0TrW688bkNcSyxNPrYQb5OnjsWJIbERkIgkOlknFONi76bChTynm8fDJKvdGLSUBbJavTXJm4y0PLhXGkxjhVVtw+4h2gkgfi7Mc/WkgYQF8v4AOkB5ThKPWZv04jTAJvvU3ucPGffoHV7PTaG9h6fjSQPD2F8IRHTFV4+ipA34TlYjPSYti+dPiq5ujpf0XtdJSGQtEBinlJb+xVnds8EUxltsmErOWnMwP64NNhK4UArDCYkbYaFwbgb66J0PbygGR97BhDwP/48lPWAysm/W9+vdUCrcoxgRRePBerbR5tqJ5grNl5FKTej7i8pCu1+wO4diVufCe17JX+vtmTArUH0gt6qoTS/kAVEeOhbtHJjkjJnvyUZlkhi9cTuLOL0cx0EnvU4IPgc8+/BgjKoL2TeABvShYmnD0CfpardN2mYz+NkC8T9TZK3zt8VNKm6HZD/SaNc+71OEhOcIiYp4PQyW3U85DDyASZjsyUCt+Ybao4kd2POFgOWq2+R2vFvugZw1qQbFkNKcNePJ9/gkzk8w6dqap95Ef2R02EhS5Mqctj8Q1OJmumZVADg1c7DVnRuVpqlLX0dBqxWbG9KwiwwgJM769MQW8+zuDQHifAN+Uja8E/HLqQRuc8bmCxZ nVf1x9z+ h98adRblWnw0Ty7GkVjzp3fhBw+6NqnGXXncNicU6YThi/x6D/EQX0QIyZsnfsRDwsUCLmWsCFCwpdEp81hOUT0OUmbP7vN3jE4FntJ6Yos6pC4dxkFd+WesO+iQWYYJrPWU4A0xTO/4MdY9m9jdi60We6vPQxLCTWepfMeCrbuvfYo0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We want to stop special casing hugetlb mappings and make them go through generic channels, so teach arch_get_unmapped_area_{topdown_}vmflags to handle those. x86 specific hugetlb function does not set either info.start_gap or info.align_offset so the same here for compability. Signed-off-by: Oscar Salvador --- arch/x86/kernel/sys_x86_64.c | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/arch/x86/kernel/sys_x86_64.c b/arch/x86/kernel/sys_x86_64.c index 01d7cd85ef97..aa7491f036a7 100644 --- a/arch/x86/kernel/sys_x86_64.c +++ b/arch/x86/kernel/sys_x86_64.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include @@ -25,8 +26,10 @@ /* * Align a virtual address to avoid aliasing in the I$ on AMD F15h. */ -static unsigned long get_align_mask(void) +static unsigned long get_align_mask(struct file *filp) { + if (filp && is_file_hugepages(filp)) + return huge_page_mask_align(filp); /* handle 32- and 64-bit case with a single conditional */ if (va_align.flags < 0 || !(va_align.flags & (2 - mmap_is_ia32()))) return 0; @@ -49,7 +52,7 @@ static unsigned long get_align_mask(void) */ static unsigned long get_align_bits(void) { - return va_align.bits & get_align_mask(); + return va_align.bits & get_align_mask(NULL); } static int __init control_va_addr_alignment(char *str) @@ -148,12 +151,15 @@ arch_get_unmapped_area_vmflags(struct file *filp, unsigned long addr, unsigned l info.length = len; info.low_limit = begin; info.high_limit = end; - info.align_offset = pgoff << PAGE_SHIFT; - info.start_gap = stack_guard_placement(vm_flags); + if (!(filp && is_file_hugepages(filp))) { + info.align_offset = pgoff << PAGE_SHIFT; + info.start_gap = stack_guard_placement(vm_flags); + } if (filp) { - info.align_mask = get_align_mask(); + info.align_mask = get_align_mask(filp); info.align_offset += get_align_bits(); } + return vm_unmapped_area(&info); } @@ -199,7 +205,10 @@ arch_get_unmapped_area_topdown_vmflags(struct file *filp, unsigned long addr0, info.low_limit = PAGE_SIZE; info.high_limit = get_mmap_base(0); - info.start_gap = stack_guard_placement(vm_flags); + if (!(filp && is_file_hugepages(filp))) { + info.start_gap = stack_guard_placement(vm_flags); + info.align_offset = pgoff << PAGE_SHIFT; + } /* * If hint address is above DEFAULT_MAP_WINDOW, look for unmapped area @@ -211,9 +220,8 @@ arch_get_unmapped_area_topdown_vmflags(struct file *filp, unsigned long addr0, if (addr > DEFAULT_MAP_WINDOW && !in_32bit_syscall()) info.high_limit += TASK_SIZE_MAX - DEFAULT_MAP_WINDOW; - info.align_offset = pgoff << PAGE_SHIFT; if (filp) { - info.align_mask = get_align_mask(); + info.align_mask = get_align_mask(filp); info.align_offset += get_align_bits(); } addr = vm_unmapped_area(&info); From patchwork Mon Jul 29 09:10:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13744628 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 286F2C3DA61 for ; Mon, 29 Jul 2024 09:10:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7CF4C6B0092; Mon, 29 Jul 2024 05:10:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 77F626B0093; Mon, 29 Jul 2024 05:10:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5843C6B0095; Mon, 29 Jul 2024 05:10:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 3782E6B0092 for ; Mon, 29 Jul 2024 05:10:42 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D89BA801FC for ; Mon, 29 Jul 2024 09:10:41 +0000 (UTC) X-FDA: 82392219882.29.A179FEC Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf05.hostedemail.com (Postfix) with ESMTP id B8BFA100003 for ; Mon, 29 Jul 2024 09:10:39 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722244186; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KPjpjgmqDfRdVPMwY8SIrDz1ElJHQdBJQeTCq0Yrxzo=; b=hv5H9LXUAtOgLnvVP/E2W1F+JWmf44OXq95/kKgwAIHeeg/Rh5mvNX/6xOL+f9KY8wAeIi afD1pUnASTlt1nQi5rP69FPx3mhkizbWcUEqi9m9WojMhZYD4epDOmUXC5XrNA7GdZkutk l4eM2tS8hOOh8QT6cEeTcIUk7ElvBlg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722244186; a=rsa-sha256; cv=none; b=pyvrsRdgvACYlqerucRa5CQOlDpl3+2zAbSQyc/tgCmRsf2/KL6BUMGGEt+ebHARi4G2K+ 3pI5KJANUU2j2vuYKmQhisXeYICoS+FIb74iGNiALtLWnq7gw89rzMBRxqGrBTTgmtKAWP 4arIQOqBrPBqkw+RkbJoUQ7HMxIpFac= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 70CFD21BCC; Mon, 29 Jul 2024 09:10:38 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 03FE91368A; Mon, 29 Jul 2024 09:10:36 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id IHMSN4xcp2Z4BgAAD6G6ig (envelope-from ); Mon, 29 Jul 2024 09:10:36 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , Donet Tom , Matthew Wilcox , Vlastimil Babka , Michal Hocko , Oscar Salvador Subject: [PATCH v2 4/9] arch/sparc: Teach arch_get_unmapped_area{_topdown} to handle hugetlb mappings Date: Mon, 29 Jul 2024 11:10:13 +0200 Message-ID: <20240729091018.2152-5-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240729091018.2152-1-osalvador@suse.de> References: <20240729091018.2152-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Stat-Signature: 3kaaagnmw1pyzbrnmij9uq6wwp8ihybi X-Rspamd-Queue-Id: B8BFA100003 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1722244239-557582 X-HE-Meta: U2FsdGVkX1/sX6Bg7Dk9boK3Ke+DuycLhd1wld4Um/nC0snJsAgt00UTJKhEMEgBdkmfWbvXXc3YSA91IKjpKbnMX3QMTPvTjcI6VaYehxEX0nBZsn58Z0jy/bBlNfNj+TbLUaIOLnKlTxJB9TggtUNc++pn6reVdyZrjl5kiw4Q8pZEvINY0kB64ZUY0XSUueMi/suToe53a3Jrc45einBRx3C+F6B5+AOspzjLfukmCnoPJ7LNzTFDtsi+opzoNNk7QKyAXomA8ybgX4pQrCmH1W2tq6Hse7kEFe8/uBMz+Mk3Ao326Ubsl1tgQSAAGe1jSTVnv7+mCfEeYrsC6LYlEvmjCuhbCFXnOKUQpWK//wzCXlCJDtfQ/7/+kNemKvzDM6WvPyzYHVbjzFY5Qbj9Vhl5hZp6Ju4+WXz+2DTi/ROyOR53sHqwCIRHPhUk0VKztyPfGFQTTwqnVYhgP+rOsumKKj7vz/0U4zFH2sHV2d3tkpBGlNxFfzq/lENWwX3eDX8ofY71aN9jYZjq0HB68SA/d8mnKyfwUhy30Bw7+4IkvJOGBFqMfzrwlSL7r1eZ2j15RVvaacnD5Byqh0OQshosrCR3OXPhMZEACbD5mABsQQgVl3DDNcsN1GKKu2buzF0tvqPO91W3ii6I6DDXmfL0Bp8ivv3OGCr5ggTHSbmtRfUITC90aPn7MKg7Y/5hc3cZ6byXgT2ur1QX7RAqb2/WtaxrSGQpFHqF02KoERnD4PXoXgIDFl6HBbQlHh1mWjYW525wKVXMV0GlfBdfWJhWJUL4Csb1f3f6ZhZsTn17BR2AOSRoejflquojnrHv614JeVE4AzLD5gDmJbzm2Dl8bQU47kP2adxf8cP7bOK6S9eP40GGW3FBub2mr2Cp8AO9BfTTUpgDm4BFzzDwXEh2rRzjxI22y3aQXuO7JdHz5ZbZRqHE9sZFEB1H2md6PFNzDWQ413MUZ88 XD0uThVB mtQYZuxTiNe78x3kwX2NE/AOKbNCmOUbnsiWoyCuWQAafObgPwq2lgUL0k5yykbHVmuO8vbDxen/lAILWvrJwa1TRkgkJnwR4BNBf1jkM50UbdjNSQVmI0zaAhvfhMrvLvBsCQTt6XpKGk1tvc7gND1dZP8RuAeHcusqF0h3pQ3QgDBQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We want to stop special casing hugetlb mappings and make them go through generic channels, so teach arch_get_unmapped_area{_topdown} to handle those. sparc specific hugetlb function does not set info.align_offset, and does not care about adjusting the align_mask for MAP_SHARED cases, so the same here for compability. Signed-off-by: Oscar Salvador --- arch/sparc/kernel/sys_sparc_32.c | 17 +++++++++++---- arch/sparc/kernel/sys_sparc_64.c | 37 +++++++++++++++++++++++++------- 2 files changed, 42 insertions(+), 12 deletions(-) diff --git a/arch/sparc/kernel/sys_sparc_32.c b/arch/sparc/kernel/sys_sparc_32.c index 08a19727795c..852b340a7f9b 100644 --- a/arch/sparc/kernel/sys_sparc_32.c +++ b/arch/sparc/kernel/sys_sparc_32.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include @@ -42,12 +43,16 @@ SYSCALL_DEFINE0(getpagesize) unsigned long arch_get_unmapped_area(struct file *filp, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags) { struct vm_unmapped_area_info info = {}; + bool file_hugepage = false; + + if (filp && is_file_hugepages(filp)) + file_hugepage = true; if (flags & MAP_FIXED) { /* We do not accept a shared mapping if it would violate * cache aliasing constraints. */ - if ((flags & MAP_SHARED) && + if (!file_hugepage && (flags & MAP_SHARED) && ((addr - (pgoff << PAGE_SHIFT)) & (SHMLBA - 1))) return -EINVAL; return addr; @@ -62,9 +67,13 @@ unsigned long arch_get_unmapped_area(struct file *filp, unsigned long addr, unsi info.length = len; info.low_limit = addr; info.high_limit = TASK_SIZE; - info.align_mask = (flags & MAP_SHARED) ? - (PAGE_MASK & (SHMLBA - 1)) : 0; - info.align_offset = pgoff << PAGE_SHIFT; + if (!file_hugepage) { + info.align_mask = (flags & MAP_SHARED) ? + (PAGE_MASK & (SHMLBA - 1)) : 0; + info.align_offset = pgoff << PAGE_SHIFT; + } else { + info.align_mask = huge_page_mask_align(filp); + } return vm_unmapped_area(&info); } diff --git a/arch/sparc/kernel/sys_sparc_64.c b/arch/sparc/kernel/sys_sparc_64.c index d9c3b34ca744..1b271db41542 100644 --- a/arch/sparc/kernel/sys_sparc_64.c +++ b/arch/sparc/kernel/sys_sparc_64.c @@ -30,6 +30,7 @@ #include #include #include +#include #include #include @@ -87,6 +88,16 @@ static inline unsigned long COLOR_ALIGN(unsigned long addr, return base + off; } +static unsigned long get_align_mask(struct file *filp, unsigned long flags) +{ + if (filp && is_file_hugepages(filp)) + return huge_page_mask_align(filp); + if (filp || (flags & MAP_SHARED)) + return PAGE_MASK & (SHMLBA - 1); + + return 0; +} + unsigned long arch_get_unmapped_area(struct file *filp, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags) { struct mm_struct *mm = current->mm; @@ -94,12 +105,16 @@ unsigned long arch_get_unmapped_area(struct file *filp, unsigned long addr, unsi unsigned long task_size = TASK_SIZE; int do_color_align; struct vm_unmapped_area_info info = {}; + bool file_hugepage = false; + + if (filp && is_file_hugepages(filp)) + file_hugepage = true; if (flags & MAP_FIXED) { /* We do not accept a shared mapping if it would violate * cache aliasing constraints. */ - if ((flags & MAP_SHARED) && + if (!file_hugepage && (flags & MAP_SHARED) && ((addr - (pgoff << PAGE_SHIFT)) & (SHMLBA - 1))) return -EINVAL; return addr; @@ -111,7 +126,7 @@ unsigned long arch_get_unmapped_area(struct file *filp, unsigned long addr, unsi return -ENOMEM; do_color_align = 0; - if (filp || (flags & MAP_SHARED)) + if ((filp || (flags & MAP_SHARED)) && !file_hugepage) do_color_align = 1; if (addr) { @@ -129,8 +144,9 @@ unsigned long arch_get_unmapped_area(struct file *filp, unsigned long addr, unsi info.length = len; info.low_limit = TASK_UNMAPPED_BASE; info.high_limit = min(task_size, VA_EXCLUDE_START); - info.align_mask = do_color_align ? (PAGE_MASK & (SHMLBA - 1)) : 0; - info.align_offset = pgoff << PAGE_SHIFT; + info.align_mask = get_align_mask(filp, flags); + if (!file_hugepage) + info.align_offset = pgoff << PAGE_SHIFT; addr = vm_unmapped_area(&info); if ((addr & ~PAGE_MASK) && task_size > VA_EXCLUDE_END) { @@ -154,15 +170,19 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, unsigned long addr = addr0; int do_color_align; struct vm_unmapped_area_info info = {}; + bool file_hugepage = false; /* This should only ever run for 32-bit processes. */ BUG_ON(!test_thread_flag(TIF_32BIT)); + if (filp && is_file_hugepages(filp)) + file_hugepage = true; + if (flags & MAP_FIXED) { /* We do not accept a shared mapping if it would violate * cache aliasing constraints. */ - if ((flags & MAP_SHARED) && + if (!file_hugepage && (flags & MAP_SHARED) && ((addr - (pgoff << PAGE_SHIFT)) & (SHMLBA - 1))) return -EINVAL; return addr; @@ -172,7 +192,7 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, return -ENOMEM; do_color_align = 0; - if (filp || (flags & MAP_SHARED)) + if ((filp || (flags & MAP_SHARED)) && !file_hugepage) do_color_align = 1; /* requesting a specific address */ @@ -192,8 +212,9 @@ arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, info.length = len; info.low_limit = PAGE_SIZE; info.high_limit = mm->mmap_base; - info.align_mask = do_color_align ? (PAGE_MASK & (SHMLBA - 1)) : 0; - info.align_offset = pgoff << PAGE_SHIFT; + info.align_mask = get_align_mask(filp, flags); + if (!file_hugepage) + info.align_offset = pgoff << PAGE_SHIFT; addr = vm_unmapped_area(&info); /* From patchwork Mon Jul 29 09:10:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13744629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 483ADC3DA7F for ; Mon, 29 Jul 2024 09:10:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 119306B0093; Mon, 29 Jul 2024 05:10:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 07BDD6B0095; Mon, 29 Jul 2024 05:10:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DC2CA6B0096; Mon, 29 Jul 2024 05:10:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B515D6B0093 for ; Mon, 29 Jul 2024 05:10:43 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 46DA4401D9 for ; Mon, 29 Jul 2024 09:10:43 +0000 (UTC) X-FDA: 82392219966.23.FA123A4 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf07.hostedemail.com (Postfix) with ESMTP id 278C840005 for ; Mon, 29 Jul 2024 09:10:40 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722244170; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4aP7LJidVwaEiyg2vEhcsAqZZw1DICdTCoxEateoTEY=; b=IOm68sWpRmPyfoOmgGECh6ghDrz13XtSl90F4FkKADWkwHj7Bux3cQOBN9Vgeug45B/EBD QNugLXl15KCcqBNbv+w1ZxJP2YlMkjNuv2W0Onfgbb/uTkWqPD484bnoy5Y+h1GQ9NgIe2 vbunvdk8JO4qCjuf0nV9M4yUKLnFKKA= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722244170; a=rsa-sha256; cv=none; b=L1ckIliN/IXbgASduJbWbLCU1zqadpIvD6GiVGUwGqAzGNBZHKhxyjuXRHa1MKOIhHYzS2 Vnk+eF/mP2mRegIVi2YRdX1/ZYw1knRXwXfrGipEhjBsukl5Okv2B5ZDCidD7KT0XIfqxp fpokHwCu8hHdFfvflhTnCxV69G02Kqg= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id DE10A1F78C; Mon, 29 Jul 2024 09:10:39 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 9D0651368A; Mon, 29 Jul 2024 09:10:38 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id mLXnH45cp2Z4BgAAD6G6ig (envelope-from ); Mon, 29 Jul 2024 09:10:38 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , Donet Tom , Matthew Wilcox , Vlastimil Babka , Michal Hocko , Oscar Salvador Subject: [PATCH v2 5/9] arch/powerpc: Teach book3s64 arch_get_unmapped_area{_topdown} to handle hugetlb mappings Date: Mon, 29 Jul 2024 11:10:14 +0200 Message-ID: <20240729091018.2152-6-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240729091018.2152-1-osalvador@suse.de> References: <20240729091018.2152-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 278C840005 X-Stat-Signature: ceq9fzsi55srtsoemiprtn6psrhxmqfe X-Rspam-User: X-HE-Tag: 1722244240-554904 X-HE-Meta: U2FsdGVkX19vZGkHX1YcgOnqb/XFdcMpU++TP60H/zz8itgw1TuYE7VplwJx9xS4gZvoa0NJ7Cuq/fDs3Vl3538+sRWbqhPeR5Y+T4tmTgVVYENq8i00vnwPbZ4EazWINjUkmGSpNqaB0uuBXSajICjsU8orcC6+cR6Y0sllWlaI927u/u5RpGyooN6adsP0yRftB9Hr7mmFKss0bip1O/iO1R85wUCCgt0XFo7vRSLrWXeESzPye621Z3SNzLxUO9FCBAw7pgzli289deCVmaKWdS1e3DcGXavaKmDSvHK87A9/7VTTXu2silmVb8euDkDYAHAAzpUaPC0y8v+gPowjEkNNdCK7YXY9UfoZRWUllZq+IizWPYUWb7FIar0TOedZzOfyQH1mk5u6ABU97mvoSB6z2r4qkRm/H496ZkRP0gizLcaAG4WO1cuVuZWfwS/0+McAL7JnJ743W7B7jSEtiHOQfYfaUshy6D4Ad9A56XJZPKldg6/URsMMmCARfQa24BZ516VMFpFtIsH4Zwu8fwmulBNIytbiHKK/iIeoOQd9zSaksDTNe7flzeFXeo5+k5pkiOWNJzKXrAbxT6C+h85M8MTthBBoFhoekqWFJ79LJvW6GeTPdI96m+YGAQoLtF+gYbV8Kl841Cff9QBkgAoNBC4sI4cmJSCqRDTvfatH0axC33kBJtkxv7H8bBJVWE7mYpLnmCf3BWlh6vJDn08gFYOTpPl0ooIUzUHxMKRVttD28tIfCv16anE6BwSooQrA8snYNRUu0AmuQusu02x9/ZWi0At6lb9otsEjwq/Aqr4lhdOUuf/ZYYny5IokNQyUSEaysC6dIVn5keWmj4jGmmNroG5QCGUecPIadX25HGzBY4ifewmZ+arUnBRKN770KCV7ksFtf86y2wLAeWCvm9GNskfpxExD5aiu0uKjpiczAPCxz6GBRsrO04sjIHIlg0xLNnN5uP9 CswdcmzO L/zof10pc2hVjWyf1Hrhk+2pU0rVyIYKsy3jLQvTihHDeygN90Gho6sWdKdljaunLaO0KvIOy62wor25ozPZFvklwQKSWXrT0QsYpNQYBYc+uIN/P2EHOOrKofiMuhoSY+thlR6+NzgMjVsBcgmXxHFGXgSi9JEBe49EjAZpGIK4hZU0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We want to stop special casing hugetlb mappings and make them go through generic channels, so teach arch_get_unmapped_area{_topdown} to handle those. Reshuffle file_to_psize() definition so arch_get_unmapped_area{_topdown} can make use of it. Signed-off-by: Oscar Salvador --- arch/powerpc/mm/book3s64/slice.c | 39 ++++++++++++++++++++++++-------- 1 file changed, 29 insertions(+), 10 deletions(-) diff --git a/arch/powerpc/mm/book3s64/slice.c b/arch/powerpc/mm/book3s64/slice.c index ef3ce37f1bb3..6914b8de627c 100644 --- a/arch/powerpc/mm/book3s64/slice.c +++ b/arch/powerpc/mm/book3s64/slice.c @@ -633,17 +633,36 @@ unsigned long slice_get_unmapped_area(unsigned long addr, unsigned long len, } EXPORT_SYMBOL_GPL(slice_get_unmapped_area); +#ifdef CONFIG_HUGETLB_PAGE +static int file_to_psize(struct file *file) +{ + struct hstate *hstate = hstate_file(file); + return shift_to_mmu_psize(huge_page_shift(hstate)); +} +#else +static int file_to_psize(struct file *file) +{ + return 0; +} +#endif + unsigned long arch_get_unmapped_area(struct file *filp, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags) { + unsigned int psize; + if (radix_enabled()) return generic_get_unmapped_area(filp, addr, len, pgoff, flags); - return slice_get_unmapped_area(addr, len, flags, - mm_ctx_user_psize(¤t->mm->context), 0); + if (filp && is_file_hugepages(filp)) + psize = file_to_psize(filp); + else + psize = mm_ctx_user_psize(¤t->mm->context); + + return slice_get_unmapped_area(addr, len, flags, psize, 0); } unsigned long arch_get_unmapped_area_topdown(struct file *filp, @@ -652,11 +671,17 @@ unsigned long arch_get_unmapped_area_topdown(struct file *filp, const unsigned long pgoff, const unsigned long flags) { + unsigned int psize; + if (radix_enabled()) return generic_get_unmapped_area_topdown(filp, addr0, len, pgoff, flags); - return slice_get_unmapped_area(addr0, len, flags, - mm_ctx_user_psize(¤t->mm->context), 1); + if (filp && is_file_hugepages(filp)) + psize = file_to_psize(filp); + else + psize = mm_ctx_user_psize(¤t->mm->context); + + return slice_get_unmapped_area(addr0, len, flags, psize, 1); } unsigned int notrace get_slice_psize(struct mm_struct *mm, unsigned long addr) @@ -787,12 +812,6 @@ unsigned long vma_mmu_pagesize(struct vm_area_struct *vma) return 1UL << mmu_psize_to_shift(get_slice_psize(vma->vm_mm, vma->vm_start)); } -static int file_to_psize(struct file *file) -{ - struct hstate *hstate = hstate_file(file); - return shift_to_mmu_psize(huge_page_shift(hstate)); -} - unsigned long hugetlb_get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags) From patchwork Mon Jul 29 09:10:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13744630 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F24EC3DA61 for ; Mon, 29 Jul 2024 09:10:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 75E1B6B0095; Mon, 29 Jul 2024 05:10:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6BC166B0096; Mon, 29 Jul 2024 05:10:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 55DB26B0098; Mon, 29 Jul 2024 05:10:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3298F6B0095 for ; Mon, 29 Jul 2024 05:10:45 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id DFF6F140200 for ; Mon, 29 Jul 2024 09:10:44 +0000 (UTC) X-FDA: 82392220008.25.A7D24B3 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf20.hostedemail.com (Postfix) with ESMTP id AD8381C000A for ; Mon, 29 Jul 2024 09:10:42 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=NBotca+t; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=14BhT2RB; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=NBotca+t; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=14BhT2RB; spf=pass (imf20.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722244189; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0D03v0CNtg8d+Z2YRTExLn0FsjKOf1vcUEKVYVI3qZA=; b=NUcP+Rih3S+yQNNcpxzXeJ5gQGBtjp7eSRr35w+kdZUhi+4ntSeIf+jD8gda3m3Jbx/b7S ct+H3fc9fuov4nCgVKFapOUTc8bxKcJJicwgD/bQMCxuWaa7qxdg4JhkiGrig9BiuYOpCx JM7c3r+QkCEm1+H7cU5Q3cG6GV1TgVA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722244189; a=rsa-sha256; cv=none; b=NzRIHZhUjr1sPTCVWAqL+FtkxqlkjoKfq71nm99kG0foZDCDY6UsHvX0+0+iJASS8a+Boq ZYyfurcWBxY147Ms2tXy+ggpHgXPA1I88cnj4kK74B9Mu21OFHmbF3B/E9hOk5qYZUzS/e eHyNR+B8siCrsW1k9YtuZ8ymANuX8hU= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=NBotca+t; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=14BhT2RB; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=NBotca+t; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=14BhT2RB; spf=pass (imf20.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 6D81A21BC1; Mon, 29 Jul 2024 09:10:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1722244241; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0D03v0CNtg8d+Z2YRTExLn0FsjKOf1vcUEKVYVI3qZA=; b=NBotca+t+NBVJkyG1KJ2BeT9oBB/GpLOboqMQvQRyRmYQLl3N6NHlzOVlDWDH1sORtTOhA 1zUXHQqSgna1nRsWzLZzaXfnvWLAeI5drBIMYRa/l6YP5SLGieif5frv4j3qGDpo5bK2v7 dvqvFn4MiD+m6D0JmS4htP6SQZtPagI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1722244241; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0D03v0CNtg8d+Z2YRTExLn0FsjKOf1vcUEKVYVI3qZA=; b=14BhT2RBgWuQS9/HaH/h2wDy/QpKlbdFsF/BybzmpSt4rFaVYQ3UWlxckbnxKScwK1hSUj ZImZpXgjy2e1LlDw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1722244241; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0D03v0CNtg8d+Z2YRTExLn0FsjKOf1vcUEKVYVI3qZA=; b=NBotca+t+NBVJkyG1KJ2BeT9oBB/GpLOboqMQvQRyRmYQLl3N6NHlzOVlDWDH1sORtTOhA 1zUXHQqSgna1nRsWzLZzaXfnvWLAeI5drBIMYRa/l6YP5SLGieif5frv4j3qGDpo5bK2v7 dvqvFn4MiD+m6D0JmS4htP6SQZtPagI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1722244241; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0D03v0CNtg8d+Z2YRTExLn0FsjKOf1vcUEKVYVI3qZA=; b=14BhT2RBgWuQS9/HaH/h2wDy/QpKlbdFsF/BybzmpSt4rFaVYQ3UWlxckbnxKScwK1hSUj ZImZpXgjy2e1LlDw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 125F91368A; Mon, 29 Jul 2024 09:10:39 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id IAAiO49cp2Z4BgAAD6G6ig (envelope-from ); Mon, 29 Jul 2024 09:10:39 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , Donet Tom , Matthew Wilcox , Vlastimil Babka , Michal Hocko , Oscar Salvador Subject: [PATCH v2 6/9] mm: Make hugetlb mappings go through mm_get_unmapped_area_vmflags Date: Mon, 29 Jul 2024 11:10:15 +0200 Message-ID: <20240729091018.2152-7-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240729091018.2152-1-osalvador@suse.de> References: <20240729091018.2152-1-osalvador@suse.de> MIME-Version: 1.0 X-Stat-Signature: k7ak77knrzixs8xs1e99kyjzyor8m6bb X-Rspamd-Queue-Id: AD8381C000A X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1722244242-630769 X-HE-Meta: U2FsdGVkX1+yXQfxQy60GNdBrBZPVaCByrDayjuS0u6lIzu2FngmuKMJYBO6leQLbnt5UG4j0ZBKtbhICPrDBYxfqikDqhCPtElxcJT06zSiS66jEX+NI1aGwZ5wGgkGbSiDlopZ7U45fTQD5/ks8KeRlwEf96te3z4MEqu3WStHP1od/bHeLkq8d0kxHPK9LE+sXVRE1iicxcgoYU13IIYcuu83BCUPQYY8Kk/4wB+SaY8lUGmsMVn1qRmKH1HVgDL0I21es96YHQRvdUA9SHqDm9q5d5kQk0Jej4gb/TwodJ92anEyu6CkNN29iOksycm0MVjlBVZK+YfogiVCvTon9mJj1IrhH+1y+Itpow4ToIrjlMKO5DMrNr7Oxef4H1PKwhaQbINrAg2prME/QOUPTtIbkhCjhP0OKP93PtFf6wHdvNiyvjStMiNHwoDIRsQREtLFBT+v+uqRI2iaO6lLG02F5CXkjc/5AQh2Xn24yA+s6XKspDsDdQGxdspNtMQkdJDlJfi6k9a05jGxxjuMemXpxE8CI6lgsGlzQ+nyAiHd5XeS2P6nyB5R4yT3nDnrdQ+N93LbuL6K8S/Rb9RFaBYV71fCm0D0PlI5J4UEaECjLSdTB6+rdKQfS7hR13kQ1WALJLb8dM79K7uJ9k4GEQ5qs60/y+7Ks5VWuHkwkWmjdC8njV+eRvygiU/83ngNpZQh0kxAGEN3bPrySsRM5R5tnXzxhFKZtcfB8irkdn9/yL5sYIUW35r2h9XVYL/ZqgtoVhyap+CyI3BVACXClVpxkKUNqtzKj4i2vlKCmNZWARs33WjNFk9OMSSkx2BvACFCfQhw5VtHzFdjTgZZWmvQq94nzkfcwIsVBmCBKCTzQz2IJjlbrJRl2pDGSAaUOX8Ol2clGXT/MXvpg6vouEujfcWlFJQ3VY2pM5eHLkiJudmjT5A0CZ0WBCXgABO+4pUk3ZgL10RTywB P3R88YFB 4QEIA1GljAFvVE/CO+Am3xSWtKtU9F9AEnvuAF/QAMS7rnCuvzAzcs4CV3p4Wjaw/YWy+J7wf5bZWrj8Y/tz5gcYV1KbX0fD80Dm8PPJJQyV3sI14XOfpsPOfnA2XumnyKtD6//rhlWtnGXIvMDSXnhxQrm5HjjmVQskcBIT2AKp54yvQ2Rf1/JPcilf4hLtNnJdtW1ECPdkcUHjcRjrJVbnmDXpLy+Ka0AEoVmDnkVuIDo5MLgGcBD64gQS3phRojTqOqLZHvEZ0J4oCHFhAjDOKaSkUJEnAbd3b X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hugetlb mappings will no longer be special cased but rather go through the generic mm_get_unmapped_area_vmflags function. For that to happen, let us remove the .get_unmapped_area from hugetlbfs_file_operations struct, and hint __get_unmapped_area that it should not send hugetlb mappings through thp_get_unmapped_area_vmflags but through mm_get_unmapped_area_vmflags. Create also a function called hugetlb_mmap_check_and_align() where a couple of safety checks are being done and the addr is aligned to the huge page size. Otherwise we will have to do this in every single function, which duplicates quite a lot of code. Signed-off-by: Oscar Salvador --- fs/hugetlbfs/inode.c | 22 ++++++++++++++-------- include/linux/hugetlb.h | 8 +++----- mm/mmap.c | 15 ++++++++++++++- 3 files changed, 31 insertions(+), 14 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 9f6cff356796..5d47a2785a5d 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -258,15 +258,22 @@ generic_hugetlb_get_unmapped_area(struct file *file, unsigned long addr, pgoff, flags); } -#ifndef HAVE_ARCH_HUGETLB_UNMAPPED_AREA -static unsigned long -hugetlb_get_unmapped_area(struct file *file, unsigned long addr, - unsigned long len, unsigned long pgoff, - unsigned long flags) +unsigned long +hugetlb_mmap_check_and_align(struct file *file, unsigned long addr, + unsigned long len, unsigned long flags) { - return generic_hugetlb_get_unmapped_area(file, addr, len, pgoff, flags); + unsigned long addr0 = 0; + struct hstate *h = hstate_file(file); + + if (len & ~huge_page_mask(h)) + return -EINVAL; + if ((flags & MAP_FIXED) && prepare_hugepage_range(file, addr, len)) + return -EINVAL; + if (addr) + addr0 = ALIGN(addr, huge_page_size(h)); + + return addr0; } -#endif /* * Someone wants to read @bytes from a HWPOISON hugetlb @page from @offset. @@ -1300,7 +1307,6 @@ static const struct file_operations hugetlbfs_file_operations = { .read_iter = hugetlbfs_read_iter, .mmap = hugetlbfs_file_mmap, .fsync = noop_fsync, - .get_unmapped_area = hugetlb_get_unmapped_area, .llseek = default_llseek, .fallocate = hugetlbfs_fallocate, .fop_flags = FOP_HUGE_PAGES, diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 0ec14e5e0890..1413cdcfdb1a 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -549,11 +549,9 @@ static inline struct hstate *hstate_inode(struct inode *i) } #endif /* !CONFIG_HUGETLBFS */ -#ifdef HAVE_ARCH_HUGETLB_UNMAPPED_AREA -unsigned long hugetlb_get_unmapped_area(struct file *file, unsigned long addr, - unsigned long len, unsigned long pgoff, - unsigned long flags); -#endif /* HAVE_ARCH_HUGETLB_UNMAPPED_AREA */ +unsigned long +hugetlb_mmap_check_and_align(struct file *file, unsigned long addr, + unsigned long len, unsigned long flags); unsigned long generic_hugetlb_get_unmapped_area(struct file *file, unsigned long addr, diff --git a/mm/mmap.c b/mm/mmap.c index 7b623811d82a..f755d8a298c5 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -849,6 +849,7 @@ __get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, unsigned long, unsigned long, unsigned long) = NULL; + bool is_hugetlb = false; unsigned long error = arch_mmap_check(addr, len, flags); if (error) return error; @@ -857,6 +858,9 @@ __get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, if (len > TASK_SIZE) return -ENOMEM; + if (file && is_file_hugepages(file)) + is_hugetlb = true; + if (file) { if (file->f_op->get_unmapped_area) get_area = file->f_op->get_unmapped_area; @@ -874,11 +878,20 @@ __get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, if (get_area) { addr = get_area(file, addr, len, pgoff, flags); - } else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) { + } else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && !is_hugetlb) { /* Ensures that larger anonymous mappings are THP aligned. */ addr = thp_get_unmapped_area_vmflags(file, addr, len, pgoff, flags, vm_flags); } else { + /* + * Consolidate hugepages checks in one place, and also align addr + * to hugepage size. + */ + if (is_hugetlb) { + addr = hugetlb_mmap_check_and_align(file, addr, len, flags); + if (IS_ERR_VALUE(addr)) + return addr; + } addr = mm_get_unmapped_area_vmflags(current->mm, file, addr, len, pgoff, flags, vm_flags); } From patchwork Mon Jul 29 09:10:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13744631 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDBAAC3DA61 for ; Mon, 29 Jul 2024 09:10:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 696D36B0096; Mon, 29 Jul 2024 05:10:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 623636B0098; Mon, 29 Jul 2024 05:10:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3D7296B0099; Mon, 29 Jul 2024 05:10:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 107216B0096 for ; Mon, 29 Jul 2024 05:10:47 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id B7FC0401E1 for ; Mon, 29 Jul 2024 09:10:46 +0000 (UTC) X-FDA: 82392220092.09.75D25FE Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf03.hostedemail.com (Postfix) with ESMTP id 9AFAB20019 for ; Mon, 29 Jul 2024 09:10:44 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722244174; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=buSYe4q0DlUswFxOLdlprWQtcTb8oStiU1k0R3CKgPA=; b=HmIun93eiedNYJ0bKHyCJbEEb2q4zC9XqNanWvQ/hpIE7parrACXhbs4Q3E7hkRnEltbWa +OOebmfeNk5dp5DRFXTSfj/o1C9srv5GNaTrz+0SXTRxKi3CZEEbbkHEgtJckdFwQXVtgG czJN+Qmc/DKRxhLSdC1TUB7+Le5nWrU= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722244174; a=rsa-sha256; cv=none; b=bvWWpEYjy+AW2NV9cJXw/sEIvgZ0PICE0f5UAOEk7lQK8CJ+d41AsVs/k1H1ByY5grs3ri 2CnjM/16PQRPFrch5LsNLID4BPeI1bnlSMB1IV89IlNjo3vougLNxrp/OLS32LXiwlJGLp Am7UR+STxkq4uOv2anK8AOK1fru6ji4= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 5DB3821BDA; Mon, 29 Jul 2024 09:10:43 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 9DAF91368A; Mon, 29 Jul 2024 09:10:41 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id SJ2GIJFcp2Z4BgAAD6G6ig (envelope-from ); Mon, 29 Jul 2024 09:10:41 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , Donet Tom , Matthew Wilcox , Vlastimil Babka , Michal Hocko , Oscar Salvador Subject: [PATCH v2 7/9] mm: Drop hugetlb_get_unmapped_area{_*} functions Date: Mon, 29 Jul 2024 11:10:16 +0200 Message-ID: <20240729091018.2152-8-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240729091018.2152-1-osalvador@suse.de> References: <20240729091018.2152-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 9AFAB20019 X-Stat-Signature: p9zndz1pjoa1wdqfwdh54cmk4gjoiwgk X-Rspam-User: X-HE-Tag: 1722244244-478872 X-HE-Meta: U2FsdGVkX18wBQ02pf/d0oIIOzVJFup/pgzssTfMB247wC9WScrR4+40o5oZGsNyVE84QaOv2PaSBetMbsMcI3vDajFwnoVbO0zFuPrzbrSI/9IQDpaDHtYcjkA/OsvS1mwlNprMRmvwcXyDN43FkcMk1SKvbqA2hZdU8vhU7zpKH0H6m06dDXV9QenrCvK/Kvq4dWOzTASW8Ii1qgu5ZA2M5VpE45Gry51lX1wBjCWYTHqi8gXeqmxVDCgSVz4XkrcRBev6+HF+AMdgYAhl2e829ARhHpnzL7dtv+16evLh3soh6LkefIhZJyiXUG8FE4UR+EsYJvznomECQ/ibgHlxRKzoEt+cCzFAtYd+WtNtLRq2txU+oWXv/VcUh1DPRg2ZwMVTXf76/O+s9OV1oQKNzDSRdy0h/CHOPUV1yUPP4v+sDDbPpmMdDGGOC9wVriAhDXd8nMit3d5Wjs39i21hVhwhIZSzHwvbSF+YlGLlRMOSUVNMCNAhpGFlJtypQQqEGlkGtVERjozSJh8CeCUlWOBHgCkdcBESjs9A4ldYZN3XsVkIpZUYj86AkvG4uFt6WlXKGCE+Qtk5R3JWtpphWvCUIVOgFZb40oNUh5UnaRXp8oIRMmG79S9qXFmAQZSqPREx+6PecJUr9XcbpOeuWaXNP5Sag2SmIkXm6FCshHei/optiPrbmV7GQE5EZ1KR4AvqlDK8cSSE6Tmx8WBqnP87KtBzKjHxNTLqsUSANPtCjF5cAj6GE8asA4n6b5aq9J37kGpD3VPqmKRvePpQDJjvpdHaEG9IUpO7jloFGfbjT8AXFnl1pJi5BIHB7BIWYlev3J3AqJnU2yoBSmIejqzQI5KTAmpLCmumDmMS6hIuPNYCOdYf5KpZ4WRePEXp/El6tfT8rv1QWYhNjh4dqcSYEx+WGzgcLDPOsTaHtAh+x6lp7sK3OKplI+bVRHauFUEPcnzHSAFpg/i OnpG7lxk hGzqen1YsKucNwHWV6zMMFHuXCihyfCOucZ/oIXiLWiwcXWFniWtj4+zJbgn9g/A/vyjflFmbbW2bp1Ft3WpwwB76J0zyITvTNLn5xvn8YHhH6gY7GeOfaHzBUMzrIVXLmj+iENWpq7YQEbBHHfOkuDCiU+GfxHY1PuD4q8zqtt2wu9g= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hugetlb mappings are now handled through normal channels just like any other mapping, so we no longer need hugetlb_get_unmapped_area* specific functions. Signed-off-by: Oscar Salvador --- arch/parisc/mm/hugetlbpage.c | 21 ------ arch/powerpc/mm/book3s64/slice.c | 10 --- arch/s390/mm/hugetlbpage.c | 85 ------------------------ arch/sparc/mm/hugetlbpage.c | 108 ------------------------------- arch/x86/mm/hugetlbpage.c | 101 ----------------------------- fs/hugetlbfs/inode.c | 91 -------------------------- include/linux/hugetlb.h | 4 -- 7 files changed, 420 deletions(-) diff --git a/arch/parisc/mm/hugetlbpage.c b/arch/parisc/mm/hugetlbpage.c index 0356199bd9e7..e9d18cf25b79 100644 --- a/arch/parisc/mm/hugetlbpage.c +++ b/arch/parisc/mm/hugetlbpage.c @@ -21,27 +21,6 @@ #include -unsigned long -hugetlb_get_unmapped_area(struct file *file, unsigned long addr, - unsigned long len, unsigned long pgoff, unsigned long flags) -{ - struct hstate *h = hstate_file(file); - - if (len & ~huge_page_mask(h)) - return -EINVAL; - if (len > TASK_SIZE) - return -ENOMEM; - - if (flags & MAP_FIXED) - if (prepare_hugepage_range(file, addr, len)) - return -EINVAL; - - if (addr) - addr = ALIGN(addr, huge_page_size(h)); - - /* we need to make sure the colouring is OK */ - return arch_get_unmapped_area(file, addr, len, pgoff, flags); -} pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, diff --git a/arch/powerpc/mm/book3s64/slice.c b/arch/powerpc/mm/book3s64/slice.c index 6914b8de627c..6ce16bc330a2 100644 --- a/arch/powerpc/mm/book3s64/slice.c +++ b/arch/powerpc/mm/book3s64/slice.c @@ -811,14 +811,4 @@ unsigned long vma_mmu_pagesize(struct vm_area_struct *vma) return 1UL << mmu_psize_to_shift(get_slice_psize(vma->vm_mm, vma->vm_start)); } - -unsigned long hugetlb_get_unmapped_area(struct file *file, unsigned long addr, - unsigned long len, unsigned long pgoff, - unsigned long flags) -{ - if (radix_enabled()) - return generic_hugetlb_get_unmapped_area(file, addr, len, pgoff, flags); - - return slice_get_unmapped_area(addr, len, flags, file_to_psize(file), 1); -} #endif diff --git a/arch/s390/mm/hugetlbpage.c b/arch/s390/mm/hugetlbpage.c index ded0eff58a19..7c79cf1bc7d7 100644 --- a/arch/s390/mm/hugetlbpage.c +++ b/arch/s390/mm/hugetlbpage.c @@ -242,88 +242,3 @@ bool __init arch_hugetlb_valid_size(unsigned long size) else return false; } - -static unsigned long hugetlb_get_unmapped_area_bottomup(struct file *file, - unsigned long addr, unsigned long len, - unsigned long pgoff, unsigned long flags) -{ - struct hstate *h = hstate_file(file); - struct vm_unmapped_area_info info = {}; - - info.length = len; - info.low_limit = current->mm->mmap_base; - info.high_limit = TASK_SIZE; - info.align_mask = PAGE_MASK & ~huge_page_mask(h); - return vm_unmapped_area(&info); -} - -static unsigned long hugetlb_get_unmapped_area_topdown(struct file *file, - unsigned long addr0, unsigned long len, - unsigned long pgoff, unsigned long flags) -{ - struct hstate *h = hstate_file(file); - struct vm_unmapped_area_info info = {}; - unsigned long addr; - - info.flags = VM_UNMAPPED_AREA_TOPDOWN; - info.length = len; - info.low_limit = PAGE_SIZE; - info.high_limit = current->mm->mmap_base; - info.align_mask = PAGE_MASK & ~huge_page_mask(h); - addr = vm_unmapped_area(&info); - - /* - * A failed mmap() very likely causes application failure, - * so fall back to the bottom-up function here. This scenario - * can happen with large stack limits and large mmap() - * allocations. - */ - if (addr & ~PAGE_MASK) { - VM_BUG_ON(addr != -ENOMEM); - info.flags = 0; - info.low_limit = TASK_UNMAPPED_BASE; - info.high_limit = TASK_SIZE; - addr = vm_unmapped_area(&info); - } - - return addr; -} - -unsigned long hugetlb_get_unmapped_area(struct file *file, unsigned long addr, - unsigned long len, unsigned long pgoff, unsigned long flags) -{ - struct hstate *h = hstate_file(file); - struct mm_struct *mm = current->mm; - struct vm_area_struct *vma; - - if (len & ~huge_page_mask(h)) - return -EINVAL; - if (len > TASK_SIZE - mmap_min_addr) - return -ENOMEM; - - if (flags & MAP_FIXED) { - if (prepare_hugepage_range(file, addr, len)) - return -EINVAL; - goto check_asce_limit; - } - - if (addr) { - addr = ALIGN(addr, huge_page_size(h)); - vma = find_vma(mm, addr); - if (TASK_SIZE - len >= addr && addr >= mmap_min_addr && - (!vma || addr + len <= vm_start_gap(vma))) - goto check_asce_limit; - } - - if (!test_bit(MMF_TOPDOWN, &mm->flags)) - addr = hugetlb_get_unmapped_area_bottomup(file, addr, len, - pgoff, flags); - else - addr = hugetlb_get_unmapped_area_topdown(file, addr, len, - pgoff, flags); - if (offset_in_page(addr)) - return addr; - -check_asce_limit: - return check_asce_limit(mm, addr, len); -} diff --git a/arch/sparc/mm/hugetlbpage.c b/arch/sparc/mm/hugetlbpage.c index cc91ca7a1e18..eee601a0d2cf 100644 --- a/arch/sparc/mm/hugetlbpage.c +++ b/arch/sparc/mm/hugetlbpage.c @@ -19,114 +19,6 @@ #include #include -/* Slightly simplified from the non-hugepage variant because by - * definition we don't have to worry about any page coloring stuff - */ - -static unsigned long hugetlb_get_unmapped_area_bottomup(struct file *filp, - unsigned long addr, - unsigned long len, - unsigned long pgoff, - unsigned long flags) -{ - struct hstate *h = hstate_file(filp); - unsigned long task_size = TASK_SIZE; - struct vm_unmapped_area_info info = {}; - - if (test_thread_flag(TIF_32BIT)) - task_size = STACK_TOP32; - - info.length = len; - info.low_limit = TASK_UNMAPPED_BASE; - info.high_limit = min(task_size, VA_EXCLUDE_START); - info.align_mask = PAGE_MASK & ~huge_page_mask(h); - addr = vm_unmapped_area(&info); - - if ((addr & ~PAGE_MASK) && task_size > VA_EXCLUDE_END) { - VM_BUG_ON(addr != -ENOMEM); - info.low_limit = VA_EXCLUDE_END; - info.high_limit = task_size; - addr = vm_unmapped_area(&info); - } - - return addr; -} - -static unsigned long -hugetlb_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0, - const unsigned long len, - const unsigned long pgoff, - const unsigned long flags) -{ - struct hstate *h = hstate_file(filp); - struct mm_struct *mm = current->mm; - unsigned long addr = addr0; - struct vm_unmapped_area_info info = {}; - - /* This should only ever run for 32-bit processes. */ - BUG_ON(!test_thread_flag(TIF_32BIT)); - - info.flags = VM_UNMAPPED_AREA_TOPDOWN; - info.length = len; - info.low_limit = PAGE_SIZE; - info.high_limit = mm->mmap_base; - info.align_mask = PAGE_MASK & ~huge_page_mask(h); - addr = vm_unmapped_area(&info); - - /* - * A failed mmap() very likely causes application failure, - * so fall back to the bottom-up function here. This scenario - * can happen with large stack limits and large mmap() - * allocations. - */ - if (addr & ~PAGE_MASK) { - VM_BUG_ON(addr != -ENOMEM); - info.flags = 0; - info.low_limit = TASK_UNMAPPED_BASE; - info.high_limit = STACK_TOP32; - addr = vm_unmapped_area(&info); - } - - return addr; -} - -unsigned long -hugetlb_get_unmapped_area(struct file *file, unsigned long addr, - unsigned long len, unsigned long pgoff, unsigned long flags) -{ - struct hstate *h = hstate_file(file); - struct mm_struct *mm = current->mm; - struct vm_area_struct *vma; - unsigned long task_size = TASK_SIZE; - - if (test_thread_flag(TIF_32BIT)) - task_size = STACK_TOP32; - - if (len & ~huge_page_mask(h)) - return -EINVAL; - if (len > task_size) - return -ENOMEM; - - if (flags & MAP_FIXED) { - if (prepare_hugepage_range(file, addr, len)) - return -EINVAL; - return addr; - } - - if (addr) { - addr = ALIGN(addr, huge_page_size(h)); - vma = find_vma(mm, addr); - if (task_size - len >= addr && - (!vma || addr + len <= vm_start_gap(vma))) - return addr; - } - if (!test_bit(MMF_TOPDOWN, &mm->flags)) - return hugetlb_get_unmapped_area_bottomup(file, addr, len, - pgoff, flags); - else - return hugetlb_get_unmapped_area_topdown(file, addr, len, - pgoff, flags); -} static pte_t sun4u_hugepage_shift_to_tte(pte_t entry, unsigned int shift) { diff --git a/arch/x86/mm/hugetlbpage.c b/arch/x86/mm/hugetlbpage.c index 807a5859a3c4..58f7f2bd535d 100644 --- a/arch/x86/mm/hugetlbpage.c +++ b/arch/x86/mm/hugetlbpage.c @@ -19,107 +19,6 @@ #include #include -#ifdef CONFIG_HUGETLB_PAGE -static unsigned long hugetlb_get_unmapped_area_bottomup(struct file *file, - unsigned long addr, unsigned long len, - unsigned long pgoff, unsigned long flags) -{ - struct hstate *h = hstate_file(file); - struct vm_unmapped_area_info info = {}; - - info.length = len; - info.low_limit = get_mmap_base(1); - - /* - * If hint address is above DEFAULT_MAP_WINDOW, look for unmapped area - * in the full address space. - */ - info.high_limit = in_32bit_syscall() ? - task_size_32bit() : task_size_64bit(addr > DEFAULT_MAP_WINDOW); - - info.align_mask = PAGE_MASK & ~huge_page_mask(h); - return vm_unmapped_area(&info); -} - -static unsigned long hugetlb_get_unmapped_area_topdown(struct file *file, - unsigned long addr, unsigned long len, - unsigned long pgoff, unsigned long flags) -{ - struct hstate *h = hstate_file(file); - struct vm_unmapped_area_info info = {}; - - info.flags = VM_UNMAPPED_AREA_TOPDOWN; - info.length = len; - info.low_limit = PAGE_SIZE; - info.high_limit = get_mmap_base(0); - - /* - * If hint address is above DEFAULT_MAP_WINDOW, look for unmapped area - * in the full address space. - */ - if (addr > DEFAULT_MAP_WINDOW && !in_32bit_syscall()) - info.high_limit += TASK_SIZE_MAX - DEFAULT_MAP_WINDOW; - - info.align_mask = PAGE_MASK & ~huge_page_mask(h); - addr = vm_unmapped_area(&info); - - /* - * A failed mmap() very likely causes application failure, - * so fall back to the bottom-up function here. This scenario - * can happen with large stack limits and large mmap() - * allocations. - */ - if (addr & ~PAGE_MASK) { - VM_BUG_ON(addr != -ENOMEM); - info.flags = 0; - info.low_limit = TASK_UNMAPPED_BASE; - info.high_limit = TASK_SIZE_LOW; - addr = vm_unmapped_area(&info); - } - - return addr; -} - -unsigned long -hugetlb_get_unmapped_area(struct file *file, unsigned long addr, - unsigned long len, unsigned long pgoff, unsigned long flags) -{ - struct hstate *h = hstate_file(file); - struct mm_struct *mm = current->mm; - struct vm_area_struct *vma; - - if (len & ~huge_page_mask(h)) - return -EINVAL; - - if (len > TASK_SIZE) - return -ENOMEM; - - /* No address checking. See comment at mmap_address_hint_valid() */ - if (flags & MAP_FIXED) { - if (prepare_hugepage_range(file, addr, len)) - return -EINVAL; - return addr; - } - - if (addr) { - addr &= huge_page_mask(h); - if (!mmap_address_hint_valid(addr, len)) - goto get_unmapped_area; - - vma = find_vma(mm, addr); - if (!vma || addr + len <= vm_start_gap(vma)) - return addr; - } - -get_unmapped_area: - if (!test_bit(MMF_TOPDOWN, &mm->flags)) - return hugetlb_get_unmapped_area_bottomup(file, addr, len, - pgoff, flags); - else - return hugetlb_get_unmapped_area_topdown(file, addr, len, - pgoff, flags); -} -#endif /* CONFIG_HUGETLB_PAGE */ #ifdef CONFIG_X86_64 bool __init arch_hugetlb_valid_size(unsigned long size) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 5d47a2785a5d..3f0b8abbf851 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -167,97 +167,6 @@ static int hugetlbfs_file_mmap(struct file *file, struct vm_area_struct *vma) return ret; } -/* - * Called under mmap_write_lock(mm). - */ - -static unsigned long -hugetlb_get_unmapped_area_bottomup(struct file *file, unsigned long addr, - unsigned long len, unsigned long pgoff, unsigned long flags) -{ - struct hstate *h = hstate_file(file); - struct vm_unmapped_area_info info = {}; - - info.length = len; - info.low_limit = current->mm->mmap_base; - info.high_limit = arch_get_mmap_end(addr, len, flags); - info.align_mask = PAGE_MASK & ~huge_page_mask(h); - return vm_unmapped_area(&info); -} - -static unsigned long -hugetlb_get_unmapped_area_topdown(struct file *file, unsigned long addr, - unsigned long len, unsigned long pgoff, unsigned long flags) -{ - struct hstate *h = hstate_file(file); - struct vm_unmapped_area_info info = {}; - - info.flags = VM_UNMAPPED_AREA_TOPDOWN; - info.length = len; - info.low_limit = PAGE_SIZE; - info.high_limit = arch_get_mmap_base(addr, current->mm->mmap_base); - info.align_mask = PAGE_MASK & ~huge_page_mask(h); - addr = vm_unmapped_area(&info); - - /* - * A failed mmap() very likely causes application failure, - * so fall back to the bottom-up function here. This scenario - * can happen with large stack limits and large mmap() - * allocations. - */ - if (unlikely(offset_in_page(addr))) { - VM_BUG_ON(addr != -ENOMEM); - info.flags = 0; - info.low_limit = current->mm->mmap_base; - info.high_limit = arch_get_mmap_end(addr, len, flags); - addr = vm_unmapped_area(&info); - } - - return addr; -} - -unsigned long -generic_hugetlb_get_unmapped_area(struct file *file, unsigned long addr, - unsigned long len, unsigned long pgoff, - unsigned long flags) -{ - struct mm_struct *mm = current->mm; - struct vm_area_struct *vma, *prev; - struct hstate *h = hstate_file(file); - const unsigned long mmap_end = arch_get_mmap_end(addr, len, flags); - - if (len & ~huge_page_mask(h)) - return -EINVAL; - if (len > mmap_end - mmap_min_addr) - return -ENOMEM; - - if (flags & MAP_FIXED) { - if (prepare_hugepage_range(file, addr, len)) - return -EINVAL; - return addr; - } - - if (addr) { - addr = ALIGN(addr, huge_page_size(h)); - vma = find_vma_prev(mm, addr, &prev); - if (mmap_end - len >= addr && addr >= mmap_min_addr && - (!vma || addr + len <= vm_start_gap(vma)) && - (!prev || addr >= vm_end_gap(prev))) - return addr; - } - - /* - * Use MMF_TOPDOWN flag as a hint to use topdown routine. - * If architectures have special needs, they should define their own - * version of hugetlb_get_unmapped_area. - */ - if (test_bit(MMF_TOPDOWN, &mm->flags)) - return hugetlb_get_unmapped_area_topdown(file, addr, len, - pgoff, flags); - return hugetlb_get_unmapped_area_bottomup(file, addr, len, - pgoff, flags); -} - unsigned long hugetlb_mmap_check_and_align(struct file *file, unsigned long addr, unsigned long len, unsigned long flags) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 1413cdcfdb1a..bd82a0f34275 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -553,10 +553,6 @@ unsigned long hugetlb_mmap_check_and_align(struct file *file, unsigned long addr, unsigned long len, unsigned long flags); -unsigned long -generic_hugetlb_get_unmapped_area(struct file *file, unsigned long addr, - unsigned long len, unsigned long pgoff, - unsigned long flags); /* * huegtlb page specific state flags. These flags are located in page.private From patchwork Mon Jul 29 09:10:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13744632 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 222BDC3DA61 for ; Mon, 29 Jul 2024 09:10:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2C6FD6B0098; Mon, 29 Jul 2024 05:10:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 24EF66B0099; Mon, 29 Jul 2024 05:10:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0A18C6B009A; Mon, 29 Jul 2024 05:10:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id DD17D6B0098 for ; Mon, 29 Jul 2024 05:10:48 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 97869C01EF for ; Mon, 29 Jul 2024 09:10:48 +0000 (UTC) X-FDA: 82392220176.17.CFA1DAE Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf10.hostedemail.com (Postfix) with ESMTP id 70F17C0022 for ; Mon, 29 Jul 2024 09:10:46 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b="volHx/Xn"; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=knVJwYfi; dkim=pass header.d=suse.de header.s=susede2_rsa header.b="volHx/Xn"; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=knVJwYfi; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf10.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722244243; a=rsa-sha256; cv=none; b=NdlspAlkaxdG1D5kg+H5hFZL+1q4OdGVI3SjmvIlbiZ55SiOOETvyfOYXUUODJsErEMGj6 6Oem7WcxCFe1nHugHQK/Lh1A/oDkdH4yIad2GHeGe8sDnsp3aHYb0DtC0JWKwV7j8F6JTR XjAu14UNAZR+4f0XHXXkUUOOCwfGfI4= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b="volHx/Xn"; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=knVJwYfi; dkim=pass header.d=suse.de header.s=susede2_rsa header.b="volHx/Xn"; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=knVJwYfi; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf10.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722244243; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JABgt96y6DRapGbl7s1JS+H2Wcm/YvsqoufOGq2+3+w=; b=jGdsnON0z+YckjkIr8kAY8VxhRxXphckylg2cASQEfdQQMeaXZHo3hkz48PP0t9Fv3s0Js 6ts20zaexjFX0laHaOQg/BNfxpZpicwp27lSD4smTnb2ts3b3/NgAGOv8jYqkuU+sQqfW0 l9+rHJAJkgQBY0uUcIQj59W5R18Do0M= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 258031F793; Mon, 29 Jul 2024 09:10:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1722244245; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JABgt96y6DRapGbl7s1JS+H2Wcm/YvsqoufOGq2+3+w=; b=volHx/XnPqMA5RFciXwgbwhjRFqMgyRXT4+kbJpga0ccIN8LhNl3ops1qKrZkwIAwGqnD4 B1A6WZEyCFCAak3QmYguTubIpnY2H2K/wyb4b4N/3HfYZ/ZsDcly7oL1N5D2P/FTPWFv5v tKhgYrSOZ+J9hxQ+zN7o/2YcyQVYKc8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1722244245; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JABgt96y6DRapGbl7s1JS+H2Wcm/YvsqoufOGq2+3+w=; b=knVJwYfiadUdI5HIXe4e8AZ6S+VUmm7xE/xmqdsqxRUJ+CsFVcNU1YW9RneEdovsyeiL2V i9lLuNjU7s0+WpCg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1722244245; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JABgt96y6DRapGbl7s1JS+H2Wcm/YvsqoufOGq2+3+w=; b=volHx/XnPqMA5RFciXwgbwhjRFqMgyRXT4+kbJpga0ccIN8LhNl3ops1qKrZkwIAwGqnD4 B1A6WZEyCFCAak3QmYguTubIpnY2H2K/wyb4b4N/3HfYZ/ZsDcly7oL1N5D2P/FTPWFv5v tKhgYrSOZ+J9hxQ+zN7o/2YcyQVYKc8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1722244245; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JABgt96y6DRapGbl7s1JS+H2Wcm/YvsqoufOGq2+3+w=; b=knVJwYfiadUdI5HIXe4e8AZ6S+VUmm7xE/xmqdsqxRUJ+CsFVcNU1YW9RneEdovsyeiL2V i9lLuNjU7s0+WpCg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 89CA11368A; Mon, 29 Jul 2024 09:10:43 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id MMqyG5Ncp2Z4BgAAD6G6ig (envelope-from ); Mon, 29 Jul 2024 09:10:43 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , Donet Tom , Matthew Wilcox , Vlastimil Babka , Michal Hocko , Oscar Salvador Subject: [PATCH v2 8/9] arch/s390: Clean up hugetlb definitions Date: Mon, 29 Jul 2024 11:10:17 +0200 Message-ID: <20240729091018.2152-9-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240729091018.2152-1-osalvador@suse.de> References: <20240729091018.2152-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 70F17C0022 X-Rspamd-Server: rspam01 X-Stat-Signature: emas1aocnrd5hcep8cry3qm8t4hb9uxp X-HE-Tag: 1722244246-842967 X-HE-Meta: U2FsdGVkX18w291VUk4RfOk53ngn22+xcITshR2O8y09BtExzUufwCj2THTorRCNyDMgLfInvTaN5f+7U/mqWmQlIxXtDA4V7oKgWOb2OEzztxjSaQ+f84722FK8ihrA40pQrH+ZA387vTjrNe4bt+grmYQlacbE9nU00w79a0WdpXBwG52V0EmeAzYeFE1QIOuAW9PbclQL577gXOMNgynNhMbUK5wVeXWdXTx0hIfSrSY+LcWDKIBFiaA8/V3Mb9H3T2y0hyVUr5cWryClApkyr0NKe+iVLEzVFfWLFhVN4sYRSKKoaA4+8/5NAG4fXfbOVXUmVJy/chUquQ+FO0e7n5jVj58nPKUFyniOu73pxcwmJ43UM3jKfLBDDnQPQv/79miKq8BQ0bCGoudNGjoUOsAzgeVzyhOPv7/LhYSjnDnLlj4FHIJaKNbt51cUwDuC0Vk+j9CNsBuIVRhQnFkyVwvhwEI6Dg2QgAD2hXL0efx2AykCWp3u0knDnVQaoRAZUTfDO5nsdCtOd7jCGN5fvRCay4jM5h5larO108fSctDs/7kdgOLwV7X97Js26oOaH00pXhUP77NojZ9EPX/cPY9kglYvfF4fZiHfVfRUUkSQEXhgiKhycWnSH9Yte7R9dH4ilyKLAy82ACofD9jYFcsV/LxvwYB746j8tbCDKKMUOsjvp2J7VpEz1qtsAVJMm4cdMRy/2UOE696CVW3kgTxGvRnPhXyJs3AkBZV6AVlPTmckMeymuMHxISNiEPAX0ZVV3Qutmh9FBshmlhbRcT9+Zv1nUFGDlWMBm1r6oy7VyJBCb2/1goA0alJCzLuzvh5el4Q/Eof2e9DDa0KonA2hnUiKxMfv+y+GAlkBJQYTA60WsbpDwUoU568ov7nYSfcQHGLrcK2l3mtDQ49HbuehtEWgGvNERg52s6r+KnZdippEQapTVlatBm72+mYZ2+EGLdjksarcWcR c7INBWep 4UJ0FJCMLi4bgnBR7D2I2Y0Sp3D3kx8KWm/PeoxLCSObdbmTotAAt07kAEz3EbAvT3588YpbDqlXdaqcosAn6Ij6jn7gffzP5pxiXJrMm/CBAjYiYFhpbzeBx55UtFHp4WAf8R01aXinFNnU3xR2audbSn88L6AEPpAQLyP24dmg5SkRwlmfUy3aD5a96RtiE5hMJHfWBOrQucCGdR92DZAWLNzTvf9Ipdm0HURA0KQ7C3D4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: s390 redefines functions that are already defined (and the same) in include/asm-generic/hugetlb.h. Do as the other architectures: 1) include include/asm-generic/hugetlb.h 2) drop the already defined functions in the generic hugetlb.h and 3) use the __HAVE_ARCH_HUGE_* macros to define our own. This gets rid of quite some code. Signed-off-by: Oscar Salvador --- arch/s390/include/asm/hugetlb.h | 58 +++++++++------------------------ include/asm-generic/hugetlb.h | 8 +++++ 2 files changed, 24 insertions(+), 42 deletions(-) diff --git a/arch/s390/include/asm/hugetlb.h b/arch/s390/include/asm/hugetlb.h index cf1b5d6fb1a6..37e80a32623a 100644 --- a/arch/s390/include/asm/hugetlb.h +++ b/arch/s390/include/asm/hugetlb.h @@ -12,21 +12,24 @@ #include #include -#define hugetlb_free_pgd_range free_pgd_range #define hugepages_supported() (MACHINE_HAS_EDAT1) +#define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, unsigned long sz); void __set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte); -pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep); -pte_t huge_ptep_get_and_clear(struct mm_struct *mm, +#define __HAVE_ARCH_HUGE_PTEP_GET +extern pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep); +#define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR +extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep); /* * If the arch doesn't supply something else, assume that hugepage * size aligned regions are ok without further preparation. */ +#define __HAVE_ARCH_PREPARE_HUGEPAGE_RANGE static inline int prepare_hugepage_range(struct file *file, unsigned long addr, unsigned long len) { @@ -45,6 +48,7 @@ static inline void arch_clear_hugetlb_flags(struct folio *folio) } #define arch_clear_hugetlb_flags arch_clear_hugetlb_flags +#define __HAVE_ARCH_HUGE_PTE_CLEAR static inline void huge_pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, unsigned long sz) { @@ -54,12 +58,14 @@ static inline void huge_pte_clear(struct mm_struct *mm, unsigned long addr, set_pte(ptep, __pte(_SEGMENT_ENTRY_EMPTY)); } +#define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH static inline pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long address, pte_t *ptep) { return huge_ptep_get_and_clear(vma->vm_mm, address, ptep); } +#define __HAVE_ARCH_HUGE_PTEP_SET_ACCESS_FLAGS static inline int huge_ptep_set_access_flags(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep, pte_t pte, int dirty) @@ -72,6 +78,7 @@ static inline int huge_ptep_set_access_flags(struct vm_area_struct *vma, return changed; } +#define __HAVE_ARCH_HUGE_PTEP_SET_WRPROTECT static inline void huge_ptep_set_wrprotect(struct mm_struct *mm, unsigned long addr, pte_t *ptep) { @@ -79,69 +86,36 @@ static inline void huge_ptep_set_wrprotect(struct mm_struct *mm, __set_huge_pte_at(mm, addr, ptep, pte_wrprotect(pte)); } -static inline pte_t mk_huge_pte(struct page *page, pgprot_t pgprot) -{ - return mk_pte(page, pgprot); -} - +#define __HAVE_ARCH_HUGE_PTE_NONE static inline int huge_pte_none(pte_t pte) { return pte_none(pte); } +#define __HAVE_ARCH_HUGE_PTE_NONE_MOSTLY static inline int huge_pte_none_mostly(pte_t pte) { return huge_pte_none(pte); } -static inline int huge_pte_write(pte_t pte) -{ - return pte_write(pte); -} - -static inline int huge_pte_dirty(pte_t pte) -{ - return pte_dirty(pte); -} - -static inline pte_t huge_pte_mkwrite(pte_t pte) -{ - return pte_mkwrite_novma(pte); -} - -static inline pte_t huge_pte_mkdirty(pte_t pte) -{ - return pte_mkdirty(pte); -} - -static inline pte_t huge_pte_wrprotect(pte_t pte) -{ - return pte_wrprotect(pte); -} - -static inline pte_t huge_pte_modify(pte_t pte, pgprot_t newprot) -{ - return pte_modify(pte, newprot); -} - +#define __HAVE_ARCH_HUGE_PTE_MKUFFD_WP static inline pte_t huge_pte_mkuffd_wp(pte_t pte) { return pte; } +#define __HAVE_ARCH_HUGE_PTE_CLEAR_UFFD_WP static inline pte_t huge_pte_clear_uffd_wp(pte_t pte) { return pte; } +#define __HAVE_ARCH_HUGE_PTE_UFFD_WP static inline int huge_pte_uffd_wp(pte_t pte) { return 0; } -static inline bool gigantic_page_runtime_supported(void) -{ - return true; -} +#include #endif /* _ASM_S390_HUGETLB_H */ diff --git a/include/asm-generic/hugetlb.h b/include/asm-generic/hugetlb.h index 594d5905f615..67bbdafcfc22 100644 --- a/include/asm-generic/hugetlb.h +++ b/include/asm-generic/hugetlb.h @@ -42,20 +42,26 @@ static inline pte_t huge_pte_modify(pte_t pte, pgprot_t newprot) return pte_modify(pte, newprot); } +#ifndef __HAVE_ARCH_HUGE_PTE_MKUFFD_WP static inline pte_t huge_pte_mkuffd_wp(pte_t pte) { return huge_pte_wrprotect(pte_mkuffd_wp(pte)); } +#endif +#ifndef __HAVE_ARCH_HUGE_PTE_CLEAR_UFFD_WP static inline pte_t huge_pte_clear_uffd_wp(pte_t pte) { return pte_clear_uffd_wp(pte); } +#endif +#ifndef __HAVE_ARCH_HUGE_PTE_UFFD_WP static inline int huge_pte_uffd_wp(pte_t pte) { return pte_uffd_wp(pte); } +#endif #ifndef __HAVE_ARCH_HUGE_PTE_CLEAR static inline void huge_pte_clear(struct mm_struct *mm, unsigned long addr, @@ -106,10 +112,12 @@ static inline int huge_pte_none(pte_t pte) #endif /* Please refer to comments above pte_none_mostly() for the usage */ +#ifndef __HAVE_ARCH_HUGE_PTE_NONE_MOSTLY static inline int huge_pte_none_mostly(pte_t pte) { return huge_pte_none(pte) || is_pte_marker(pte); } +#endif #ifndef __HAVE_ARCH_PREPARE_HUGEPAGE_RANGE static inline int prepare_hugepage_range(struct file *file, From patchwork Mon Jul 29 09:10:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13744633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02417C3DA4A for ; Mon, 29 Jul 2024 09:11:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 07EAD6B0099; Mon, 29 Jul 2024 05:10:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EFB306B009A; Mon, 29 Jul 2024 05:10:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB31C6B009B; Mon, 29 Jul 2024 05:10:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A7ECF6B0099 for ; Mon, 29 Jul 2024 05:10:50 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 236AA120223 for ; Mon, 29 Jul 2024 09:10:50 +0000 (UTC) X-FDA: 82392220260.26.0F5DDDB Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf03.hostedemail.com (Postfix) with ESMTP id EDAF120021 for ; Mon, 29 Jul 2024 09:10:47 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf03.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722244188; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HUoJ5P1SSuIDOAYoJjkR+eCpNCd+zUZ3Jg0HpuhS4Gw=; b=X+ahm+NVgFRhsCb2tNkmxIdlBAiCx+od3HMeA0/mlN+68bVSlWSoL56xgEfLwSozeb14Jh U0iZWjQ2f+tGI/1NIbuopZLWOVMqJwjdAXSusHFFzGCq6zhS8rJ4/T8P+cMLoO0XLffAX7 /wyM77pEJkf9/ezerrinZyfb+ipkcOI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722244188; a=rsa-sha256; cv=none; b=DVhfy2hedA+NTsms59z45dPrGZQ20c/i7uZ+E1933YhqWATPEj1lBSP/tOu8Adpemiuc00 7T9v05T+gZDDx6aJXp6pCYztOSLOtGURMTganaKo9wM25HrG3sbCeTwO0+5T8u6TVNHfxK hunZkLjqkMXfi8wfZiIBRAhvo8mJ9dw= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf03.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.131 as permitted sender) smtp.mailfrom=osalvador@suse.de Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 9E4FF1F794; Mon, 29 Jul 2024 09:10:46 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4411D1368A; Mon, 29 Jul 2024 09:10:45 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id iPArC5Vcp2Z4BgAAD6G6ig (envelope-from ); Mon, 29 Jul 2024 09:10:45 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , Donet Tom , Matthew Wilcox , Vlastimil Babka , Michal Hocko , Oscar Salvador Subject: [PATCH v2 9/9] mm: Consolidate common checks in hugetlb_mmap_check_and_align Date: Mon, 29 Jul 2024 11:10:18 +0200 Message-ID: <20240729091018.2152-10-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240729091018.2152-1-osalvador@suse.de> References: <20240729091018.2152-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Action: no action X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: EDAF120021 X-Stat-Signature: gdfwopnig8u5mcfjkyp3mch5uyhuoxcn X-Rspam-User: X-HE-Tag: 1722244247-542115 X-HE-Meta: U2FsdGVkX18+Km3IerfiyhfU0YiVk3zSofwdMvY5BLBfNjYc3AbpdtTyrckqccewMK6vGMWSJmO1OTYb2qbRoRAbFabXNaWoFO3VPaFw6vAGBHOWYL6FYks5K7iKh1RLXCuCrSbjFgvcsINJdviYUGnGbvacMtZkcfYujdvv9XDWm3QkHUluctOwOjRRW2AnOXdVRTAB9zN0pMbpoEBotnJMvkwA1WlcYlNL3reAkeUnoeKp+GZSpGw2Ce28IGQfhAywlhaFCBWdtzeWQ6kthKqN3oSfvYbhI8Ap53FODrdEPFTrl/JcpXe1U1DAhwIvsQjKRNYu1vHZAt4Yw9VGlLNm7KPxw95AfZfQUuqRbssK8QDf8VnPlu8qvvuMv35j71/S1XhbTRcRjUHYcVtZwrZmDQShvWmfq2u6UCXvNhVG5eaFGSDjjzcJHoIle3zTdoRdu2CMFMkWilqtpHsNhykBFd367IQAN/g+KdCNcDEtgR+jaemjI2V5CFdWiN/5B9/G5+rEbdowvMTJlHYNuaCKz+wnlbXlI8g2TCyTshO6qT4044yG8TlGV+KotVHR8mYm0p9mO24k4o3P17dreXacNDsiVu1ZXqnfA6VdF2Flvjdq/KhzWVfOxE5zO4aCzrZSmu8j0kEDAilZ21yyXjf0ZSIWZF+ENdTd5IbzGNmvfxf+tLwoHaolt16BDePLaurp1bylozeBaLDe4wkrfbLTEnWBUQSCRz73/o1u1p+Ua7RBxccuL6moFz6u87cdzW4D7v1D2yC2H8EpmI+PLUUuFoy8ZviZjDPKg83O9k1rVcTS5k6TSEkkalK+qxRKwJwgNuNTPNz1gOMfhYe1nCwXsDsddAxv0PxrOge4oqWI7xOBJYPJf7HSXrttrv5ot2anp8UMKkm4vApadeMwZmPS4n+lVNasBGFVJ/PA2oW3bSYF+XkfrRJGAicXj+VLW/aJ53MSQFavi47tzrR 2H9WLvpc mIhmuSn0bmiVeyvJeQL4M+MmQHRQX+gmQ/NxVvR7HHaNBWSRT46o23bBFQFocZ4mCYqDnAGsnq00cpYqpD4T1ANJ9SsHIGch+/1OICRlm5sS+4i0XwD4yiaSjGX65SB05RuFWeTxWLfWeKRf5aQozjdgVvpsK2QroOTH86GclKQhi7R0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: prepare_hugepage_range() performs almost the same checks for all architectures that define it, with the exception of mips and loongarch that also check for overflows. The rest checks for the addr and len to be properly aligned, so we can move that to the generic hugetlb_mmap_check_and_align() function and get rid of a fair amount of duplicated code. Signed-off-by: Oscar Salvador --- arch/loongarch/include/asm/hugetlb.h | 4 ---- arch/mips/include/asm/hugetlb.h | 4 ---- arch/parisc/include/asm/hugetlb.h | 15 --------------- arch/s390/include/asm/hugetlb.h | 17 ----------------- arch/sh/include/asm/hugetlb.h | 15 --------------- fs/hugetlbfs/inode.c | 8 ++++++-- include/asm-generic/hugetlb.h | 7 ------- 7 files changed, 6 insertions(+), 64 deletions(-) diff --git a/arch/loongarch/include/asm/hugetlb.h b/arch/loongarch/include/asm/hugetlb.h index aa44b3fe43dd..107566c98938 100644 --- a/arch/loongarch/include/asm/hugetlb.h +++ b/arch/loongarch/include/asm/hugetlb.h @@ -18,10 +18,6 @@ static inline int prepare_hugepage_range(struct file *file, unsigned long task_size = STACK_TOP; struct hstate *h = hstate_file(file); - if (len & ~huge_page_mask(h)) - return -EINVAL; - if (addr & ~huge_page_mask(h)) - return -EINVAL; if (len > task_size) return -ENOMEM; if (task_size - len < addr) diff --git a/arch/mips/include/asm/hugetlb.h b/arch/mips/include/asm/hugetlb.h index fd69c8808554..6a63d82a8ab3 100644 --- a/arch/mips/include/asm/hugetlb.h +++ b/arch/mips/include/asm/hugetlb.h @@ -19,10 +19,6 @@ static inline int prepare_hugepage_range(struct file *file, unsigned long task_size = STACK_TOP; struct hstate *h = hstate_file(file); - if (len & ~huge_page_mask(h)) - return -EINVAL; - if (addr & ~huge_page_mask(h)) - return -EINVAL; if (len > task_size) return -ENOMEM; if (task_size - len < addr) diff --git a/arch/parisc/include/asm/hugetlb.h b/arch/parisc/include/asm/hugetlb.h index 72daacc472a0..5b3a5429f71b 100644 --- a/arch/parisc/include/asm/hugetlb.h +++ b/arch/parisc/include/asm/hugetlb.h @@ -12,21 +12,6 @@ void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep); -/* - * If the arch doesn't supply something else, assume that hugepage - * size aligned regions are ok without further preparation. - */ -#define __HAVE_ARCH_PREPARE_HUGEPAGE_RANGE -static inline int prepare_hugepage_range(struct file *file, - unsigned long addr, unsigned long len) -{ - if (len & ~HPAGE_MASK) - return -EINVAL; - if (addr & ~HPAGE_MASK) - return -EINVAL; - return 0; -} - #define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH static inline pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) diff --git a/arch/s390/include/asm/hugetlb.h b/arch/s390/include/asm/hugetlb.h index 37e80a32623a..6f815d4ba0ca 100644 --- a/arch/s390/include/asm/hugetlb.h +++ b/arch/s390/include/asm/hugetlb.h @@ -25,23 +25,6 @@ extern pte_t huge_ptep_get(struct mm_struct *mm, unsigned long addr, pte_t *ptep extern pte_t huge_ptep_get_and_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep); -/* - * If the arch doesn't supply something else, assume that hugepage - * size aligned regions are ok without further preparation. - */ -#define __HAVE_ARCH_PREPARE_HUGEPAGE_RANGE -static inline int prepare_hugepage_range(struct file *file, - unsigned long addr, unsigned long len) -{ - struct hstate *h = hstate_file(file); - - if (len & ~huge_page_mask(h)) - return -EINVAL; - if (addr & ~huge_page_mask(h)) - return -EINVAL; - return 0; -} - static inline void arch_clear_hugetlb_flags(struct folio *folio) { clear_bit(PG_arch_1, &folio->flags); diff --git a/arch/sh/include/asm/hugetlb.h b/arch/sh/include/asm/hugetlb.h index 75028bd568ba..4a92e6e4d627 100644 --- a/arch/sh/include/asm/hugetlb.h +++ b/arch/sh/include/asm/hugetlb.h @@ -5,21 +5,6 @@ #include #include -/* - * If the arch doesn't supply something else, assume that hugepage - * size aligned regions are ok without further preparation. - */ -#define __HAVE_ARCH_PREPARE_HUGEPAGE_RANGE -static inline int prepare_hugepage_range(struct file *file, - unsigned long addr, unsigned long len) -{ - if (len & ~HPAGE_MASK) - return -EINVAL; - if (addr & ~HPAGE_MASK) - return -EINVAL; - return 0; -} - #define __HAVE_ARCH_HUGE_PTEP_CLEAR_FLUSH static inline pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 3f0b8abbf851..1e35f0d3698b 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -176,8 +176,12 @@ hugetlb_mmap_check_and_align(struct file *file, unsigned long addr, if (len & ~huge_page_mask(h)) return -EINVAL; - if ((flags & MAP_FIXED) && prepare_hugepage_range(file, addr, len)) - return -EINVAL; + if (flags & MAP_FIXED) { + if (addr & ~huge_page_mask(h)) + return -EINVAL; + if (prepare_hugepage_range(file, addr, len)) + return -EINVAL; + } if (addr) addr0 = ALIGN(addr, huge_page_size(h)); diff --git a/include/asm-generic/hugetlb.h b/include/asm-generic/hugetlb.h index 67bbdafcfc22..f42133dae68e 100644 --- a/include/asm-generic/hugetlb.h +++ b/include/asm-generic/hugetlb.h @@ -123,13 +123,6 @@ static inline int huge_pte_none_mostly(pte_t pte) static inline int prepare_hugepage_range(struct file *file, unsigned long addr, unsigned long len) { - struct hstate *h = hstate_file(file); - - if (len & ~huge_page_mask(h)) - return -EINVAL; - if (addr & ~huge_page_mask(h)) - return -EINVAL; - return 0; } #endif