From patchwork Wed Jul 10 10:50:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13729175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F7FFC3DA42 for ; Wed, 10 Jul 2024 10:51:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 63EF16B00A2; Wed, 10 Jul 2024 06:51:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 57A846B00A3; Wed, 10 Jul 2024 06:51:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 37C4D6B00A4; Wed, 10 Jul 2024 06:51:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 16AAD6B00A2 for ; Wed, 10 Jul 2024 06:51:34 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id C7B3A1C1B15 for ; Wed, 10 Jul 2024 10:51:33 +0000 (UTC) X-FDA: 82323526866.13.D3814A9 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf16.hostedemail.com (Postfix) with ESMTP id 9B92A18001B for ; Wed, 10 Jul 2024 10:51:31 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=fSC1Gv8H; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=bYfs3lDT; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=fSC1Gv8H; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=bYfs3lDT; spf=pass (imf16.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720608676; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ph9NUUUfzYdgAX+SWvx0C4X8fQI8fkpyj62wUbkyYlY=; b=o18gCJf8umAA9IWuGJD2vHj0qAGAohpV4xMG0iyUlLvWXA8NzAezdrteUOmsFh86Rgz6hb rpmx/g9TSXxGYYEZVEiEt30I/clTX2XSJ6d5JcFgNtxNX5oGSwpa3I1ZwqQxZkbCWePN1w 9iithzGtNgyaaKsXFLyynQCbvX0oyLg= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=fSC1Gv8H; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=bYfs3lDT; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=fSC1Gv8H; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=bYfs3lDT; spf=pass (imf16.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720608676; a=rsa-sha256; cv=none; b=XXFHeD64/Cmz31TdRbTeCbkjaCv/nWVT8jujs1fB+9Qigval5U5+8I0595QKoPJ08cS7Td nuVSmaDpT7tdi1xJb7/0h7fuuA1+g6TxTBgr+c4bJdi/VWcD0Sqg0l19bQcyrpACVC9PvC Ic0WRz+U5CL2ndn+Cp42Tw8yWPQKPoY= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 5224F21BBB; Wed, 10 Jul 2024 10:51:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720608690; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ph9NUUUfzYdgAX+SWvx0C4X8fQI8fkpyj62wUbkyYlY=; b=fSC1Gv8HhwSvd1I/53WmRpPzjVucOsDas8A9DH6+m8UO0/3EcWcAfdBUqvLL3fVYEUdxfT EsPHGkqdoqpAUI+m/WEE1R03tBMUXoqLQQdRlXSPLZsfpwuMQc/78IfuRGhs4gwXuIQApN AHnPUdJV/mCCIE9Jew3Xrz/9bkaVxT0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720608690; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ph9NUUUfzYdgAX+SWvx0C4X8fQI8fkpyj62wUbkyYlY=; b=bYfs3lDTbUwBqvuiMOVaLuUilJTPz7MxAdnYxiVBZOgU7ZnrtqSMbRBzsTaiu4XCP4upCW gI+uI2I6ZXW1VZDw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1720608690; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ph9NUUUfzYdgAX+SWvx0C4X8fQI8fkpyj62wUbkyYlY=; b=fSC1Gv8HhwSvd1I/53WmRpPzjVucOsDas8A9DH6+m8UO0/3EcWcAfdBUqvLL3fVYEUdxfT EsPHGkqdoqpAUI+m/WEE1R03tBMUXoqLQQdRlXSPLZsfpwuMQc/78IfuRGhs4gwXuIQApN AHnPUdJV/mCCIE9Jew3Xrz/9bkaVxT0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1720608690; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ph9NUUUfzYdgAX+SWvx0C4X8fQI8fkpyj62wUbkyYlY=; b=bYfs3lDTbUwBqvuiMOVaLuUilJTPz7MxAdnYxiVBZOgU7ZnrtqSMbRBzsTaiu4XCP4upCW gI+uI2I6ZXW1VZDw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 30AA81369A; Wed, 10 Jul 2024 10:51:29 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id WPJEB7FnjmazcwAAD6G6ig (envelope-from ); Wed, 10 Jul 2024 10:51:29 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Muchun Song , David Hildenbrand , Michal Hocko , Donet Tom , "Kirill A . Shutemov" , Matthew Wilcox , Vlastimil Babka , Oscar Salvador Subject: [RFC PATCH 6/8] mm: Make hugetlb mappings go through mm_get_unmapped_area_vmflags Date: Wed, 10 Jul 2024 12:50:40 +0200 Message-ID: <20240710105042.30165-7-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240710105042.30165-1-osalvador@suse.de> References: <20240710105042.30165-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 9B92A18001B X-Stat-Signature: topuq3rwxhsz5bmqdtmjs4c46ygobpze X-HE-Tag: 1720608691-695205 X-HE-Meta: U2FsdGVkX1+SR/6t/POLT5C1yv+fuQ6vCHtC2HoYrIv52ZTQAm+MwugpaFMHNsQ4UfyB6PqAfusDjib8LT62U1UVJwgiFTXoEfpTRRQ1UpDYfRWH8wT5zW/whexXmbnl/GkXM/rzZZ39gp4RWVPDBbQ463X6QeGHGpRJOS4AVEEHdV3vBI6a1bZjdTD5mpfiVz783yPr4LcrBwhjimLhb6MgxKli2p0WGt8SrxX0LFE3BE4SOmvmeytdgpU2sH4Y16/bVCObIvP47wYXIHrG6vrVEDRR5rNm/9umBZwPkPo7Br4C5bGNJAm0g88KxayoYSt1anlhBWPdNSourKAJjgWkHadGCavf+PIydJlPGIaF7sGmibCdhzFtEWV1h6H0IYUhWzZvyZC2c7fRcmXnaC4zuOlX2+mcrZqI9b3oYjMr+FoR+Z2XLPq0JEXWwhtGDHvFN75u4TZi84vuIvdXBXw+5XMdA17N7f8Hax2VeLL1lcoZFVilIViSVNFY73jqLoiVEW2WplVCo258rqOLExMnKT/0tC8r5DPHbMwc9JPV7r8o4Nk39ZqvzQiM/4YAd2nsKicPqhwAMt/JLbcb+g6f0Qz6emWJ4XxDW/bOSYpS3a6WaLQRXTHbc/BXwGGYCARxTU6HXzyhjywaAiHGoLn8/gLPJtFSWORzsqr7qW5a9qEvZzz5P25srfI8t63+aXYT0Zuzpfgvf042RBPRa6d989+MY980YtG0HG3fjMHJuXAcUte7uqvQ02VTcHMGcCouQ+B3zMzCkS3UdfjKnK6MqnvcK/SyEJmopfN1kFohZL4tLxahcoNKtRKNxZY3FXrfd98A0rBqKBVwq4rTq+JEjFRg804nHo4tEXUm1iM4D00IQ/LIKtQW/NGQqiRNHaGorV64DKkDnmncKYE2K380r66TpP+Cvtf5GgOZ1+hHiuV9qDhwmp6d2We+ekhNrapWSjK6QVQvnKNjxix SZt3wHeq bxMcdMGaYsTANfgGU2frOh2q6UoEn7GKg76xPZjuMRC8sHVdvrLmk5DjW9rH6vranG+GOqYwN/3adjOsUQ6jNpFUpwUF8WP5qzPtHl3P1TtdcMozEKL0YitCBZgaWTq0f5hUsUFZaFwSZLYbCUBlH8/PpD2Q6cnHCiYTyM3GcJoTK9P2JTbGa9WM68sL8gnuw5g+zeBWBrJMYVcFK9uZa2tdcFCeYPSHKwsYoExd/q66SpuWZPnO9VpUjXVX4IrqFKi5XqadL22PinD9g8EEYHleG2Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hugetlb mappings will no longer be special cased but rather go through the generic mm_get_unmapped_area_vmflags function. For that to happen, let us remove the .get_unmapped_area from hugetlbfs_file_operations struct, and hint __get_unmapped_area that it should not send hugetlb mappings through thp_get_unmapped_area_vmflags but through mm_get_unmapped_area_vmflags. Create also a function called hugetlb_mmap_check_and_align() where a couple of safety checks are being done and the addr is aligned to the huge page size. Otherwise we will have to do this in every single function, which duplicates quite a lot of code. Signed-off-by: Oscar Salvador --- fs/hugetlbfs/inode.c | 22 ++++++++++++++-------- include/linux/hugetlb.h | 8 +++----- mm/mmap.c | 15 ++++++++++++++- 3 files changed, 31 insertions(+), 14 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 412f295acebe..b2d7fcecdb15 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -257,15 +257,22 @@ generic_hugetlb_get_unmapped_area(struct file *file, unsigned long addr, pgoff, flags); } -#ifndef HAVE_ARCH_HUGETLB_UNMAPPED_AREA -static unsigned long -hugetlb_get_unmapped_area(struct file *file, unsigned long addr, - unsigned long len, unsigned long pgoff, - unsigned long flags) +unsigned long +hugetlb_mmap_check_and_align(struct file *file, unsigned long addr, + unsigned long len, unsigned long flags) { - return generic_hugetlb_get_unmapped_area(file, addr, len, pgoff, flags); + unsigned long addr0 = 0; + struct hstate *h = hstate_file(file); + + if (len & ~huge_page_mask(h)) + return -EINVAL; + if ((flags & MAP_FIXED) && prepare_hugepage_range(file, addr, len)) + return -EINVAL; + if (addr) + addr0 = ALIGN(addr, huge_page_size(h)); + + return addr0; } -#endif /* * Someone wants to read @bytes from a HWPOISON hugetlb @page from @offset. @@ -1302,7 +1309,6 @@ static const struct file_operations hugetlbfs_file_operations = { .read_iter = hugetlbfs_read_iter, .mmap = hugetlbfs_file_mmap, .fsync = noop_fsync, - .get_unmapped_area = hugetlb_get_unmapped_area, .llseek = default_llseek, .fallocate = hugetlbfs_fallocate, .fop_flags = FOP_HUGE_PAGES, diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 1c7b0b32ff7e..9183ef95dfb6 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -555,11 +555,9 @@ static inline struct hstate *hstate_inode(struct inode *i) } #endif /* !CONFIG_HUGETLBFS */ -#ifdef HAVE_ARCH_HUGETLB_UNMAPPED_AREA -unsigned long hugetlb_get_unmapped_area(struct file *file, unsigned long addr, - unsigned long len, unsigned long pgoff, - unsigned long flags); -#endif /* HAVE_ARCH_HUGETLB_UNMAPPED_AREA */ +unsigned long +hugetlb_mmap_check_and_align(struct file *file, unsigned long addr, + unsigned long len, unsigned long flags); unsigned long generic_hugetlb_get_unmapped_area(struct file *file, unsigned long addr, diff --git a/mm/mmap.c b/mm/mmap.c index 09131b705e7b..8130b25b8cf5 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1860,6 +1860,7 @@ __get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, unsigned long, unsigned long, unsigned long) = NULL; + bool is_hugetlb = false; unsigned long error = arch_mmap_check(addr, len, flags); if (error) return error; @@ -1868,6 +1869,9 @@ __get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, if (len > TASK_SIZE) return -ENOMEM; + if (file && is_file_hugepages(file)) + is_hugetlb = true; + if (file) { if (file->f_op->get_unmapped_area) get_area = file->f_op->get_unmapped_area; @@ -1885,11 +1889,20 @@ __get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, if (get_area) { addr = get_area(file, addr, len, pgoff, flags); - } else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) { + } else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && !is_hugetlb) { /* Ensures that larger anonymous mappings are THP aligned. */ addr = thp_get_unmapped_area_vmflags(file, addr, len, pgoff, flags, vm_flags); } else { + /* + * Consolidate hugepages checks in one place, and also align addr + * to hugepage size. + */ + if (is_hugetlb) { + addr = hugetlb_mmap_check_and_align(file, addr, len, flags); + if (IS_ERR_VALUE(addr)) + return addr; + } addr = mm_get_unmapped_area_vmflags(current->mm, file, addr, len, pgoff, flags, vm_flags); }