From patchwork Mon Jul 29 09:10:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13744630 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F24EC3DA61 for ; Mon, 29 Jul 2024 09:10:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 75E1B6B0095; Mon, 29 Jul 2024 05:10:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6BC166B0096; Mon, 29 Jul 2024 05:10:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 55DB26B0098; Mon, 29 Jul 2024 05:10:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3298F6B0095 for ; Mon, 29 Jul 2024 05:10:45 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id DFF6F140200 for ; Mon, 29 Jul 2024 09:10:44 +0000 (UTC) X-FDA: 82392220008.25.A7D24B3 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf20.hostedemail.com (Postfix) with ESMTP id AD8381C000A for ; Mon, 29 Jul 2024 09:10:42 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=NBotca+t; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=14BhT2RB; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=NBotca+t; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=14BhT2RB; spf=pass (imf20.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722244189; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0D03v0CNtg8d+Z2YRTExLn0FsjKOf1vcUEKVYVI3qZA=; b=NUcP+Rih3S+yQNNcpxzXeJ5gQGBtjp7eSRr35w+kdZUhi+4ntSeIf+jD8gda3m3Jbx/b7S ct+H3fc9fuov4nCgVKFapOUTc8bxKcJJicwgD/bQMCxuWaa7qxdg4JhkiGrig9BiuYOpCx JM7c3r+QkCEm1+H7cU5Q3cG6GV1TgVA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722244189; a=rsa-sha256; cv=none; b=NzRIHZhUjr1sPTCVWAqL+FtkxqlkjoKfq71nm99kG0foZDCDY6UsHvX0+0+iJASS8a+Boq ZYyfurcWBxY147Ms2tXy+ggpHgXPA1I88cnj4kK74B9Mu21OFHmbF3B/E9hOk5qYZUzS/e eHyNR+B8siCrsW1k9YtuZ8ymANuX8hU= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=NBotca+t; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=14BhT2RB; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=NBotca+t; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=14BhT2RB; spf=pass (imf20.hostedemail.com: domain of osalvador@suse.de designates 195.135.223.130 as permitted sender) smtp.mailfrom=osalvador@suse.de; dmarc=pass (policy=none) header.from=suse.de Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 6D81A21BC1; Mon, 29 Jul 2024 09:10:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1722244241; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0D03v0CNtg8d+Z2YRTExLn0FsjKOf1vcUEKVYVI3qZA=; b=NBotca+t+NBVJkyG1KJ2BeT9oBB/GpLOboqMQvQRyRmYQLl3N6NHlzOVlDWDH1sORtTOhA 1zUXHQqSgna1nRsWzLZzaXfnvWLAeI5drBIMYRa/l6YP5SLGieif5frv4j3qGDpo5bK2v7 dvqvFn4MiD+m6D0JmS4htP6SQZtPagI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1722244241; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0D03v0CNtg8d+Z2YRTExLn0FsjKOf1vcUEKVYVI3qZA=; b=14BhT2RBgWuQS9/HaH/h2wDy/QpKlbdFsF/BybzmpSt4rFaVYQ3UWlxckbnxKScwK1hSUj ZImZpXgjy2e1LlDw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1722244241; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0D03v0CNtg8d+Z2YRTExLn0FsjKOf1vcUEKVYVI3qZA=; b=NBotca+t+NBVJkyG1KJ2BeT9oBB/GpLOboqMQvQRyRmYQLl3N6NHlzOVlDWDH1sORtTOhA 1zUXHQqSgna1nRsWzLZzaXfnvWLAeI5drBIMYRa/l6YP5SLGieif5frv4j3qGDpo5bK2v7 dvqvFn4MiD+m6D0JmS4htP6SQZtPagI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1722244241; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0D03v0CNtg8d+Z2YRTExLn0FsjKOf1vcUEKVYVI3qZA=; b=14BhT2RBgWuQS9/HaH/h2wDy/QpKlbdFsF/BybzmpSt4rFaVYQ3UWlxckbnxKScwK1hSUj ZImZpXgjy2e1LlDw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 125F91368A; Mon, 29 Jul 2024 09:10:39 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id IAAiO49cp2Z4BgAAD6G6ig (envelope-from ); Mon, 29 Jul 2024 09:10:39 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Peter Xu , Muchun Song , David Hildenbrand , Donet Tom , Matthew Wilcox , Vlastimil Babka , Michal Hocko , Oscar Salvador Subject: [PATCH v2 6/9] mm: Make hugetlb mappings go through mm_get_unmapped_area_vmflags Date: Mon, 29 Jul 2024 11:10:15 +0200 Message-ID: <20240729091018.2152-7-osalvador@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240729091018.2152-1-osalvador@suse.de> References: <20240729091018.2152-1-osalvador@suse.de> MIME-Version: 1.0 X-Stat-Signature: k7ak77knrzixs8xs1e99kyjzyor8m6bb X-Rspamd-Queue-Id: AD8381C000A X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1722244242-630769 X-HE-Meta: U2FsdGVkX1+yXQfxQy60GNdBrBZPVaCByrDayjuS0u6lIzu2FngmuKMJYBO6leQLbnt5UG4j0ZBKtbhICPrDBYxfqikDqhCPtElxcJT06zSiS66jEX+NI1aGwZ5wGgkGbSiDlopZ7U45fTQD5/ks8KeRlwEf96te3z4MEqu3WStHP1od/bHeLkq8d0kxHPK9LE+sXVRE1iicxcgoYU13IIYcuu83BCUPQYY8Kk/4wB+SaY8lUGmsMVn1qRmKH1HVgDL0I21es96YHQRvdUA9SHqDm9q5d5kQk0Jej4gb/TwodJ92anEyu6CkNN29iOksycm0MVjlBVZK+YfogiVCvTon9mJj1IrhH+1y+Itpow4ToIrjlMKO5DMrNr7Oxef4H1PKwhaQbINrAg2prME/QOUPTtIbkhCjhP0OKP93PtFf6wHdvNiyvjStMiNHwoDIRsQREtLFBT+v+uqRI2iaO6lLG02F5CXkjc/5AQh2Xn24yA+s6XKspDsDdQGxdspNtMQkdJDlJfi6k9a05jGxxjuMemXpxE8CI6lgsGlzQ+nyAiHd5XeS2P6nyB5R4yT3nDnrdQ+N93LbuL6K8S/Rb9RFaBYV71fCm0D0PlI5J4UEaECjLSdTB6+rdKQfS7hR13kQ1WALJLb8dM79K7uJ9k4GEQ5qs60/y+7Ks5VWuHkwkWmjdC8njV+eRvygiU/83ngNpZQh0kxAGEN3bPrySsRM5R5tnXzxhFKZtcfB8irkdn9/yL5sYIUW35r2h9XVYL/ZqgtoVhyap+CyI3BVACXClVpxkKUNqtzKj4i2vlKCmNZWARs33WjNFk9OMSSkx2BvACFCfQhw5VtHzFdjTgZZWmvQq94nzkfcwIsVBmCBKCTzQz2IJjlbrJRl2pDGSAaUOX8Ol2clGXT/MXvpg6vouEujfcWlFJQ3VY2pM5eHLkiJudmjT5A0CZ0WBCXgABO+4pUk3ZgL10RTywB P3R88YFB 4QEIA1GljAFvVE/CO+Am3xSWtKtU9F9AEnvuAF/QAMS7rnCuvzAzcs4CV3p4Wjaw/YWy+J7wf5bZWrj8Y/tz5gcYV1KbX0fD80Dm8PPJJQyV3sI14XOfpsPOfnA2XumnyKtD6//rhlWtnGXIvMDSXnhxQrm5HjjmVQskcBIT2AKp54yvQ2Rf1/JPcilf4hLtNnJdtW1ECPdkcUHjcRjrJVbnmDXpLy+Ka0AEoVmDnkVuIDo5MLgGcBD64gQS3phRojTqOqLZHvEZ0J4oCHFhAjDOKaSkUJEnAbd3b X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hugetlb mappings will no longer be special cased but rather go through the generic mm_get_unmapped_area_vmflags function. For that to happen, let us remove the .get_unmapped_area from hugetlbfs_file_operations struct, and hint __get_unmapped_area that it should not send hugetlb mappings through thp_get_unmapped_area_vmflags but through mm_get_unmapped_area_vmflags. Create also a function called hugetlb_mmap_check_and_align() where a couple of safety checks are being done and the addr is aligned to the huge page size. Otherwise we will have to do this in every single function, which duplicates quite a lot of code. Signed-off-by: Oscar Salvador --- fs/hugetlbfs/inode.c | 22 ++++++++++++++-------- include/linux/hugetlb.h | 8 +++----- mm/mmap.c | 15 ++++++++++++++- 3 files changed, 31 insertions(+), 14 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 9f6cff356796..5d47a2785a5d 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -258,15 +258,22 @@ generic_hugetlb_get_unmapped_area(struct file *file, unsigned long addr, pgoff, flags); } -#ifndef HAVE_ARCH_HUGETLB_UNMAPPED_AREA -static unsigned long -hugetlb_get_unmapped_area(struct file *file, unsigned long addr, - unsigned long len, unsigned long pgoff, - unsigned long flags) +unsigned long +hugetlb_mmap_check_and_align(struct file *file, unsigned long addr, + unsigned long len, unsigned long flags) { - return generic_hugetlb_get_unmapped_area(file, addr, len, pgoff, flags); + unsigned long addr0 = 0; + struct hstate *h = hstate_file(file); + + if (len & ~huge_page_mask(h)) + return -EINVAL; + if ((flags & MAP_FIXED) && prepare_hugepage_range(file, addr, len)) + return -EINVAL; + if (addr) + addr0 = ALIGN(addr, huge_page_size(h)); + + return addr0; } -#endif /* * Someone wants to read @bytes from a HWPOISON hugetlb @page from @offset. @@ -1300,7 +1307,6 @@ static const struct file_operations hugetlbfs_file_operations = { .read_iter = hugetlbfs_read_iter, .mmap = hugetlbfs_file_mmap, .fsync = noop_fsync, - .get_unmapped_area = hugetlb_get_unmapped_area, .llseek = default_llseek, .fallocate = hugetlbfs_fallocate, .fop_flags = FOP_HUGE_PAGES, diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 0ec14e5e0890..1413cdcfdb1a 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -549,11 +549,9 @@ static inline struct hstate *hstate_inode(struct inode *i) } #endif /* !CONFIG_HUGETLBFS */ -#ifdef HAVE_ARCH_HUGETLB_UNMAPPED_AREA -unsigned long hugetlb_get_unmapped_area(struct file *file, unsigned long addr, - unsigned long len, unsigned long pgoff, - unsigned long flags); -#endif /* HAVE_ARCH_HUGETLB_UNMAPPED_AREA */ +unsigned long +hugetlb_mmap_check_and_align(struct file *file, unsigned long addr, + unsigned long len, unsigned long flags); unsigned long generic_hugetlb_get_unmapped_area(struct file *file, unsigned long addr, diff --git a/mm/mmap.c b/mm/mmap.c index 7b623811d82a..f755d8a298c5 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -849,6 +849,7 @@ __get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, unsigned long, unsigned long, unsigned long) = NULL; + bool is_hugetlb = false; unsigned long error = arch_mmap_check(addr, len, flags); if (error) return error; @@ -857,6 +858,9 @@ __get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, if (len > TASK_SIZE) return -ENOMEM; + if (file && is_file_hugepages(file)) + is_hugetlb = true; + if (file) { if (file->f_op->get_unmapped_area) get_area = file->f_op->get_unmapped_area; @@ -874,11 +878,20 @@ __get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, if (get_area) { addr = get_area(file, addr, len, pgoff, flags); - } else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) { + } else if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && !is_hugetlb) { /* Ensures that larger anonymous mappings are THP aligned. */ addr = thp_get_unmapped_area_vmflags(file, addr, len, pgoff, flags, vm_flags); } else { + /* + * Consolidate hugepages checks in one place, and also align addr + * to hugepage size. + */ + if (is_hugetlb) { + addr = hugetlb_mmap_check_and_align(file, addr, len, flags); + if (IS_ERR_VALUE(addr)) + return addr; + } addr = mm_get_unmapped_area_vmflags(current->mm, file, addr, len, pgoff, flags, vm_flags); }