From patchwork Tue Feb 2 11:05:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 12061383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98F5CC433DB for ; Tue, 2 Feb 2021 11:06:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 480AF64DAF for ; Tue, 2 Feb 2021 11:06:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 480AF64DAF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 203AE6B0075; Tue, 2 Feb 2021 06:06:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 13FFF6B0083; Tue, 2 Feb 2021 06:06:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0038F6B0085; Tue, 2 Feb 2021 06:06:47 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0157.hostedemail.com [216.40.44.157]) by kanga.kvack.org (Postfix) with ESMTP id DD2D46B0075 for ; Tue, 2 Feb 2021 06:06:47 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id AB48A180AD802 for ; Tue, 2 Feb 2021 11:06:47 +0000 (UTC) X-FDA: 77773050054.08.rain57_370037e275ca Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin08.hostedemail.com (Postfix) with ESMTP id 8A2561819E773 for ; Tue, 2 Feb 2021 11:06:47 +0000 (UTC) X-HE-Tag: rain57_370037e275ca X-Filterd-Recvd-Size: 5997 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Tue, 2 Feb 2021 11:06:47 +0000 (UTC) Received: by mail-pl1-f173.google.com with SMTP id a16so640679plh.8 for ; Tue, 02 Feb 2021 03:06:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FaeqGDPDVUhZKacvJWFVXk0fi+RckdTnjepUSOw+GFs=; b=GjDhNjlo8WM1mrgLHvbOwrxZc7nRXMU/VBM1Tv3rS4CBALQMxNJhejkdeZWxgsByVQ r4yA/LHh0e7VOATygXnQKPpfmunhm+FbzuphRBa8mrDo2u6pt5O43a/qtl6zUGt1RfPF VK4vr39Wvi728wAAs7nvRQZo1BcofDZiCLayrH71cK1QQh1IXyidpqT/jXEB+/E2kNyA aTI6AcJujut2yUfbv8gauXhu6ABhOF/47kNxf9e8vgmxLCnhWJS7+AwxsGyKRCLVB346 /71rQxibTZobs+WTxOlJ4F6F8kdQT0qcnnrrhRg+CI8PNVsnkS9xCwNpFdBNipMMn2Wc JKeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FaeqGDPDVUhZKacvJWFVXk0fi+RckdTnjepUSOw+GFs=; b=ZuKwtQ6+/16TIaNth9TQrYr2IJC3MhjZpM2cQxOkAQr/nH+huOdGLv9/CbEYKKv+ZM +RHOYXR6EX5aBQ33RXcua57rBqfC1f0BokxJVdklGWyVft8ZJGIeidkjWX7SdzV5LuSk tnI3mShRi+vne5BHxy/wA9P2gw4xwbwPiRez36PuxyWIZmtJ7WXVs10Pho8itX9bvVWF ZZyk6wPB0YblyzI5jm71LHo9kvS+wTK+T9DCLs+f+o6txQ1rJkhobiuGb4qkcjLeA9ZK wMpDrPYVCjzVPR6WZ7sVS76DR8oE/TUS/ouz4o9uTD5ecueLv8xMVtDJP/YpUEyMARuZ 2dLg== X-Gm-Message-State: AOAM531VlE3bjPgYwqIeTLhCvdQCTvhacD37QHUOLQJu9fLh6RHhUbTw pdwk6Whlr8/NNBLikkjGKYe3b/MA0gA= X-Google-Smtp-Source: ABdhPJynX/oPNKQRM+SLHmtxwuqqGcjp2rkz5lI2AnOP58cVv+dR7CCN1OyW9GXsEsXailS2IkyO1g== X-Received: by 2002:a17:90b:17ca:: with SMTP id me10mr936389pjb.60.1612264006127; Tue, 02 Feb 2021 03:06:46 -0800 (PST) Received: from bobo.ozlabs.ibm.com (60-242-11-44.static.tpgi.com.au. [60.242.11.44]) by smtp.gmail.com with ESMTPSA id g19sm3188979pfk.113.2021.02.02.03.06.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Feb 2021 03:06:45 -0800 (PST) From: Nicholas Piggin To: linux-mm@kvack.org, Andrew Morton Cc: Nicholas Piggin , linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, Jonathan Cameron , Christoph Hellwig , Christophe Leroy , Rick Edgecombe , Ding Tianhong , Michael Ellerman Subject: [PATCH v12 14/14] powerpc/64s/radix: Enable huge vmalloc mappings Date: Tue, 2 Feb 2021 21:05:15 +1000 Message-Id: <20210202110515.3575274-15-npiggin@gmail.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20210202110515.3575274-1-npiggin@gmail.com> References: <20210202110515.3575274-1-npiggin@gmail.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This reduces TLB misses by nearly 30x on a `git diff` workload on a 2-node POWER9 (59,800 -> 2,100) and reduces CPU cycles by 0.54%, due to vfs hashes being allocated with 2MB pages. Cc: linuxppc-dev@lists.ozlabs.org Acked-by: Michael Ellerman Signed-off-by: Nicholas Piggin Reported-by: kernel test robot --- .../admin-guide/kernel-parameters.txt | 2 ++ arch/powerpc/Kconfig | 1 + arch/powerpc/kernel/module.c | 21 +++++++++++++++---- 3 files changed, 20 insertions(+), 4 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index a10b545c2070..d62df53e5200 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3225,6 +3225,8 @@ nohugeiomap [KNL,X86,PPC,ARM64] Disable kernel huge I/O mappings. + nohugevmalloc [PPC] Disable kernel huge vmalloc mappings. + nosmt [KNL,S390] Disable symmetric multithreading (SMT). Equivalent to smt=1. diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 107bb4319e0e..781da6829ab7 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -181,6 +181,7 @@ config PPC select GENERIC_GETTIMEOFDAY select HAVE_ARCH_AUDITSYSCALL select HAVE_ARCH_HUGE_VMAP if PPC_BOOK3S_64 && PPC_RADIX_MMU + select HAVE_ARCH_HUGE_VMALLOC if HAVE_ARCH_HUGE_VMAP select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_KASAN if PPC32 && PPC_PAGE_SHIFT <= 14 select HAVE_ARCH_KASAN_VMALLOC if PPC32 && PPC_PAGE_SHIFT <= 14 diff --git a/arch/powerpc/kernel/module.c b/arch/powerpc/kernel/module.c index a211b0253cdb..07026335d24d 100644 --- a/arch/powerpc/kernel/module.c +++ b/arch/powerpc/kernel/module.c @@ -87,13 +87,26 @@ int module_finalize(const Elf_Ehdr *hdr, return 0; } -#ifdef MODULES_VADDR void *module_alloc(unsigned long size) { + unsigned long start = VMALLOC_START; + unsigned long end = VMALLOC_END; + +#ifdef MODULES_VADDR BUILD_BUG_ON(TASK_SIZE > MODULES_VADDR); + start = MODULES_VADDR; + end = MODULES_END; +#endif + + /* + * Don't do huge page allocations for modules yet until more testing + * is done. STRICT_MODULE_RWX may require extra work to support this + * too. + */ - return __vmalloc_node_range(size, 1, MODULES_VADDR, MODULES_END, GFP_KERNEL, - PAGE_KERNEL_EXEC, VM_FLUSH_RESET_PERMS, NUMA_NO_NODE, + return __vmalloc_node_range(size, 1, start, end, GFP_KERNEL, + PAGE_KERNEL_EXEC, + VM_NO_HUGE_VMAP | VM_FLUSH_RESET_PERMS, + NUMA_NO_NODE, __builtin_return_address(0)); } -#endif