From patchwork Fri Feb 15 22:08:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 10815995 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 51481139A for ; Fri, 15 Feb 2019 22:10:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4036630235 for ; Fri, 15 Feb 2019 22:10:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 349F03025E; Fri, 15 Feb 2019 22:10:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9320A30235 for ; Fri, 15 Feb 2019 22:10:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 848A18E001C; Fri, 15 Feb 2019 17:09:39 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7F7578E0014; Fri, 15 Feb 2019 17:09:39 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6A1318E001C; Fri, 15 Feb 2019 17:09:39 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by kanga.kvack.org (Postfix) with ESMTP id 34A048E0014 for ; Fri, 15 Feb 2019 17:09:39 -0500 (EST) Received: by mail-qk1-f197.google.com with SMTP id 207so9421048qkl.2 for ; Fri, 15 Feb 2019 14:09:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:dkim-signature:from:to:cc:subject :date:message-id:in-reply-to:references:reply-to:mime-version :content-transfer-encoding; bh=wEhGNSHqHzp3fZwWQfZiyRwXqVI8VIbN+Kjw76/hmYo=; b=IEpqWGzckibGioNm5xU23bsX+kxqPBZ5HFKUbR9RxJycOB2t7WPju8ng9T8uoYkJNi ADUZHVQ02OIwKY+rQ5JFVQpjPuXETMPlRk1OjkyCrHKik0aJaugUpdaOneRYR6moLCLt aijqz2NhUFmXrfIdrhIT8vlYFQKJNOZVlN3/D64tS2acaoEUK49PBBAMpGyO0Fyv8ZwX sahkM+pAtZJbPTPBhA5PUdo17C+Z1dcA/V/egJlBGogLuE27IAK6qIPHlWk366OJJ5HI 0EZ0ZFOBju6SyFyD3T/frBkZfCEc38AeLdWWKw6VRc9uHYQnRGrJC1R7Ep1+vzRquXm7 SRUw== X-Gm-Message-State: AHQUAua6/pJmHsul2rPRr5Cw9u+FZ06vgt7O9o3j/OQRBh4BSs3z/07K dZ6E9R2+yUCPPs5edhd/xT7IGCAqk0E3zgnTlE15G4wTnPUxd0+nLDB7isduJ5P5V4tiEAgEUzw dFhuDsDJEQCMsDCpUhXZ4ZRiLcj1vkf0tI27ZhZcCpGVpVlnoBKvQ+f5v+7bzrAfWng== X-Received: by 2002:a37:4c0f:: with SMTP id z15mr8811588qka.180.1550268578981; Fri, 15 Feb 2019 14:09:38 -0800 (PST) X-Google-Smtp-Source: AHgI3IbWh6EFsQF/MhUQuTlQN410Mr7lHLNrwZYCEqH5h2J8Djcln23ikkfHCixBH249ghvEAUf2 X-Received: by 2002:a37:4c0f:: with SMTP id z15mr8811552qka.180.1550268578392; Fri, 15 Feb 2019 14:09:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550268578; cv=none; d=google.com; s=arc-20160816; b=QMHdBKOCuv8Mu0Vrb09DxjbHrL3uJBlKyTMm7cuxWZMDE2aJPlLFdxEml+EALV2oJ2 HYmSPhjTn+JKzRqUi6orYl08v8BIy+Eq8TxOX+Sz+QW+aErq0uTEzi8cwSpT5L8V9RXD HEdHySlLD9mzytIq9tKdamlp4l8j+tqovA0NRah9M6Sg/iCWUL6w503aAxOPgkv/WDuH W/fYA4+f558pPIWxmMwRvxKzkpblKdheoIbgt5l6yi5eMdNuXZzMiHddnQOhFjbhZgti t1jy4Ax0/75I9msay/VK/HqE/P874uR2GUznBsqcSQUyuOj7xrHtZwveDjhaNMpIIWyR tR/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-signature; bh=wEhGNSHqHzp3fZwWQfZiyRwXqVI8VIbN+Kjw76/hmYo=; b=Zpvkp/Hi/i8hMvXPTqz/3VjmE5rk4KNby3QgMD+OnlXdjSLtggOPdRGxMSoDPa1typ Gk6uP68Xihv+Z+jUeX/DzP/WMgzdvvb1lRezvLecuRJsiiiGtkd3f7Yzl+uzslrJMsUq PW96S3CgaOCw5jZmNGe90ShrFhG/cJZ4GgzWszQ3GGUk0gCAAKrpls1H2bLITKMnbOsi gg4xOaLu0tj2RLbfdmDX5NSscVoIXW0P3sz/U4mbji7jRGhvyxSgLwaka5OvGTmEPKyO yY0pIwLb/bwvJgXYH4OU6k7N5S1Urb5Na5ChnWOPT2wMQVQp0DrknCT5hQT/8nfB29S5 MAEg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sent.com header.s=fm2 header.b=D2kj6qCW; dkim=pass header.i=@messagingengine.com header.s=fm2 header.b=ffAj6nKR; spf=pass (google.com: domain of zi.yan@sent.com designates 64.147.123.25 as permitted sender) smtp.mailfrom=zi.yan@sent.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=sent.com Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com. [64.147.123.25]) by mx.google.com with ESMTPS id b203si1317806qka.144.2019.02.15.14.09.38 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 15 Feb 2019 14:09:38 -0800 (PST) Received-SPF: pass (google.com: domain of zi.yan@sent.com designates 64.147.123.25 as permitted sender) client-ip=64.147.123.25; Authentication-Results: mx.google.com; dkim=pass header.i=@sent.com header.s=fm2 header.b=D2kj6qCW; dkim=pass header.i=@messagingengine.com header.s=fm2 header.b=ffAj6nKR; spf=pass (google.com: domain of zi.yan@sent.com designates 64.147.123.25 as permitted sender) smtp.mailfrom=zi.yan@sent.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=sent.com Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id 98633310A; Fri, 15 Feb 2019 17:09:36 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Fri, 15 Feb 2019 17:09:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=from :to:cc:subject:date:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; s=fm2; bh=wEhGNSHqHzp3f ZwWQfZiyRwXqVI8VIbN+Kjw76/hmYo=; b=D2kj6qCWlFHmfotrf7Ke/LUEg7yzF ccqkfGEBNm0+EFFa2BDH/ySxrvGb/NiberlTcmtesFpTwjwSX2IUByvzblQC9tV3 YaxcXdZHTIXlk8X78XfJvT9oaqB3zhzIEFiwsPcNsu9mvDzBz2u4LCzXLPQ5eYRL EEBPGf2UpUtRVzYD0aZVrqq4dhb9xAqFo0VueTxjXsyaDxrji9/dqGPMb1b8H35s b5hEFGa0vo/18cMkaS612oUF0lIS8/VcYCZmi0aQKo1D4FBbSRANloT0DSTKJvkW s1dQYQho3lzL1LloYPswExDB/KOBIua9YUgMDj/xKAEZaP9XDxqfG+K3A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :in-reply-to:message-id:mime-version:references:reply-to:subject :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; bh=wEhGNSHqHzp3fZwWQfZiyRwXqVI8VIbN+Kjw76/hmYo=; b=ffAj6nKR isuG/XWvtnwhlqGNCVF/7dfvaD3uRkyHj3qBgmMFVcgbFJ4mjsVhgdnP3qaUSn4S jQpEbyHMq6z5+DfnMRdW7XoYeqtkecJaYBqduQnzb1H5lANGGuArF4zQI5ehh8Du MxVzLbv6VcPGC9ZNvyAi+k44ayAVaVbeqsKXoI4wOx1ntnVXWRmLsyCkFhY3ZWVm kFBzy9E3KyzsJWwAe4FrqV+RyqeqyaPDphpnbaopgyAURNg7y+VOvcajVk+TeWoF W+gtlXt5fTLFizyrJn9acCdALc41v/sbuk0LMn2VDWpj6w277EnLUZCk06BKjGIa p8EMmDiC/sMxag== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedtledruddtjedgudehkecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfhuthenuceurghilhhouhhtmecu fedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmnecujfgurhephffvufffkf fojghfrhgggfestdekredtredttdenucfhrhhomhepkghiucgjrghnuceoiihirdihrghn sehsvghnthdrtghomheqnecukfhppedvudeirddvvdekrdduuddvrddvvdenucfrrghrrg hmpehmrghilhhfrhhomhepiihirdihrghnsehsvghnthdrtghomhenucevlhhushhtvghr ufhiiigvpedvvd X-ME-Proxy: Received: from nvrsysarch5.nvidia.com (thunderhill.nvidia.com [216.228.112.22]) by mail.messagingengine.com (Postfix) with ESMTPA id AAF68E4597; Fri, 15 Feb 2019 17:09:34 -0500 (EST) From: Zi Yan To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Dave Hansen , Michal Hocko , "Kirill A . Shutemov" , Andrew Morton , Vlastimil Babka , Mel Gorman , John Hubbard , Mark Hairgrove , Nitin Gupta , David Nellans , Zi Yan Subject: [RFC PATCH 25/31] mm: thp: add a knob to enable/disable 1GB THPs. Date: Fri, 15 Feb 2019 14:08:50 -0800 Message-Id: <20190215220856.29749-26-zi.yan@sent.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190215220856.29749-1-zi.yan@sent.com> References: <20190215220856.29749-1-zi.yan@sent.com> Reply-To: ziy@nvidia.com MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Zi Yan It does not affect existing 1GB THPs. It is similar to the knob for 2MB THPs. Signed-off-by: Zi Yan --- include/linux/huge_mm.h | 14 ++++++++++++++ mm/huge_memory.c | 42 ++++++++++++++++++++++++++++++++++++++++- mm/memory.c | 2 +- 3 files changed, 56 insertions(+), 2 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index b1acada9ce8c..687c7d59df8b 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -84,6 +84,8 @@ enum transparent_hugepage_flag { #ifdef CONFIG_DEBUG_VM TRANSPARENT_HUGEPAGE_DEBUG_COW_FLAG, #endif + TRANSPARENT_PUD_HUGEPAGE_FLAG, + TRANSPARENT_PUD_HUGEPAGE_REQ_MADV_FLAG, }; struct kobject; @@ -146,6 +148,18 @@ static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) } bool transparent_hugepage_enabled(struct vm_area_struct *vma); +static inline bool transparent_pud_hugepage_enabled(struct vm_area_struct *vma) +{ + if (transparent_hugepage_enabled(vma)) { + if (transparent_hugepage_flags & (1 << TRANSPARENT_PUD_HUGEPAGE_FLAG)) + return true; + if (transparent_hugepage_flags & + (1 << TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG)) + return !!(vma->vm_flags & VM_HUGEPAGE); + } + + return false; +} #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 191261771452..fa3e12b17621 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -50,9 +50,11 @@ unsigned long transparent_hugepage_flags __read_mostly = #ifdef CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS (1<ptl); alloc: - if (transparent_hugepage_enabled(vma) && + if (transparent_pud_hugepage_enabled(vma) && !transparent_hugepage_debug_cow()) { huge_gfp = alloc_hugepage_direct_gfpmask(vma); new_page = alloc_hugepage_vma(huge_gfp, vma, haddr, HPAGE_PUD_ORDER); diff --git a/mm/memory.c b/mm/memory.c index c875cc1a2600..5b8ad19cc439 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3859,7 +3859,7 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, vmf.pud = pud_alloc(mm, p4d, address); if (!vmf.pud) return VM_FAULT_OOM; - if (pud_none(*vmf.pud) && __transparent_hugepage_enabled(vma)) { + if (pud_none(*vmf.pud) && transparent_pud_hugepage_enabled(vma)) { ret = create_huge_pud(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret;