From patchwork Tue Nov 20 10:35:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Hocko X-Patchwork-Id: 10690205 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3AD9C15A7 for ; Tue, 20 Nov 2018 10:35:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2BFAB2906E for ; Tue, 20 Nov 2018 10:35:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 204A02A87C; Tue, 20 Nov 2018 10:35:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9A1402906E for ; Tue, 20 Nov 2018 10:35:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E1CB6B1FCC; Tue, 20 Nov 2018 05:35:33 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4681E6B1FCD; Tue, 20 Nov 2018 05:35:33 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 35B3D6B1FCE; Tue, 20 Nov 2018 05:35:33 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id CBB886B1FCC for ; Tue, 20 Nov 2018 05:35:32 -0500 (EST) Received: by mail-ed1-f71.google.com with SMTP id v4so1010256edm.18 for ; Tue, 20 Nov 2018 02:35:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=uxsJPkLcKckgPrN9D2d3v7GbIbgMcMizVEHoIL7/qsw=; b=MBQ6oFSFKiWYQTAQnNWw4wFehXhnKVn7bEuHBvA2KwnUDkEIhbhBQoOqftC81rt3X/ ki5BTRYoXFLohYCbIr76fBrUBbTh39PzQIM802UaY5DN8F+PetMIW/94K9vTGDLIEBOG pJq1XHA9RBg6gxLEdvlYKxqmEKBt4ohybBUj/lGGe4gex2eQT93O3kUOzz6S6szlp9TD fSAq+02l5bggkrz/B2MRYXfhrU69bjnaCOFHoDvZTKHivdbGFUuc2UmRP2n9pTjSc3qN y7XS9Y+xYGxrgsRFHuVan3G8PX/r40kqyugIT0n3D/faPg6w514Z65teu5dB+/TcKZ5V WZtQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Gm-Message-State: AGRZ1gKCDjJMloQzlW1/a9kfBmESA/fhCJmfZjhStyRk6DxlOFfUJZXK B4790MiINN6PinzbBc4aW8V0tyYcAG9IvBeN8BxsgyqDLEqU66omuUrIcO5zgLTmbkam4qyPvxu 4suqPDo7nswkXRH3C77+0tmTVOdJy9cqdaDzH9hoQqKWltKJ1VvbuwUZ4D0ABvzTtQtKqqT97k5 U4IIt2IssgP3IGIEWLlyNH161ujA0U6iDMfJj8ovPAafzNtjMXbk+AOF3HX4KpSNyQzKe/4Ad7B hfxWQtITAA0ZNkXQv+8mznmWOdilhj6DHr6rRSdJ4tJIDlBtIgKiRA+kYhc3zOKFZTP9aU3mAGM MP3tVUnwJIrHy/rhuTG9tFC9dVwaqoXbMw5YjVlAVmecOUevsjCejT993VBhjADRkfhqlrJw4Q= = X-Received: by 2002:a17:906:b243:: with SMTP id ce3-v6mr1540383ejb.87.1542710132285; Tue, 20 Nov 2018 02:35:32 -0800 (PST) X-Received: by 2002:a17:906:b243:: with SMTP id ce3-v6mr1540346ejb.87.1542710131222; Tue, 20 Nov 2018 02:35:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542710131; cv=none; d=google.com; s=arc-20160816; b=M3btcxA3GOWtd4FV/Ns94oZ0jTFMCbBkAApAz4Lv9iaUyq5uWA/EA954TnSlXN1ZCO lgyAF08w6Rd+6daz/EbsFISvqtCl0a4+j8j7TlqiT15vZNb4Y+nJco07rVyon+fUaqDo YMPN0LfhztNdSKNFm6+CnRlcH/E49rGQCP2StuUf9QXPi/NfjnX7jIEKz7SMJAKdFocE Do2mxzNt3mC2/BS8jEC4Zk+zL8WQ0pLnJnmPYbdkeb4XA7O7sUWDX0OnGTLev3W+dlil uXDz73Qywyk8DaIdxddiyzjnaK15Su4n9ZzW5VJL4bK+6EYQ+lu03V2Zq/vvv+Wrtmdi 7M6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=uxsJPkLcKckgPrN9D2d3v7GbIbgMcMizVEHoIL7/qsw=; b=XLkUeZLVTeoGC47D+ZfI2e8zyfKon3J0iFyBkJfxsx1zo78LxZs5Kp/3+w+1lT4q74 PiI2NrMB8/46Aw+laLoI0gfej/NP2jLivW6yCzAC/8PmFipovk1pk6rIP1inQxmtZaH+ K9jYaIRFCrps0DhZogAkxBInBncEXXg8ELToVdk+0hAtDdCTYa6jLeGY9cf3oVXMWwW6 rg98C7enRyDRlKRp/aUTuJnZ7/7ukkVXJwenQ6yv4/A4RjpAcoKdAAq+JzgTqXa0kqeE CjdaN26VHEDZo/V3/Qsm0iF6XrgFWVutncgduyrH2rfEdHGkKuTc2+hOE0oZ0yWxn+GF wg2A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id h34sor12026592edb.28.2018.11.20.02.35.31 for (Google Transport Security); Tue, 20 Nov 2018 02:35:31 -0800 (PST) Received-SPF: pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Google-Smtp-Source: AFSGD/XcnnedeezeTZmmFpCMjh4dc1ajw+LXBhUFZ7D3LPQ8LJxLKr4zWSjKw58X7+vbshNyIL2LJA== X-Received: by 2002:a50:8799:: with SMTP id a25mr1831824eda.96.1542710130747; Tue, 20 Nov 2018 02:35:30 -0800 (PST) Received: from tiehlicka.suse.cz (prg-ext-pat.suse.com. [213.151.95.130]) by smtp.gmail.com with ESMTPSA id a15-v6sm5967233ejj.5.2018.11.20.02.35.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 02:35:29 -0800 (PST) From: Michal Hocko To: linux-api@vger.kernel.org Cc: Andrew Morton , Alexey Dobriyan , , LKML , Michal Hocko , Jan Kara , Dan Williams , David Rientjes Subject: [RFC PATCH 1/3] mm, proc: be more verbose about unstable VMA flags in /proc//smaps Date: Tue, 20 Nov 2018 11:35:13 +0100 Message-Id: <20181120103515.25280-2-mhocko@kernel.org> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181120103515.25280-1-mhocko@kernel.org> References: <20181120103515.25280-1-mhocko@kernel.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Michal Hocko Even though vma flags exported via /proc//smaps are explicitly documented to be not guaranteed for future compatibility the warning doesn't go far enough because it doesn't mention semantic changes to those flags. And they are important as well because these flags are a deep implementation internal to the MM code and the semantic might change at any time. Let's consider two recent examples: http://lkml.kernel.org/r/20181002100531.GC4135@quack2.suse.cz : commit e1fb4a086495 "dax: remove VM_MIXEDMAP for fsdax and device dax" has : removed VM_MIXEDMAP flag from DAX VMAs. Now our testing shows that in the : mean time certain customer of ours started poking into /proc//smaps : and looks at VMA flags there and if VM_MIXEDMAP is missing among the VMA : flags, the application just fails to start complaining that DAX support is : missing in the kernel. http://lkml.kernel.org/r/alpine.DEB.2.21.1809241054050.224429@chino.kir.corp.google.com : Commit 1860033237d4 ("mm: make PR_SET_THP_DISABLE immediately active") : introduced a regression in that userspace cannot always determine the set : of vmas where thp is ineligible. : Userspace relies on the "nh" flag being emitted as part of /proc/pid/smaps : to determine if a vma is eligible to be backed by hugepages. : Previous to this commit, prctl(PR_SET_THP_DISABLE, 1) would cause thp to : be disabled and emit "nh" as a flag for the corresponding vmas as part of : /proc/pid/smaps. After the commit, thp is disabled by means of an mm : flag and "nh" is not emitted. : This causes smaps parsing libraries to assume a vma is eligible for thp : and ends up puzzling the user on why its memory is not backed by thp. In both cases userspace was relying on a semantic of a specific VMA flag. The primary reason why that happened is a lack of a proper internface. While this has been worked on and it will be fixed properly, it seems that our wording could see some refinement and be more vocal about semantic aspect of these flags as well. Cc: Jan Kara Cc: Dan Williams Cc: David Rientjes Signed-off-by: Michal Hocko Acked-by: Jan Kara Acked-by: Dan Williams Acked-by: David Rientjes Acked-by: Mike Rapoport Acked-by: Vlastimil Babka --- Documentation/filesystems/proc.txt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt index 12a5e6e693b6..b1fda309f067 100644 --- a/Documentation/filesystems/proc.txt +++ b/Documentation/filesystems/proc.txt @@ -496,7 +496,9 @@ flags associated with the particular virtual memory area in two letter encoded Note that there is no guarantee that every flag and associated mnemonic will be present in all further kernel releases. Things get changed, the flags may -be vanished or the reverse -- new added. +be vanished or the reverse -- new added. Interpretatation of their meaning +might change in future as well. So each consumnent of these flags have to +follow each specific kernel version for the exact semantic. This file is only present if the CONFIG_MMU kernel configuration option is enabled. From patchwork Tue Nov 20 10:35:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Hocko X-Patchwork-Id: 10690207 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E88F15A7 for ; Tue, 20 Nov 2018 10:35:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 111B52A87C for ; Tue, 20 Nov 2018 10:35:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 058402A881; Tue, 20 Nov 2018 10:35:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 543FF2A87C for ; Tue, 20 Nov 2018 10:35:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5CB1B6B1FCD; Tue, 20 Nov 2018 05:35:34 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 57AD56B1FCE; Tue, 20 Nov 2018 05:35:34 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41C7C6B1FCF; Tue, 20 Nov 2018 05:35:34 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id DBE676B1FCD for ; Tue, 20 Nov 2018 05:35:33 -0500 (EST) Received: by mail-ed1-f71.google.com with SMTP id w15so1041231edl.21 for ; Tue, 20 Nov 2018 02:35:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=QWerdv5NokFZCG7auzm1OEN/qgY42YRYuNB88bkmlcY=; b=uI9K8p/iOpqcLU6Cq9ZDheUDQcHLTmxaqUoFot0JG65TqKnqtimK+UKUel3OuGL6JH g8ta3PRYpRflD7eYJUhjYOrSKFxqjVF579pQYwCF6fa5cpHdd3Q+qeocCkoknuec/JND i5cCjg2SzE6D/ka2aqUM2vLwP/PnznEw7ErAZWDgqulaDd3YFfzJHooeWMdeQtXouo5o LKN08YpvfCeGKREE341zUAAjFl0lCDbUabHe1NuVF0/5VCNoHh25k5NkvPj3+TrIacQp h3PDHM6Phn9yNdXwMtJd6NERFaKiAK13rB/qWIl6xtz6nZPvVw+5vWim17S0gLnFSu9+ VB5A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Gm-Message-State: AA+aEWa+NELIL8ewKH7SgXV+kWcWA3Z8Ema37SzX8OEtzT3VCm6zoM0m dNmYP1hA6npHp8u2Yv9eJGmCqgv8KIPsmu67lvr+91nX3N5H+YQlEDQ+jPkhiNh4vmzpPfXyd0F MyaiD/qxRMeMfZD5SvoX7XiMbfVi2F5fR6RmriupHiK+Qaqf4sjlp5kWnqcADq1IBxaBlywFOtA 6CHjoURmAQg9gSgav9RrMPY3tDrK4C5A0LMmIyxBjJOe/Eb68F+4IL0/vOEdlhk3xR3w/H+FXyR UHSzY46HqJ60HJShx8z3Gyr2d6Im1KpTAxPisqVbPA21Je496V5Gx+UVH5f+DH/MH5DzP4eko31 u081jHxfNPlbbtH65d8mMth8XOSFPo756J/qnsOFgz1Q4T2Z+KFiDNYr70vlo2VPq1k6FJNFkA= = X-Received: by 2002:a50:f5af:: with SMTP id u44mr1870385edm.172.1542710133348; Tue, 20 Nov 2018 02:35:33 -0800 (PST) X-Received: by 2002:a50:f5af:: with SMTP id u44mr1870334edm.172.1542710132298; Tue, 20 Nov 2018 02:35:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542710132; cv=none; d=google.com; s=arc-20160816; b=eWKWRaOLZ2xzhtpKrv1O0CIpT9ghPuLQLf1Q69z0HKNtC6mNHTKI9RNsr84VSBZIeI 3+eytSpV5mYAtdlIdETs1RKc+DVo78aeREfT+4isXLJy2Ag+26wWAgC3ZUVFzCnLmFQu XhJohId10cr0LG8nXtWTEsy5CShX0B2tgj/BFfrRr0NvT9bWS72ZF8+SpBEUAIMPWpjp K58AJ/xX8jUonmlIDMxhwF9HD9zPl8nRuDLWed1osaMZdiE8qlv1EQOEMTs3Ny47d42e IQLYjK1KGItqHtJCt7xoz63Ko6NOmMhjsQIuQheldQre8KdNrlIXwWK2d/nUxE5lmunj PY/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=QWerdv5NokFZCG7auzm1OEN/qgY42YRYuNB88bkmlcY=; b=iBFtzekS7Lys6OWz9288kXUYefSm4Iu5/kjJLk/ZINJymx811TEBZFoZ9zc0sMfLGt +UZXcBNjClmMcTOWtZ7H0P4e1FL8loKh6utxJ09LYkt3C+uXqSAyl4Y2EUj58MG6qTEk kLfkt/etq1QzvzEXZ6e35newHirhNdsgBBBQdjLVND/7FDF/SoQnnZGX4lUHz20g7zuz maZ4MxNEnbCiy+cS6HxW0QY3AIPeNYUcNi0S/aBrLfiLC2d6tU4lEZpVNjddc3U453Ji RvXDd0Gn/xzjryb8z46CjXyCVYxtYszMdBzey1slDLi5FnCbI1NreWX1Z4oV7v862sVf Kr+A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id f47sor8782037edb.4.2018.11.20.02.35.32 for (Google Transport Security); Tue, 20 Nov 2018 02:35:32 -0800 (PST) Received-SPF: pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Google-Smtp-Source: AFSGD/U8yv77zjPMMDkFdhuOBD/tAZDxskV5Ravx718ZFAdo8bVLSrpGjbCwriOcV5GVV4qvNOllvg== X-Received: by 2002:a50:e045:: with SMTP id g5mr1862061edl.152.1542710131793; Tue, 20 Nov 2018 02:35:31 -0800 (PST) Received: from tiehlicka.suse.cz (prg-ext-pat.suse.com. [213.151.95.130]) by smtp.gmail.com with ESMTPSA id a15-v6sm5967233ejj.5.2018.11.20.02.35.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 02:35:31 -0800 (PST) From: Michal Hocko To: linux-api@vger.kernel.org Cc: Andrew Morton , Alexey Dobriyan , , LKML , Michal Hocko Subject: [RFC PATCH 2/3] mm, thp, proc: report THP eligibility for each vma Date: Tue, 20 Nov 2018 11:35:14 +0100 Message-Id: <20181120103515.25280-3-mhocko@kernel.org> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181120103515.25280-1-mhocko@kernel.org> References: <20181120103515.25280-1-mhocko@kernel.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Michal Hocko Userspace falls short when trying to find out whether a specific memory range is eligible for THP. There are usecases that would like to know that http://lkml.kernel.org/r/alpine.DEB.2.21.1809251248450.50347@chino.kir.corp.google.com : This is used to identify heap mappings that should be able to fault thp : but do not, and they normally point to a low-on-memory or fragmentation : issue. The only way to deduce this now is to query for hg resp. nh flags and confronting the state with the global setting. Except that there is also PR_SET_THP_DISABLE that might change the picture. So the final logic is not trivial. Moreover the eligibility of the vma depends on the type of VMA as well. In the past we have supported only anononymous memory VMAs but things have changed and shmem based vmas are supported as well these days and the query logic gets even more complicated because the eligibility depends on the mount option and another global configuration knob. Simplify the current state and report the THP eligibility in /proc//smaps for each existing vma. Reuse transparent_hugepage_enabled for this purpose. The original implementation of this function assumes that the caller knows that the vma itself is supported for THP so make the core checks into __transparent_hugepage_enabled and use it for existing callers. __show_smap just use the new transparent_hugepage_enabled which also checks the vma support status (please note that this one has to be out of line due to include dependency issues). Signed-off-by: Michal Hocko Acked-by: Vlastimil Babka --- Documentation/filesystems/proc.txt | 3 +++ fs/proc/task_mmu.c | 2 ++ include/linux/huge_mm.h | 13 ++++++++++++- mm/huge_memory.c | 12 +++++++++++- mm/memory.c | 4 ++-- 5 files changed, 30 insertions(+), 4 deletions(-) diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt index b1fda309f067..06562bab509a 100644 --- a/Documentation/filesystems/proc.txt +++ b/Documentation/filesystems/proc.txt @@ -425,6 +425,7 @@ SwapPss: 0 kB KernelPageSize: 4 kB MMUPageSize: 4 kB Locked: 0 kB +THPeligible: 0 VmFlags: rd ex mr mw me dw the first of these lines shows the same information as is displayed for the @@ -462,6 +463,8 @@ replaced by copy-on-write) part of the underlying shmem object out on swap. "SwapPss" shows proportional swap share of this mapping. Unlike "Swap", this does not take into account swapped out page of underlying shmem objects. "Locked" indicates whether the mapping is locked in memory or not. +"THPeligible" indicates whether the mapping is eligible for THP pages - 1 if +true, 0 otherwise. "VmFlags" field deserves a separate description. This member represents the kernel flags associated with the particular virtual memory area in two letter encoded diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 47c3764c469b..c9f160eb9fbc 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -790,6 +790,8 @@ static int show_smap(struct seq_file *m, void *v) __show_smap(m, &mss); + seq_printf(m, "THPeligible: %d\n", transparent_hugepage_enabled(vma)); + if (arch_pkeys_enabled()) seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma)); show_smap_vma_flags(m, vma); diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 4663ee96cf59..381e872bfde0 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -93,7 +93,11 @@ extern bool is_vma_temporary_stack(struct vm_area_struct *vma); extern unsigned long transparent_hugepage_flags; -static inline bool transparent_hugepage_enabled(struct vm_area_struct *vma) +/* + * to be used on vmas which are known to support THP. + * Use transparent_hugepage_enabled otherwise + */ +static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) { if (vma->vm_flags & VM_NOHUGEPAGE) return false; @@ -117,6 +121,8 @@ static inline bool transparent_hugepage_enabled(struct vm_area_struct *vma) return false; } +bool transparent_hugepage_enabled(struct vm_area_struct *vma); + #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ (1<vm_file->f_mapping) && shmem_huge_enabled(vma)) + return __transparent_hugepage_enabled(vma); + + return false; +} + static struct page *get_huge_zero_page(void) { struct page *zero_page; @@ -1303,7 +1313,7 @@ vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd) get_page(page); spin_unlock(vmf->ptl); alloc: - if (transparent_hugepage_enabled(vma) && + if (__transparent_hugepage_enabled(vma) && !transparent_hugepage_debug_cow()) { huge_gfp = alloc_hugepage_direct_gfpmask(vma, haddr); new_page = alloc_pages_vma(huge_gfp, HPAGE_PMD_ORDER, vma, diff --git a/mm/memory.c b/mm/memory.c index 4ad2d293ddc2..3c2716ec7fbd 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3830,7 +3830,7 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, vmf.pud = pud_alloc(mm, p4d, address); if (!vmf.pud) return VM_FAULT_OOM; - if (pud_none(*vmf.pud) && transparent_hugepage_enabled(vma)) { + if (pud_none(*vmf.pud) && __transparent_hugepage_enabled(vma)) { ret = create_huge_pud(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; @@ -3856,7 +3856,7 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, vmf.pmd = pmd_alloc(mm, vmf.pud, address); if (!vmf.pmd) return VM_FAULT_OOM; - if (pmd_none(*vmf.pmd) && transparent_hugepage_enabled(vma)) { + if (pmd_none(*vmf.pmd) && __transparent_hugepage_enabled(vma)) { ret = create_huge_pmd(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; From patchwork Tue Nov 20 10:35:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Hocko X-Patchwork-Id: 10690209 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 534BD14BD for ; Tue, 20 Nov 2018 10:35:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4561F2906E for ; Tue, 20 Nov 2018 10:35:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 393912A87C; Tue, 20 Nov 2018 10:35:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 212372906E for ; Tue, 20 Nov 2018 10:35:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5AB256B1FCE; Tue, 20 Nov 2018 05:35:35 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4B8A96B1FCF; Tue, 20 Nov 2018 05:35:35 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 30FC06B1FD0; Tue, 20 Nov 2018 05:35:35 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id C74AE6B1FCE for ; Tue, 20 Nov 2018 05:35:34 -0500 (EST) Received: by mail-ed1-f72.google.com with SMTP id e17so1051916edr.7 for ; Tue, 20 Nov 2018 02:35:34 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=UipYMHSvtb8aOnY17kKqlQt8aWhzlLzqt0JzwQhgn3Q=; b=uLKGdfahYBHt2iXTlEzd/gdhVRxhlNs03L8wB0j/JWRZY8uetKcz7r0IQOHPoMfaoH esEnUumVyTV7uhENUw6+udBPnxlkBOKCxTJ9qHWDxgrB7oYAtJRw9jfXilUCUfWIU6IW nYzLs63NjzEZR5JDzrzRsZe8qWl175JErp/4kwqSmQ2ebVSU/ybnjhvjdHmjILqLjXxR ZFsr3xqrP5kmj80iQNRqmpX+KM8u6ExALvGMAD3Cfgg61AcrvtYhathSUKkq8vJAKcJy WX2SCgkI3FAqkxdwcaDu6m8fX09cctRnncJq3KXXTCE5kCPRVM5ibrbcFPlqQOY9wxPZ 6o4A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Gm-Message-State: AGRZ1gLtPjouujQ9qs00j5bEWuepNbjHIbV5C+jTh0zeN465tcx9OMo4 NEIy4qIalr48yFmgApGaqhUJGYMINjhnobsB8bwpnsL14GVpoaMxztY5fPv1vQ5fovQ95podYQ5 WjSN4l+z/0O7VsncMnEdc4GJObw3ouRYOb7e2eQZ6mNRmFpRoIOakaXcDX3lU7dBU2pSDsBvrMW MR+nlINpSwqCsqwnz25GU0avUW+8T3FsqEtult4Pi2dR9o9RQR5E/Y+O2XPLaf+ACEbPdZ+wqiW IX/2mNiVmKA+I59F0FOjeZ40L3KtIeXGqhylNqbw4Z3qGDjDtzawKKWhBtMstk/PLeAYvU0RYEy k0KWjbIAIA/ztj63KMiXo7a1HxczEpETt5BkftSsQz+ufYbgWJiXF1vFfeHNnk518laM2IGLcA= = X-Received: by 2002:a17:906:4003:: with SMTP id v3-v6mr1578404ejj.240.1542710134309; Tue, 20 Nov 2018 02:35:34 -0800 (PST) X-Received: by 2002:a17:906:4003:: with SMTP id v3-v6mr1578363ejj.240.1542710133351; Tue, 20 Nov 2018 02:35:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542710133; cv=none; d=google.com; s=arc-20160816; b=vHvZb4JCFckBTWAYY04OekA8wGumc2VkIZqRcr6xUaQ+7gAGQfIeTkUR12XwbPRuXN 3+alGeRApG0su42ISKO7kSFhqyfF5qvRuubl3iKtWaFElAlizRYIgBM+xPWQDInLrG1g E2L7TGqcjgu0cptldTFVuizw6VzBUcn5ZQmSLcRrT0NLzG6vitjmiwJtd6F4QbsJMvem bD58PUxNhI22e3uVPbK449uNoPA3EYoIgaUV213ox4Ot6GrNGAPp5WKHR3ro9WTpyqbb mVN1+MiH8ijDQQzmzoanEVPYmexarzrkmQR5JMravEY18dMr9AaHUm6YkPCMT0vTYx52 8ZZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=UipYMHSvtb8aOnY17kKqlQt8aWhzlLzqt0JzwQhgn3Q=; b=Pg4aoaoi3dvxI+pyi5oS+SQcWj6dVLN6M+r00ghOyBcUHHoLO2W5LFVbzH6o6RaGyd D2Kub6vF6lv3EgREAkVaqc3EdwhsRgHktN7xNWASJTpxbaZYHjIK2QUMoV39R3whgXZ5 JHVtll+YCTD2Ykh2pPdOIkTzVn4tyMyQR8ciXB5iR+REoSoijseQgCORJyDI2tYUsmF9 8qzs0kHIqzxUI/NAW+4SNvHC8awm45461eoRzuYye/Fgrw+YglNNUxGe1ststihEbldU S7+ALsla+CgA0inKPHqkDDI1l22Y6uDKJO4rvDMVfkmmvkQlEMhbvp6M7dzfQ6gE2igo T7yQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id a37sor5172224edd.23.2018.11.20.02.35.33 for (Google Transport Security); Tue, 20 Nov 2018 02:35:33 -0800 (PST) Received-SPF: pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Google-Smtp-Source: AFSGD/V5yVod2Y7gafwz/wymq1xETuPkqzv1mUxA0rNo7S2XrmUm5lrOtF+JGoyJr0YZVRuMcFZIxA== X-Received: by 2002:a05:6402:144c:: with SMTP id d12mr1853102edx.83.1542710132817; Tue, 20 Nov 2018 02:35:32 -0800 (PST) Received: from tiehlicka.suse.cz (prg-ext-pat.suse.com. [213.151.95.130]) by smtp.gmail.com with ESMTPSA id a15-v6sm5967233ejj.5.2018.11.20.02.35.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Nov 2018 02:35:32 -0800 (PST) From: Michal Hocko To: linux-api@vger.kernel.org Cc: Andrew Morton , Alexey Dobriyan , , LKML , Michal Hocko Subject: [RFC PATCH 3/3] mm, proc: report PR_SET_THP_DISABLE in proc Date: Tue, 20 Nov 2018 11:35:15 +0100 Message-Id: <20181120103515.25280-4-mhocko@kernel.org> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181120103515.25280-1-mhocko@kernel.org> References: <20181120103515.25280-1-mhocko@kernel.org> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Michal Hocko David Rientjes has reported that 1860033237d4 ("mm: make PR_SET_THP_DISABLE immediately active") has changed the way how we report THPable VMAs to the userspace. Their monitoring tool is triggering false alarms on PR_SET_THP_DISABLE tasks because it considers an insufficient THP usage as a memory fragmentation resp. memory pressure issue. Before the said commit each newly created VMA inherited VM_NOHUGEPAGE flag and that got exposed to the userspace via /proc//smaps file. This implementation had its downsides as explained in the commit message but it is true that the userspace doesn't have any means to query for the process wide THP enabled/disabled status. PR_SET_THP_DISABLE is a process wide flag so it makes a lot of sense to export in the process wide context rather than per-vma. Introduce a new field to /proc//status which export this status. If PR_SET_THP_DISABLE is used then it reports false same as when the THP is not compiled in. It doesn't consider the global THP status because we already export that information via sysfs Fixes: 1860033237d4 ("mm: make PR_SET_THP_DISABLE immediately active") Signed-off-by: Michal Hocko Acked-by: Vlastimil Babka --- Documentation/filesystems/proc.txt | 3 +++ fs/proc/array.c | 10 ++++++++++ 2 files changed, 13 insertions(+) diff --git a/Documentation/filesystems/proc.txt b/Documentation/filesystems/proc.txt index 06562bab509a..7995e9322889 100644 --- a/Documentation/filesystems/proc.txt +++ b/Documentation/filesystems/proc.txt @@ -182,6 +182,7 @@ For example, to get the status information of a process, all you have to do is VmSwap: 0 kB HugetlbPages: 0 kB CoreDumping: 0 + THP_enabled: 1 Threads: 1 SigQ: 0/28578 SigPnd: 0000000000000000 @@ -256,6 +257,8 @@ Table 1-2: Contents of the status files (as of 4.8) HugetlbPages size of hugetlb memory portions CoreDumping process's memory is currently being dumped (killing the process may lead to a corrupted core) + THP_enabled process is allowed to use THP (returns 0 when + PR_SET_THP_DISABLE is set on the process Threads number of threads SigQ number of signals queued/max. number for queue SigPnd bitmap of pending signals for the thread diff --git a/fs/proc/array.c b/fs/proc/array.c index 0ceb3b6b37e7..9d428d5a0ac8 100644 --- a/fs/proc/array.c +++ b/fs/proc/array.c @@ -392,6 +392,15 @@ static inline void task_core_dumping(struct seq_file *m, struct mm_struct *mm) seq_putc(m, '\n'); } +static inline void task_thp_status(struct seq_file *m, struct mm_struct *mm) +{ + bool thp_enabled = IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE); + + if (thp_enabled) + thp_enabled = !test_bit(MMF_DISABLE_THP, &mm->flags); + seq_printf(m, "THP_enabled:\t%d\n", thp_enabled); +} + int proc_pid_status(struct seq_file *m, struct pid_namespace *ns, struct pid *pid, struct task_struct *task) { @@ -406,6 +415,7 @@ int proc_pid_status(struct seq_file *m, struct pid_namespace *ns, if (mm) { task_mem(m, mm); task_core_dumping(m, mm); + task_thp_status(m, mm); mmput(mm); } task_sig(m, task);