From patchwork Sun Dec 1 01:49:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268179 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B8E26921 for ; Sun, 1 Dec 2019 01:49:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6A1612084E for ; Sun, 1 Dec 2019 01:49:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="TzSagZSl" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6A1612084E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 893B76B0005; Sat, 30 Nov 2019 20:49:11 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 844026B0006; Sat, 30 Nov 2019 20:49:11 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 781226B0007; Sat, 30 Nov 2019 20:49:11 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0146.hostedemail.com [216.40.44.146]) by kanga.kvack.org (Postfix) with ESMTP id 5B74A6B0005 for ; Sat, 30 Nov 2019 20:49:11 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 07738181AEF0B for ; Sun, 1 Dec 2019 01:49:11 +0000 (UTC) X-FDA: 76214889702.07.ant78_8a54b1194ce4a X-Spam-Summary: 2,0,0,2bcd116a29828586,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:colin.king@canonical.com::mm-commits@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:2:41:355:379:800:960:967:973:982:988:989:1260:1263:1345:1381:1431:1437:1535:1606:1730:1747:1777:1792:2393:2525:2553:2559:2563:2682:2685:2859:2893:2902:2914:2915:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3865:3866:3867:3868:3870:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4118:4321:5007:6119:6261:6653:7576:8599:9025:9545:10004:10913:11658:11914:12043:12048:12291:12297:12517:12519:12555:12679:12783:12986:13846:13869:14096:21063:21080:21433:21451:21611:21627:21740:21939:30054:30090,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: ant78_8a54b1194ce4a X-Filterd-Recvd-Size: 7128 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:10 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 653472082E; Sun, 1 Dec 2019 01:49:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164949; bh=jA3V692RnS6jK5pY5fXlcPyh7uJoE1K1VKgd+lHHU9Y=; h=Date:From:To:Subject:From; b=TzSagZSlclIImUEg3WArQptsdIMwJtBPnF5pokRRzwstQe9UZav/fhCG2tau2KNdU HnV9sW2j7afkjgKCWsmDl++tBTOzw33ljvPtcqachHX0FaUR9EMs7ZZEmFBUJBVsBj udLoP7scRt6gj+9h7WJhbyQZZV/fMqtzTP2Yofng= Date: Sat, 30 Nov 2019 17:49:08 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, colin.king@canonical.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 001/158] scripts/spelling.txt: add more spellings to spelling.txt Message-ID: <20191201014908.h3H96s7ip%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Colin Ian King Subject: scripts/spelling.txt: add more spellings to spelling.txt Here are some of the more common spelling mistakes and typos that I've found while fixing up spelling mistakes in the kernel since July 2019. Link: http://lkml.kernel.org/r/20191112092142.97989-1-colin.king@canonical.com Signed-off-by: Colin Ian King Signed-off-by: Andrew Morton --- scripts/spelling.txt | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) --- a/scripts/spelling.txt~scripts-spellingtxt-add-more-spellings-to-spellingtxt +++ a/scripts/spelling.txt @@ -87,6 +87,7 @@ algorith||algorithm algorithmical||algorithmically algoritm||algorithm algoritms||algorithms +algorithmn||algorithm algorrithm||algorithm algorritm||algorithm aligment||alignment @@ -109,6 +110,7 @@ alredy||already altough||although alue||value ambigious||ambiguous +ambigous||ambiguous amoung||among amout||amount amplifer||amplifier @@ -179,6 +181,7 @@ attepmpt||attempt attnetion||attention attruibutes||attributes authentification||authentication +authenicated||authenticated automaticaly||automatically automaticly||automatically automatize||automate @@ -286,6 +289,7 @@ claread||cleared clared||cleared closeing||closing clustred||clustered +cnfiguration||configuration coexistance||coexistence colescing||coalescing collapsable||collapsible @@ -325,9 +329,11 @@ comression||compression comunication||communication conbination||combination conditionaly||conditionally +conditon||condition conected||connected conector||connector connecetd||connected +configration||configuration configuartion||configuration configuation||configuration configued||configured @@ -347,6 +353,7 @@ containts||contains contaisn||contains contant||contact contence||contents +contiguos||contiguous continious||continuous continous||continuous continously||continuously @@ -380,6 +387,7 @@ cylic||cyclic dafault||default deafult||default deamon||daemon +debouce||debounce decompres||decompress decsribed||described decription||description @@ -448,6 +456,7 @@ diffrent||different differenciate||differentiate diffrentiate||differentiate difinition||definition +digial||digital dimention||dimension dimesions||dimensions dispalying||displaying @@ -489,6 +498,7 @@ droput||dropout druing||during dynmaic||dynamic eanable||enable +eanble||enable easilly||easily ecspecially||especially edditable||editable @@ -502,6 +512,7 @@ elementry||elementary eletronic||electronic embeded||embedded enabledi||enabled +enbale||enable enble||enable enchanced||enhanced encorporating||incorporating @@ -536,6 +547,7 @@ excellant||excellent execeeded||exceeded execeeds||exceeds exeed||exceed +exeuction||execution existance||existence existant||existent exixt||exist @@ -601,10 +613,12 @@ frambuffer||framebuffer framming||framing framwork||framework frequncy||frequency +frequancy||frequency frome||from fucntion||function fuction||function fuctions||functions +fullill||fulfill funcation||function funcion||function functionallity||functionality @@ -642,6 +656,7 @@ happend||happened harware||hardware heirarchically||hierarchically helpfull||helpful +hexdecimal||hexadecimal hybernate||hibernate hierachy||hierarchy hierarchie||hierarchy @@ -709,12 +724,14 @@ initalize||initialize initation||initiation initators||initiators initialiazation||initialization +initializationg||initialization initializiation||initialization initialze||initialize initialzed||initialized initialzing||initializing initilization||initialization initilize||initialize +initliaze||initialize inofficial||unofficial inrerface||interface insititute||institute @@ -779,6 +796,7 @@ itertation||iteration itslef||itself jave||java jeffies||jiffies +jumpimng||jumping juse||just jus||just kown||known @@ -839,6 +857,7 @@ messags||messages messgaes||messages messsage||message messsages||messages +metdata||metadata micropone||microphone microprocesspr||microprocessor migrateable||migratable @@ -857,6 +876,7 @@ mismactch||mismatch missign||missing missmanaged||mismanaged missmatch||mismatch +misssing||missing miximum||maximum mmnemonic||mnemonic mnay||many @@ -912,6 +932,7 @@ occured||occurred occuring||occurring offser||offset offet||offset +offlaod||offload offloded||offloaded offseting||offsetting omited||omitted @@ -993,6 +1014,7 @@ poiter||pointer posible||possible positon||position possibilites||possibilities +potocol||protocol powerfull||powerful pramater||parameter preamle||preamble @@ -1061,11 +1083,13 @@ psychadelic||psychedelic pwoer||power queing||queuing quering||querying +queus||queues randomally||randomly raoming||roaming reasearcher||researcher reasearchers||researchers reasearch||research +receieve||receive recepient||recipient recevied||received receving||receiving @@ -1166,6 +1190,7 @@ scaleing||scaling scaned||scanned scaning||scanning scarch||search +schdule||schedule seach||search searchs||searches secquence||sequence @@ -1308,6 +1333,7 @@ taskelt||tasklet teh||the temorary||temporary temproarily||temporarily +temperture||temperature thead||thread therfore||therefore thier||their @@ -1354,6 +1380,7 @@ uknown||unknown usupported||unsupported uncommited||uncommitted unconditionaly||unconditionally +undeflow||underflow underun||underrun unecessary||unnecessary unexecpted||unexpected @@ -1414,6 +1441,7 @@ varible||variable varient||variant vaule||value verbse||verbose +veify||verify verisons||versions verison||version verson||version From patchwork Sun Dec 1 01:49:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268181 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 47431112B for ; Sun, 1 Dec 2019 01:49:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 146162086A for ; Sun, 1 Dec 2019 01:49:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="Cf3JPE6I" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 146162086A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 252FD6B0006; Sat, 30 Nov 2019 20:49:15 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 228D76B0007; Sat, 30 Nov 2019 20:49:15 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 140276B0008; Sat, 30 Nov 2019 20:49:15 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0021.hostedemail.com [216.40.44.21]) by kanga.kvack.org (Postfix) with ESMTP id F1ADA6B0006 for ; Sat, 30 Nov 2019 20:49:14 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 6BCDA180AD817 for ; Sun, 1 Dec 2019 01:49:14 +0000 (UTC) X-FDA: 76214889828.01.truck97_8ad8711b78421 X-Spam-Summary: 2,0,0,8f38cb13656e9a00,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:dingxiang@cmss.chinamobile.com:gechangwei@live.cn:ghe@suse.com:jlbec@evilplan.org:joseph.qi@linux.alibaba.com:junxiao.bi@oracle.com::mark@fasheh.com:mm-commits@vger.kernel.org:piaojun@huawei.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2553:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3150:3352:3867:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:5007:6261:6642:6653:6737:7576:9025:9545:10004:11026:11257:11473:11658:11914:12043:12048:12297:12438:12517:12519:12679:12783:13069:13311:13357:14181:14384:14721:14849:21080:21451:21627:21939:30012:30054:30064:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMM ARY:none X-HE-Tag: truck97_8ad8711b78421 X-Filterd-Recvd-Size: 2751 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:14 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C1C862082E; Sun, 1 Dec 2019 01:49:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164953; bh=g3wxdX89bC1Cju5U8lE0wJ4zA2EnOOVdhahDX2luYlU=; h=Date:From:To:Subject:From; b=Cf3JPE6IeD9wMuK31cvTuIrcgDmHzYZuvhYShtRqIUzPP3VCr2IH5LDHDP520zDCJ rN0g8eEKgFrsJFbqtG5ndWt1f28ZYyAgYOcKjYahPahKL/4eL9pt/SGOfD5BMHjBPf QQrBOjYqXsu0ipxD+A1AJy9vAXKU7vCLL99voPKc= Date: Sat, 30 Nov 2019 17:49:12 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, dingxiang@cmss.chinamobile.com, gechangwei@live.cn, ghe@suse.com, jlbec@evilplan.org, joseph.qi@linux.alibaba.com, junxiao.bi@oracle.com, linux-mm@kvack.org, mark@fasheh.com, mm-commits@vger.kernel.org, piaojun@huawei.com, torvalds@linux-foundation.org Subject: [patch 002/158] ocfs2: fix passing zero to 'PTR_ERR' warning Message-ID: <20191201014912.ctRmxGgMW%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Ding Xiang Subject: ocfs2: fix passing zero to 'PTR_ERR' warning Fix a static code checker warning: fs/ocfs2/acl.c:331 ocfs2_acl_chmod() warn: passing zero to 'PTR_ERR' Link: http://lkml.kernel.org/r/1dee278b-6c96-eec2-ce76-fe6e07c6e20f@linux.alibaba.com Fixes: 5ee0fbd50fd ("ocfs2: revert using ocfs2_acl_chmod to avoid inode cluster lock hang") Signed-off-by: Ding Xiang Reviewed-by: Joseph Qi Cc: Mark Fasheh Cc: Joel Becker Cc: Junxiao Bi Cc: Changwei Ge Cc: Gang He Cc: Jun Piao Signed-off-by: Andrew Morton --- fs/ocfs2/acl.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/fs/ocfs2/acl.c~ocfs2-fix-passing-zero-to-ptr_err-warning +++ a/fs/ocfs2/acl.c @@ -327,8 +327,8 @@ int ocfs2_acl_chmod(struct inode *inode, down_read(&OCFS2_I(inode)->ip_xattr_sem); acl = ocfs2_get_acl_nolock(inode, ACL_TYPE_ACCESS, bh); up_read(&OCFS2_I(inode)->ip_xattr_sem); - if (IS_ERR(acl) || !acl) - return PTR_ERR(acl); + if (IS_ERR_OR_NULL(acl)) + return PTR_ERR_OR_ZERO(acl); ret = __posix_acl_chmod(&acl, GFP_KERNEL, inode->i_mode); if (ret) return ret; From patchwork Sun Dec 1 01:49:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268183 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9ADF2112B for ; Sun, 1 Dec 2019 01:49:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 67311208C3 for ; Sun, 1 Dec 2019 01:49:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="wm1fKbnr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 67311208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 426BA6B0007; Sat, 30 Nov 2019 20:49:18 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3D71F6B0008; Sat, 30 Nov 2019 20:49:18 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2A0776B000A; Sat, 30 Nov 2019 20:49:18 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0214.hostedemail.com [216.40.44.214]) by kanga.kvack.org (Postfix) with ESMTP id 1278F6B0007 for ; Sat, 30 Nov 2019 20:49:18 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 8CF9F824999B for ; Sun, 1 Dec 2019 01:49:17 +0000 (UTC) X-FDA: 76214889954.07.coach13_8b4de7899080f X-Spam-Summary: 2,0,0,ae74b8151266d536,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org::mm-commits@vger.kernel.org:saurav.girepunje@gmail.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1540:1568:1711:1714:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3866:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6114:6261:6642:6653:7514:7576:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12297:12517:12519:12555:12679:12696:12737:12783:12986:13069:13311:13357:14094:14181:14384:14721:14849:21080:21451:21627:21939:30054,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: coach13_8b4de7899080f X-Filterd-Recvd-Size: 2042 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:17 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1F77C20873; Sun, 1 Dec 2019 01:49:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164956; bh=KLMatcQYrmmT8aYyl83ZaKHpGqDP2xsM+GZwpYRisbI=; h=Date:From:To:Subject:From; b=wm1fKbnrJd2Mc3LSUWT3fobwSJ5MxUiD2aVH+i0KRc3z1oWe3dSjkWHJBbljmMIwm eJKFW4Pq6FBVt1uVnWOv+GF6C01qwGm+Xuu3iU9uTYeuw8B2v1YvZRyEA2PNdbpLJJ 7vmCvt4H/8789pYlWm1hIdQNzON7Pc44M3COKeBY= Date: Sat, 30 Nov 2019 17:49:15 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, saurav.girepunje@gmail.com, torvalds@linux-foundation.org Subject: [patch 003/158] fs/buffer.c: fix use true/false for bool type Message-ID: <20191201014915.JybSfMECc%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Saurav Girepunje Subject: fs/buffer.c: fix use true/false for bool type Use true/false for bool return type of has_bh_in_lru(). Link: http://lkml.kernel.org/r/20191029040529.GA7625@saurav Signed-off-by: Saurav Girepunje Reviewed-by: Andrew Morton Signed-off-by: Andrew Morton --- fs/buffer.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/fs/buffer.c~fs-bufferc-fix-use-true-false-for-bool-type +++ a/fs/buffer.c @@ -1423,10 +1423,10 @@ static bool has_bh_in_lru(int cpu, void for (i = 0; i < BH_LRU_SIZE; i++) { if (b->bhs[i]) - return 1; + return true; } - return 0; + return false; } void invalidate_bh_lrus(void) From patchwork Sun Dec 1 01:49:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268185 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 79A27921 for ; Sun, 1 Dec 2019 01:49:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 45EFF20873 for ; Sun, 1 Dec 2019 01:49:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="UV9JCZ6z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 45EFF20873 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2C72E6B0008; Sat, 30 Nov 2019 20:49:21 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 27B716B000A; Sat, 30 Nov 2019 20:49:21 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 16A976B000C; Sat, 30 Nov 2019 20:49:21 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0041.hostedemail.com [216.40.44.41]) by kanga.kvack.org (Postfix) with ESMTP id F10D36B0008 for ; Sat, 30 Nov 2019 20:49:20 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 877FE40D9 for ; Sun, 1 Dec 2019 01:49:20 +0000 (UTC) X-FDA: 76214890080.09.sense47_8bbe6bfb8d65b X-Spam-Summary: 2,0,0,fbd4923fedcbe782,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:ben.dooks@codethink.co.uk:jack@suse.cz::mm-commits@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:334:355:368:369:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1434:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2553:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:8599:9010:9025:9545:10004:10913:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13069:13161:13229:13311:13357:14094:14181:14384:14721:14849:21080:21451:21627:21939:30054:30089:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: sense47_8bbe6bfb8d65b X-Filterd-Recvd-Size: 2409 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:20 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 40ABE2084E; Sun, 1 Dec 2019 01:49:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164959; bh=7Ssz0t8R07ydobDyOrwbuXI3iJjMplcN9toGpSzLBLA=; h=Date:From:To:Subject:From; b=UV9JCZ6zmRlu/x93aKwmgmXwLy9wQs33uHyveCAhyIxd52g7x0D1rANLIVxk/mj/d +noQn37wO9D5bcw4rqZKstbJ1JPrDFxfeXMcvoWMYzyNpyo2PvYuwpujN2NhO8rr5o o4/v6RF0x3sYXg6ah75VYCvn45wbrc8GwBVa3PU4= Date: Sat, 30 Nov 2019 17:49:18 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, ben.dooks@codethink.co.uk, jack@suse.cz, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 004/158] fs/buffer.c: include internal.h for missing declarations Message-ID: <20191201014918.ogCnbxSwd%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Ben Dooks Subject: fs/buffer.c: include internal.h for missing declarations The declarations of __block_write_begin_int and guard_bio_eod are needed from internal.h so include it to fix the following sparse warnings: fs/buffer.c:1930:5: warning: symbol '__block_write_begin_int' was not declared. Should it be static? fs/buffer.c:2994:6: warning: symbol 'guard_bio_eod' was not declared. Should it be static? Link: http://lkml.kernel.org/r/20191011170039.16100-1-ben.dooks@codethink.co.uk Signed-off-by: Ben Dooks Reviewed-by: Jan Kara Signed-off-by: Andrew Morton --- fs/buffer.c | 2 ++ 1 file changed, 2 insertions(+) --- a/fs/buffer.c~fs-include-internalh-for-missing-declarations +++ a/fs/buffer.c @@ -49,6 +49,8 @@ #include #include +#include "internal.h" + static int fsync_buffers_list(spinlock_t *lock, struct list_head *list); static int submit_bh_wbc(int op, int op_flags, struct buffer_head *bh, enum rw_hint hint, struct writeback_control *wbc); From patchwork Sun Dec 1 01:49:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268187 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BACD0921 for ; Sun, 1 Dec 2019 01:49:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7DD322084E for ; Sun, 1 Dec 2019 01:49:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="PSfsQ5U4" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7DD322084E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5FCBF6B000C; Sat, 30 Nov 2019 20:49:24 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5D2EB6B000D; Sat, 30 Nov 2019 20:49:24 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4EA5A6B000E; Sat, 30 Nov 2019 20:49:24 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0167.hostedemail.com [216.40.44.167]) by kanga.kvack.org (Postfix) with ESMTP id 357166B000C for ; Sat, 30 Nov 2019 20:49:24 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id D5DED180AD817 for ; Sun, 1 Dec 2019 01:49:23 +0000 (UTC) X-FDA: 76214890206.17.sail92_8c361461f4928 X-Spam-Summary: 2,0,0,4584b0df0a1dbbad,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:cl@linux.com:guro@fb.com:iamjoonsoo.kim@lge.com::lpf.vector@gmail.com:mm-commits@vger.kernel.org:penberg@kernel.org:rientjes@google.com:torvalds@linux-foundation.org:vbabka@suse.cz,RULES_HIT:2:41:69:355:379:421:800:960:967:968:973:981:988:989:1260:1263:1345:1381:1431:1437:1500:1535:1605:1730:1747:1777:1792:1801:2198:2199:2393:2525:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4050:4119:4321:4605:5007:6238:6261:6653:6737:7514:7576:7903:8603:9025:9121:9391:9545:9592:10004:10913:11026:11232:11233:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12783:12895:12986:13161:13229:13846:21080:21433:21450:21451:21627:21740:21819:21939:30029:30030:30054:30064,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5, Netcheck X-HE-Tag: sail92_8c361461f4928 X-Filterd-Recvd-Size: 8386 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:23 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 52686208E4; Sun, 1 Dec 2019 01:49:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164962; bh=QnzAKFkV/z6rW3wsvoU+snGKoMFYSfukfkSO2TWD4SU=; h=Date:From:To:Subject:From; b=PSfsQ5U4MSu1T+cehqW7q3e8giA91GV1uxQ7bjFoIDqXNjVTTzClB6+ye1SQ03/d6 12qRMpZMhrVKBh3HVHNLeCAbfMo0qXrU0hNt/WqIMU4ObiN1MipUkhFuGZwUHcrtM8 t1iGamgQXCaYRXEjy3vLpTObCgHD/aqqDufZ8EoY= Date: Sat, 30 Nov 2019 17:49:21 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, cl@linux.com, guro@fb.com, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, lpf.vector@gmail.com, mm-commits@vger.kernel.org, penberg@kernel.org, rientjes@google.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 005/158] mm, slab: make kmalloc_info[] contain all types of names Message-ID: <20191201014921.zdPUUREVo%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Pengfei Li Subject: mm, slab: make kmalloc_info[] contain all types of names Patch series "mm, slab: Make kmalloc_info[] contain all types of names", v6. There are three types of kmalloc, KMALLOC_NORMAL, KMALLOC_RECLAIM and KMALLOC_DMA. The name of KMALLOC_NORMAL is contained in kmalloc_info[].name, but the names of KMALLOC_RECLAIM and KMALLOC_DMA are dynamically generated by kmalloc_cache_name(). Patch1 predefines the names of all types of kmalloc to save the time spent dynamically generating names. These changes make sense, and the time spent by new_kmalloc_cache() has been reduced by approximately 36.3%. Time spent by new_kmalloc_cache() (CPU cycles) 5.3-rc7 66264 5.3-rc7+patch 42188 This patch (of 3): There are three types of kmalloc, KMALLOC_NORMAL, KMALLOC_RECLAIM and KMALLOC_DMA. The name of KMALLOC_NORMAL is contained in kmalloc_info[].name, but the names of KMALLOC_RECLAIM and KMALLOC_DMA are dynamically generated by kmalloc_cache_name(). This patch predefines the names of all types of kmalloc to save the time spent dynamically generating names. Besides, remove the kmalloc_cache_name() that is no longer used. Link: http://lkml.kernel.org/r/1569241648-26908-2-git-send-email-lpf.vector@gmail.com Signed-off-by: Pengfei Li Acked-by: Vlastimil Babka Acked-by: Roman Gushchin Acked-by: David Rientjes Cc: Christoph Lameter Cc: Pekka Enberg Cc: Joonsoo Kim Signed-off-by: Andrew Morton --- mm/slab.c | 2 mm/slab.h | 2 mm/slab_common.c | 91 ++++++++++++++++++++++++--------------------- 3 files changed, 51 insertions(+), 44 deletions(-) --- a/mm/slab.c~mm-slab-make-kmalloc_info-contain-all-types-of-names +++ a/mm/slab.c @@ -1247,7 +1247,7 @@ void __init kmem_cache_init(void) * structures first. Without this, further allocations will bug. */ kmalloc_caches[KMALLOC_NORMAL][INDEX_NODE] = create_kmalloc_cache( - kmalloc_info[INDEX_NODE].name, + kmalloc_info[INDEX_NODE].name[KMALLOC_NORMAL], kmalloc_size(INDEX_NODE), ARCH_KMALLOC_FLAGS, 0, kmalloc_size(INDEX_NODE)); slab_state = PARTIAL_NODE; --- a/mm/slab_common.c~mm-slab-make-kmalloc_info-contain-all-types-of-names +++ a/mm/slab_common.c @@ -1139,26 +1139,56 @@ struct kmem_cache *kmalloc_slab(size_t s return kmalloc_caches[kmalloc_type(flags)][index]; } +#ifdef CONFIG_ZONE_DMA +#define INIT_KMALLOC_INFO(__size, __short_size) \ +{ \ + .name[KMALLOC_NORMAL] = "kmalloc-" #__short_size, \ + .name[KMALLOC_RECLAIM] = "kmalloc-rcl-" #__short_size, \ + .name[KMALLOC_DMA] = "dma-kmalloc-" #__short_size, \ + .size = __size, \ +} +#else +#define INIT_KMALLOC_INFO(__size, __short_size) \ +{ \ + .name[KMALLOC_NORMAL] = "kmalloc-" #__short_size, \ + .name[KMALLOC_RECLAIM] = "kmalloc-rcl-" #__short_size, \ + .size = __size, \ +} +#endif + /* * kmalloc_info[] is to make slub_debug=,kmalloc-xx option work at boot time. * kmalloc_index() supports up to 2^26=64MB, so the final entry of the table is * kmalloc-67108864. */ const struct kmalloc_info_struct kmalloc_info[] __initconst = { - {NULL, 0}, {"kmalloc-96", 96}, - {"kmalloc-192", 192}, {"kmalloc-8", 8}, - {"kmalloc-16", 16}, {"kmalloc-32", 32}, - {"kmalloc-64", 64}, {"kmalloc-128", 128}, - {"kmalloc-256", 256}, {"kmalloc-512", 512}, - {"kmalloc-1k", 1024}, {"kmalloc-2k", 2048}, - {"kmalloc-4k", 4096}, {"kmalloc-8k", 8192}, - {"kmalloc-16k", 16384}, {"kmalloc-32k", 32768}, - {"kmalloc-64k", 65536}, {"kmalloc-128k", 131072}, - {"kmalloc-256k", 262144}, {"kmalloc-512k", 524288}, - {"kmalloc-1M", 1048576}, {"kmalloc-2M", 2097152}, - {"kmalloc-4M", 4194304}, {"kmalloc-8M", 8388608}, - {"kmalloc-16M", 16777216}, {"kmalloc-32M", 33554432}, - {"kmalloc-64M", 67108864} + INIT_KMALLOC_INFO(0, 0), + INIT_KMALLOC_INFO(96, 96), + INIT_KMALLOC_INFO(192, 192), + INIT_KMALLOC_INFO(8, 8), + INIT_KMALLOC_INFO(16, 16), + INIT_KMALLOC_INFO(32, 32), + INIT_KMALLOC_INFO(64, 64), + INIT_KMALLOC_INFO(128, 128), + INIT_KMALLOC_INFO(256, 256), + INIT_KMALLOC_INFO(512, 512), + INIT_KMALLOC_INFO(1024, 1k), + INIT_KMALLOC_INFO(2048, 2k), + INIT_KMALLOC_INFO(4096, 4k), + INIT_KMALLOC_INFO(8192, 8k), + INIT_KMALLOC_INFO(16384, 16k), + INIT_KMALLOC_INFO(32768, 32k), + INIT_KMALLOC_INFO(65536, 64k), + INIT_KMALLOC_INFO(131072, 128k), + INIT_KMALLOC_INFO(262144, 256k), + INIT_KMALLOC_INFO(524288, 512k), + INIT_KMALLOC_INFO(1048576, 1M), + INIT_KMALLOC_INFO(2097152, 2M), + INIT_KMALLOC_INFO(4194304, 4M), + INIT_KMALLOC_INFO(8388608, 8M), + INIT_KMALLOC_INFO(16777216, 16M), + INIT_KMALLOC_INFO(33554432, 32M), + INIT_KMALLOC_INFO(67108864, 64M) }; /* @@ -1208,36 +1238,14 @@ void __init setup_kmalloc_cache_index_ta } } -static const char * -kmalloc_cache_name(const char *prefix, unsigned int size) -{ - - static const char units[3] = "\0kM"; - int idx = 0; - - while (size >= 1024 && (size % 1024 == 0)) { - size /= 1024; - idx++; - } - - return kasprintf(GFP_NOWAIT, "%s-%u%c", prefix, size, units[idx]); -} - static void __init new_kmalloc_cache(int idx, int type, slab_flags_t flags) { - const char *name; - - if (type == KMALLOC_RECLAIM) { + if (type == KMALLOC_RECLAIM) flags |= SLAB_RECLAIM_ACCOUNT; - name = kmalloc_cache_name("kmalloc-rcl", - kmalloc_info[idx].size); - BUG_ON(!name); - } else { - name = kmalloc_info[idx].name; - } - kmalloc_caches[type][idx] = create_kmalloc_cache(name, + kmalloc_caches[type][idx] = create_kmalloc_cache( + kmalloc_info[idx].name[type], kmalloc_info[idx].size, flags, 0, kmalloc_info[idx].size); } @@ -1279,11 +1287,10 @@ void __init create_kmalloc_caches(slab_f if (s) { unsigned int size = kmalloc_size(i); - const char *n = kmalloc_cache_name("dma-kmalloc", size); - BUG_ON(!n); kmalloc_caches[KMALLOC_DMA][i] = create_kmalloc_cache( - n, size, SLAB_CACHE_DMA | flags, 0, 0); + kmalloc_info[i].name[KMALLOC_DMA], + size, SLAB_CACHE_DMA | flags, 0, 0); } } #endif --- a/mm/slab.h~mm-slab-make-kmalloc_info-contain-all-types-of-names +++ a/mm/slab.h @@ -139,7 +139,7 @@ extern struct kmem_cache *kmem_cache; /* A table of kmalloc cache names and sizes */ extern const struct kmalloc_info_struct { - const char *name; + const char *name[NR_KMALLOC_TYPES]; unsigned int size; } kmalloc_info[]; From patchwork Sun Dec 1 01:49:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268189 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EDDAE112B for ; Sun, 1 Dec 2019 01:49:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BAB4A208E4 for ; Sun, 1 Dec 2019 01:49:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="dZj46PKh" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BAB4A208E4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AB6776B000E; Sat, 30 Nov 2019 20:49:27 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A68476B0010; Sat, 30 Nov 2019 20:49:27 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 981596B0266; Sat, 30 Nov 2019 20:49:27 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0125.hostedemail.com [216.40.44.125]) by kanga.kvack.org (Postfix) with ESMTP id 7E5A46B000E for ; Sat, 30 Nov 2019 20:49:27 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 2AA96824999B for ; Sun, 1 Dec 2019 01:49:27 +0000 (UTC) X-FDA: 76214890374.11.work88_8cae81c366b43 X-Spam-Summary: 2,0,0,dd91184dd35c57e8,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:cl@linux.com:guro@fb.com:iamjoonsoo.kim@lge.com::lpf.vector@gmail.com:mm-commits@vger.kernel.org:penberg@kernel.org:rientjes@google.com:torvalds@linux-foundation.org:vbabka@suse.cz,RULES_HIT:41:69:355:379:800:960:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4605:5007:6119:6261:6653:6737:7514:7576:7903:8603:9025:9391:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:13846:14181:14721:14849:21080:21433:21451:21627:21819:21939:30029:30054:30064,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom _rules:0 X-HE-Tag: work88_8cae81c366b43 X-Filterd-Recvd-Size: 3899 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:26 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 8120A208E4; Sun, 1 Dec 2019 01:49:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164965; bh=REeKEyyopYmTdi2ZVoRvNSsV6B1BYOcXt+FTcCba8Hs=; h=Date:From:To:Subject:From; b=dZj46PKhYINnru9FUfqyxCo22FHUR/vbkV92/xtWfLzB71w3cqMeyyZdF8ZvMh/2q gCEp0RDpeCE6LSBhk0Eo6UfkhGnQMIMgOUtZJxbyU1Rx0F9LNKRuhLCqUarcomyBiX AhdiVJ0iYer2GI3sFllMlzcSBTWolkRzihfPKtT4= Date: Sat, 30 Nov 2019 17:49:25 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, cl@linux.com, guro@fb.com, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, lpf.vector@gmail.com, mm-commits@vger.kernel.org, penberg@kernel.org, rientjes@google.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 006/158] mm, slab: remove unused kmalloc_size() Message-ID: <20191201014925.1hEJTHm63%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Pengfei Li Subject: mm, slab: remove unused kmalloc_size() The size of kmalloc can be obtained from kmalloc_info[], so remove kmalloc_size() that will not be used anymore. Link: http://lkml.kernel.org/r/1569241648-26908-3-git-send-email-lpf.vector@gmail.com Signed-off-by: Pengfei Li Acked-by: Vlastimil Babka Acked-by: Roman Gushchin Acked-by: David Rientjes Cc: Christoph Lameter Cc: Joonsoo Kim Cc: Pekka Enberg Signed-off-by: Andrew Morton --- include/linux/slab.h | 20 -------------------- mm/slab.c | 5 +++-- mm/slab_common.c | 5 ++--- 3 files changed, 5 insertions(+), 25 deletions(-) --- a/include/linux/slab.h~mm-slab-remove-unused-kmalloc_size +++ a/include/linux/slab.h @@ -561,26 +561,6 @@ static __always_inline void *kmalloc(siz return __kmalloc(size, flags); } -/* - * Determine size used for the nth kmalloc cache. - * return size or 0 if a kmalloc cache for that - * size does not exist - */ -static __always_inline unsigned int kmalloc_size(unsigned int n) -{ -#ifndef CONFIG_SLOB - if (n > 2) - return 1U << n; - - if (n == 1 && KMALLOC_MIN_SIZE <= 32) - return 96; - - if (n == 2 && KMALLOC_MIN_SIZE <= 64) - return 192; -#endif - return 0; -} - static __always_inline void *kmalloc_node(size_t size, gfp_t flags, int node) { #ifndef CONFIG_SLOB --- a/mm/slab.c~mm-slab-remove-unused-kmalloc_size +++ a/mm/slab.c @@ -1248,8 +1248,9 @@ void __init kmem_cache_init(void) */ kmalloc_caches[KMALLOC_NORMAL][INDEX_NODE] = create_kmalloc_cache( kmalloc_info[INDEX_NODE].name[KMALLOC_NORMAL], - kmalloc_size(INDEX_NODE), ARCH_KMALLOC_FLAGS, - 0, kmalloc_size(INDEX_NODE)); + kmalloc_info[INDEX_NODE].size, + ARCH_KMALLOC_FLAGS, 0, + kmalloc_info[INDEX_NODE].size); slab_state = PARTIAL_NODE; setup_kmalloc_cache_index_table(); --- a/mm/slab_common.c~mm-slab-remove-unused-kmalloc_size +++ a/mm/slab_common.c @@ -1286,11 +1286,10 @@ void __init create_kmalloc_caches(slab_f struct kmem_cache *s = kmalloc_caches[KMALLOC_NORMAL][i]; if (s) { - unsigned int size = kmalloc_size(i); - kmalloc_caches[KMALLOC_DMA][i] = create_kmalloc_cache( kmalloc_info[i].name[KMALLOC_DMA], - size, SLAB_CACHE_DMA | flags, 0, 0); + kmalloc_info[i].size, + SLAB_CACHE_DMA | flags, 0, 0); } } #endif From patchwork Sun Dec 1 01:49:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268191 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EC77E112B for ; Sun, 1 Dec 2019 01:49:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B9602208C3 for ; Sun, 1 Dec 2019 01:49:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="eVgvOgr3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B9602208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A418F6B0266; Sat, 30 Nov 2019 20:49:30 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9F2106B0269; Sat, 30 Nov 2019 20:49:30 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9088E6B026A; Sat, 30 Nov 2019 20:49:30 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0056.hostedemail.com [216.40.44.56]) by kanga.kvack.org (Postfix) with ESMTP id 73E296B0266 for ; Sat, 30 Nov 2019 20:49:30 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 21D0040D0 for ; Sun, 1 Dec 2019 01:49:30 +0000 (UTC) X-FDA: 76214890500.04.cry08_8d23a9901f64b X-Spam-Summary: 2,0,0,89f48f1b21e4b936,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:cl@linux.com:guro@fb.com:iamjoonsoo.kim@lge.com::lpf.vector@gmail.com:mm-commits@vger.kernel.org:penberg@kernel.org:rientjes@google.com:torvalds@linux-foundation.org:vbabka@suse.cz,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3867:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:6737:7514:7576:9025:9391:9545:10004:10913:11026:11473:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12783:12986:13069:13161:13229:13255:13311:13357:13846:14181:14384:14721:14777:14849:21080:21433:21450:21451:21627:21819:21939:30054:30064,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_r ules:0:0 X-HE-Tag: cry08_8d23a9901f64b X-Filterd-Recvd-Size: 2902 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:29 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B07DA2086A; Sun, 1 Dec 2019 01:49:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164969; bh=oXdOGy7PFkFQdixH2UvYWKz8XJ8MEtX2XEiBLTrGah8=; h=Date:From:To:Subject:From; b=eVgvOgr3HyfllHgHhNZzec8T6hh51UMO0XJsvWRrfDNPwDaYByy4tf/h86wJJxNSM HY+hvfG+yay6KQt8pvSBcSMiWrUc+SEHAo5fSvJlJLX6Q3YUT+hDfd8eNbuZCyTc2j xUnlVnEjV81C4Qt9C5H5uuHuaekIagSGqYEBNsIc= Date: Sat, 30 Nov 2019 17:49:28 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, cl@linux.com, guro@fb.com, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, lpf.vector@gmail.com, mm-commits@vger.kernel.org, penberg@kernel.org, rientjes@google.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 007/158] mm, slab_common: use enum kmalloc_cache_type to iterate over kmalloc caches Message-ID: <20191201014928.40nQ9dUfS%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Pengfei Li Subject: mm, slab_common: use enum kmalloc_cache_type to iterate over kmalloc caches The type of local variable *type* of new_kmalloc_cache() should be enum kmalloc_cache_type instead of int, so correct it. Link: http://lkml.kernel.org/r/1569241648-26908-4-git-send-email-lpf.vector@gmail.com Signed-off-by: Pengfei Li Acked-by: Vlastimil Babka Acked-by: Roman Gushchin Acked-by: David Rientjes Cc: Christoph Lameter Cc: Joonsoo Kim Cc: Pekka Enberg Signed-off-by: Andrew Morton --- mm/slab_common.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) --- a/mm/slab_common.c~mm-slab_common-use-enum-kmalloc_cache_type-to-iterate-over-kmalloc-caches +++ a/mm/slab_common.c @@ -1239,7 +1239,7 @@ void __init setup_kmalloc_cache_index_ta } static void __init -new_kmalloc_cache(int idx, int type, slab_flags_t flags) +new_kmalloc_cache(int idx, enum kmalloc_cache_type type, slab_flags_t flags) { if (type == KMALLOC_RECLAIM) flags |= SLAB_RECLAIM_ACCOUNT; @@ -1257,7 +1257,8 @@ new_kmalloc_cache(int idx, int type, sla */ void __init create_kmalloc_caches(slab_flags_t flags) { - int i, type; + int i; + enum kmalloc_cache_type type; for (type = KMALLOC_NORMAL; type <= KMALLOC_RECLAIM; type++) { for (i = KMALLOC_SHIFT_LOW; i <= KMALLOC_SHIFT_HIGH; i++) { From patchwork Sun Dec 1 01:49:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268193 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8DCEB921 for ; Sun, 1 Dec 2019 01:49:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4427920880 for ; Sun, 1 Dec 2019 01:49:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="Y+zdsl9E" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4427920880 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2C84A6B026A; Sat, 30 Nov 2019 20:49:34 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 278136B026B; Sat, 30 Nov 2019 20:49:34 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 18EF36B026C; Sat, 30 Nov 2019 20:49:34 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0253.hostedemail.com [216.40.44.253]) by kanga.kvack.org (Postfix) with ESMTP id F205F6B026A for ; Sat, 30 Nov 2019 20:49:33 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id ABCB0181AEF0B for ; Sun, 1 Dec 2019 01:49:33 +0000 (UTC) X-FDA: 76214890626.29.coat43_8d9e0e995bb1c X-Spam-Summary: 2,0,0,6969507a1246912d,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:cl@linux.com::miles.chen@mediatek.com:mm-commits@vger.kernel.org:rientjes@google.com:torvalds@linux-foundation.org:willy@infradead.org,RULES_HIT:2:41:69:146:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1535:1605:1606:1730:1747:1777:1792:2393:2525:2553:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4117:4321:5007:6261:6653:7264:7576:7875:7903:8599:9025:9545:10004:10913:11026:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13192:13229:13846:21080:21220:21451:21524:21627:21939:30003:30054:30075:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:84,LUA_SUMMARY:none X-HE-Tag: coat43_8d9e0e995bb1c X-Filterd-Recvd-Size: 6820 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:33 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 103B52086A; Sun, 1 Dec 2019 01:49:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164972; bh=aYM3S9kWGFByG7iEBcAYRZQ705tbtSvRpaPXWhZ4KCk=; h=Date:From:To:Subject:From; b=Y+zdsl9Egxw3Q0TJcl1Di3dVhzjV67ESqRcwv4qZnnoqOskZclfC9B6j0ukZjlTi/ aaLG4s3YV5HOaSqa5AVCK/xxaT4Aod6g/vPeJ2EFgdmP0FIuQrzb+ABpbZJ2Hstizr xEK3X2ey/ugK70eP4/hUYaDC9d5MmcUbnG1pyCBE= Date: Sat, 30 Nov 2019 17:49:31 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, cl@linux.com, linux-mm@kvack.org, miles.chen@mediatek.com, mm-commits@vger.kernel.org, rientjes@google.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 008/158] mm: slub: print the offset of fault addresses Message-ID: <20191201014931.7HW4m0bz7%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miles Chen Subject: mm: slub: print the offset of fault addresses With commit ad67b74d2469d9b8 ("printk: hash addresses printed with %p"), it is a little bit harder to match the fault addresses printed by check_bytes_and_report() or slab_pad_check() in the dump because the fault addresses may not show up in the dump. Print the offset of the fault addresses to make it easier to match the incorrect poison or padding values in the dump. Before: We have to search the "63" in the dump. If we want to get the offset of 63, we have to count it from the start of Object dump. ============================================================= BUG kmalloc-128 (Not tainted): Poison overwritten ------------------------------------------------------------- Disabling lock debugging due to kernel taint INFO: 0x00000000570da294-0x00000000570da294. First byte 0x63 instead of 0x6b ... INFO: Object 0x000000006ebb3b9e @offset=14208 fp=0x0000000065862488 Redzone 00000000a6abccff: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Redzone 00000000741c16f0: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Redzone 0000000061ad278f: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Redzone 000000000467c1bd: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Redzone 000000008812766b: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Redzone 000000003d9b8f25: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Redzone 0000000000d80c33: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Redzone 00000000867b0d90: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Object 000000006ebb3b9e: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b Object 000000005ea59a9f: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b Object 000000003ef8bddc: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b Object 000000008190375d: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b Object 000000006df7fb32: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b Object 0000000069474eae: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b Object 0000000008073b7d: 6b 6b 6b 6b 63 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b Object 00000000b45ae74d: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 After: We know the fault address is at @offset=1508, and the Object is at @offset=1408, so we know the fault address is at offset=100 within the object. ========================================================= BUG kmalloc-128 (Not tainted): Poison overwritten --------------------------------------------------------- Disabling lock debugging due to kernel taint INFO: 0x00000000638ec1d1-0x00000000638ec1d1 @offset=1508. First byte 0x63 instead of 0x6b ... INFO: Object 0x000000008171818d @offset=1408 fp=0x0000000066dae230 Redzone 00000000e2697ab6: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Redzone 0000000064b6a381: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Redzone 00000000e413a234: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Redzone 0000000004c1dfeb: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Redzone 000000009ad24d42: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Redzone 000000002a196a23: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Redzone 00000000a7b8468a: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Redzone 0000000088db6da3: bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb bb Object 000000008171818d: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b Object 000000007c4035d4: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b Object 000000004dd281a4: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b Object 0000000079121dff: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b Object 00000000756682a9: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b Object 0000000053b7e541: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b Object 0000000091f8d530: 6b 6b 6b 6b 63 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b Object 000000009c76035c: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 Link: http://lkml.kernel.org/r/20190925140807.20490-1-miles.chen@mediatek.com Signed-off-by: Miles Chen Reviewed-by: Andrew Morton Cc: Christoph Lameter Cc: Matthew Wilcox Cc: David Rientjes Signed-off-by: Andrew Morton --- mm/slub.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) --- a/mm/slub.c~mm-slub-print-the-offset-of-fault-addresses +++ a/mm/slub.c @@ -736,6 +736,7 @@ static int check_bytes_and_report(struct { u8 *fault; u8 *end; + u8 *addr = page_address(page); metadata_access_enable(); fault = memchr_inv(start, value, bytes); @@ -748,8 +749,9 @@ static int check_bytes_and_report(struct end--; slab_bug(s, "%s overwritten", what); - pr_err("INFO: 0x%p-0x%p. First byte 0x%x instead of 0x%x\n", - fault, end - 1, fault[0], value); + pr_err("INFO: 0x%p-0x%p @offset=%tu. First byte 0x%x instead of 0x%x\n", + fault, end - 1, fault - addr, + fault[0], value); print_trailer(s, page, object); restore_bytes(s, what, value, fault, end); @@ -844,7 +846,8 @@ static int slab_pad_check(struct kmem_ca while (end > fault && end[-1] == POISON_INUSE) end--; - slab_err(s, page, "Padding overwritten. 0x%p-0x%p", fault, end - 1); + slab_err(s, page, "Padding overwritten. 0x%p-0x%p @offset=%tu", + fault, end - 1, fault - start); print_section(KERN_ERR, "Padding ", pad, remainder); restore_bytes(s, "slab padding", POISON_INUSE, fault, end); From patchwork Sun Dec 1 01:49:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268195 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 75A30921 for ; Sun, 1 Dec 2019 01:49:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 392D62086A for ; Sun, 1 Dec 2019 01:49:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="TS8twS18" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 392D62086A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F266D6B026C; Sat, 30 Nov 2019 20:49:36 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EAC776B026D; Sat, 30 Nov 2019 20:49:36 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D9C216B026E; Sat, 30 Nov 2019 20:49:36 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0167.hostedemail.com [216.40.44.167]) by kanga.kvack.org (Postfix) with ESMTP id B51516B026C for ; Sat, 30 Nov 2019 20:49:36 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 77ED72A87 for ; Sun, 1 Dec 2019 01:49:36 +0000 (UTC) X-FDA: 76214890752.07.tank54_8e0f377011234 X-Spam-Summary: 2,0,0,5d9a2b3f9688308a,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:cl@linux.com:iamjoonsoo.kim@lge.com::mm-commits@vger.kernel.org:penberg@kernel.org:rientjes@google.com:torvalds@linux-foundation.org:yuzhao@google.com,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2196:2199:2393:2525:2553:2559:2563:2682:2685:2859:2895:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:5007:6261:6653:7576:8599:8957:9010:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12783:12986:13069:13311:13357:13846:14096:14181:14384:14721:14849:21080:21451:21627:21939:30054:30062:30070:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFti me:3,LUA X-HE-Tag: tank54_8e0f377011234 X-Filterd-Recvd-Size: 2822 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:36 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2C13B2084E; Sun, 1 Dec 2019 01:49:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164975; bh=aC3UXHsLJBaoXZvVZuJ5ZHBRDyl7oCuZnjX4Fv4bwp4=; h=Date:From:To:Subject:From; b=TS8twS18zGxIvxnu3lfMawKwr+7QhKbCE4+B3w0YxT6Ps2/nTmZ7AQGMiV0UrXJot YTGNLhmJ7AdJ4071mzVEMIkG4WYVsN+iwkzXVrVajgi91UIWT1OY40s4wcQF39YlvK NYeuc7MJlouchjPnKN8cvAw3GtD4X4tfj82og0oc= Date: Sat, 30 Nov 2019 17:49:34 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, cl@linux.com, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, penberg@kernel.org, rientjes@google.com, torvalds@linux-foundation.org, yuzhao@google.com Subject: [patch 009/158] mm/slub.c: update comments Message-ID: <20191201014934.ftAafosqZ%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Zhao Subject: mm/slub.c: update comments Slub doesn't use PG_active and PG_error anymore. Link: http://lkml.kernel.org/r/20191007222023.162256-1-yuzhao@google.com Signed-off-by: Yu Zhao Reviewed-by: Andrew Morton Acked-by: David Rientjes Cc: Christoph Lameter Cc: Pekka Enberg Cc: Joonsoo Kim Signed-off-by: Andrew Morton --- mm/slub.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) --- a/mm/slub.c~mm-update-comments-in-slubc +++ a/mm/slub.c @@ -93,9 +93,7 @@ * minimal so we rely on the page allocators per cpu caches for * fast frees and allocs. * - * Overloading of page flags that are otherwise used for LRU management. - * - * PageActive The slab is frozen and exempt from list processing. + * page->frozen The slab is frozen and exempt from list processing. * This means that the slab is dedicated to a purpose * such as satisfying allocations for a specific * processor. Objects may be freed in the slab while @@ -111,7 +109,7 @@ * free objects in addition to the regular freelist * that requires the slab lock. * - * PageError Slab requires special handling due to debug + * SLAB_DEBUG_FLAGS Slab requires special handling due to debug * options set. This moves slab handling out of * the fast path and disables lockless freelists. */ From patchwork Sun Dec 1 01:49:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268197 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D9827112B for ; Sun, 1 Dec 2019 01:49:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9DA042084E for ; Sun, 1 Dec 2019 01:49:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ydu5u0GK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9DA042084E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5380A6B026E; Sat, 30 Nov 2019 20:49:40 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4E8036B026F; Sat, 30 Nov 2019 20:49:40 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4271E6B0270; Sat, 30 Nov 2019 20:49:40 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0182.hostedemail.com [216.40.44.182]) by kanga.kvack.org (Postfix) with ESMTP id 2AC1B6B026E for ; Sat, 30 Nov 2019 20:49:40 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id CA164180AD817 for ; Sun, 1 Dec 2019 01:49:39 +0000 (UTC) X-FDA: 76214890878.11.ocean85_8e8a99973c22a X-Spam-Summary: 2,0,0,9739c5d83ac1b492,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:cl@linux.com:iamjoonsoo.kim@lge.com:kirill.shutemov@linux.intel.com::mm-commits@vger.kernel.org:penberg@kernel.org:penguin-kernel@i-love.sakura.ne.jp:rientjes@google.com:torvalds@linux-foundation.org:yuzhao@google.com,RULES_HIT:41:69:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:5007:6261:6653:6737:7576:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12294:12296:12297:12517:12519:12533:12555:12679:12783:12986:13161:13229:13846:14096:14181:14721:14849:21080:21451:21611:21627:21796:21939:30036:30054:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk, SPF:fp,M X-HE-Tag: ocean85_8e8a99973c22a X-Filterd-Recvd-Size: 3603 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:39 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 650B92084E; Sun, 1 Dec 2019 01:49:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164978; bh=0/7Z4HwHshKCRIhe5GE7/7YbQ7hdbshNKoBl52DZVx0=; h=Date:From:To:Subject:From; b=Ydu5u0GKfl0Z0I0RV3IIxnkwRDxjJVI+3RtOqcpD+GwvimwYU2Q/i6Cy1MVJikxQr pJv1d/HsnoISFGb7Lif/cq0IBLOQGHQ7y85wzOOiTr2OlCWOzHEKySSnLC5jlSi2hW 1+ojTY9+geH8CjNuMBDNj1X3aMU7ffu2wsWXyEUw= Date: Sat, 30 Nov 2019 17:49:37 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, cl@linux.com, iamjoonsoo.kim@lge.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, penberg@kernel.org, penguin-kernel@i-love.sakura.ne.jp, rientjes@google.com, torvalds@linux-foundation.org, yuzhao@google.com Subject: [patch 010/158] mm/slub.c: clean up validate_slab() Message-ID: <20191201014937.AUI2VWkkc%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Zhao Subject: mm/slub.c: clean up validate_slab() The function doesn't need to return any value, and the check can be done in one pass. There is a behavior change: before the patch, we stop at the first invalid free object; after the patch, we stop at the first invalid object, free or in use. This shouldn't matter because the original behavior isn't intended anyway. Link: http://lkml.kernel.org/r/20191108193958.205102-1-yuzhao@google.com Signed-off-by: Yu Zhao Acked-by: Kirill A. Shutemov Cc: Christoph Lameter Cc: Pekka Enberg Cc: David Rientjes Cc: Joonsoo Kim Cc: Tetsuo Handa Cc: Yu Zhao Signed-off-by: Andrew Morton --- mm/slub.c | 21 ++++++++------------- 1 file changed, 8 insertions(+), 13 deletions(-) --- a/mm/slub.c~mm-clean-up-validate_slab +++ a/mm/slub.c @@ -4384,31 +4384,26 @@ static int count_total(struct page *page #endif #ifdef CONFIG_SLUB_DEBUG -static int validate_slab(struct kmem_cache *s, struct page *page, +static void validate_slab(struct kmem_cache *s, struct page *page, unsigned long *map) { void *p; void *addr = page_address(page); - if (!check_slab(s, page) || - !on_freelist(s, page, NULL)) - return 0; + if (!check_slab(s, page) || !on_freelist(s, page, NULL)) + return; /* Now we know that a valid freelist exists */ bitmap_zero(map, page->objects); get_map(s, page, map); for_each_object(p, s, addr, page->objects) { - if (test_bit(slab_index(p, s, addr), map)) - if (!check_object(s, page, p, SLUB_RED_INACTIVE)) - return 0; - } + u8 val = test_bit(slab_index(p, s, addr), map) ? + SLUB_RED_INACTIVE : SLUB_RED_ACTIVE; - for_each_object(p, s, addr, page->objects) - if (!test_bit(slab_index(p, s, addr), map)) - if (!check_object(s, page, p, SLUB_RED_ACTIVE)) - return 0; - return 1; + if (!check_object(s, page, p, val)) + break; + } } static void validate_slab_slab(struct kmem_cache *s, struct page *page, From patchwork Sun Dec 1 01:49:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268199 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F1A8A112B for ; Sun, 1 Dec 2019 01:49:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BE72C2084E for ; Sun, 1 Dec 2019 01:49:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="1jpG+VGX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BE72C2084E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 696F36B0270; Sat, 30 Nov 2019 20:49:43 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6479C6B0271; Sat, 30 Nov 2019 20:49:43 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 55F436B0272; Sat, 30 Nov 2019 20:49:43 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0033.hostedemail.com [216.40.44.33]) by kanga.kvack.org (Postfix) with ESMTP id 3E98C6B0270 for ; Sat, 30 Nov 2019 20:49:43 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id E7E062C8A for ; Sun, 1 Dec 2019 01:49:42 +0000 (UTC) X-FDA: 76214891004.01.fold97_8effe18baf045 X-Spam-Summary: 2,0,0,2efe36cb6af5776e,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:axboe@kernel.dk:jack@suse.cz:khlebnikov@yandex-team.ru::mm-commits@vger.kernel.org:torvalds@linux-foundation.org:viro@zeniv.linux.org.uk,RULES_HIT:41:355:379:800:960:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2194:2199:2393:2525:2553:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:7903:8599:9025:9036:9207:9545:10004:10913:11026:11473:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12783:12986:13069:13255:13311:13357:14181:14384:14721:14849:14915:21080:21451:21627:21939:30054:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:4,LUA_SUMMARY:none X-HE-Tag: fold97_8effe18baf045 X-Filterd-Recvd-Size: 2911 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:42 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A987A2087F; Sun, 1 Dec 2019 01:49:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164982; bh=wyIuFMwdHndUacoDN2X9t1SsIpxk59Q6d1p0Nw2A270=; h=Date:From:To:Subject:From; b=1jpG+VGXuNhw8ad8DO8O+q2nadYtLGNlqoJvzI4AYWQn2Gx2NmtDQ8FJnUUaBDNrL Myo4ApS+wa1qL262lyHTWJosPHD5m+zau1CskOJFSQhdPoUwWKf6w09Rf3ymuUrNgD BDCAB0duJ+XxiqD8Cm2SQMeibE3SRMbBbBm6yib4= Date: Sat, 30 Nov 2019 17:49:41 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, axboe@kernel.dk, jack@suse.cz, khlebnikov@yandex-team.ru, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 011/158] mm/filemap.c: remove redundant cache invalidation after async direct-io write Message-ID: <20191201014941.t-tqhhEsZ%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Konstantin Khlebnikov Subject: mm/filemap.c: remove redundant cache invalidation after async direct-io write generic_file_direct_write() invalidates cache at entry. Second time this should be done when request completes. But this function calls second invalidation at exit unconditionally even for async requests. This patch skips second invalidation for async requests (-EIOCBQUEUED). Link: http://lkml.kernel.org/r/157270037850.4812.15036239021726025572.stgit@buzz Signed-off-by: Konstantin Khlebnikov Reviewed-by: Andrew Morton Reviewed-by: Jan Kara Cc: Jens Axboe Cc: Alexander Viro Signed-off-by: Andrew Morton --- mm/filemap.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) --- a/mm/filemap.c~fs-remove-redundant-cache-invalidation-after-async-direct-io-write +++ a/mm/filemap.c @@ -3218,9 +3218,11 @@ generic_file_direct_write(struct kiocb * * Most of the time we do not need this since dio_complete() will do * the invalidation for us. However there are some file systems that * do not end up with dio_complete() being called, so let's not break - * them by removing it completely + * them by removing it completely. + * + * Skip invalidation for async writes or if mapping has no pages. */ - if (mapping->nrpages) + if (written > 0 && mapping->nrpages) invalidate_inode_pages2_range(mapping, pos >> PAGE_SHIFT, end); From patchwork Sun Dec 1 01:49:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268201 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 98C52921 for ; Sun, 1 Dec 2019 01:49:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6485D2086A for ; Sun, 1 Dec 2019 01:49:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="OI9FCHDu" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6485D2086A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D4E0A6B0272; Sat, 30 Nov 2019 20:49:46 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CFED86B0273; Sat, 30 Nov 2019 20:49:46 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3DA26B0274; Sat, 30 Nov 2019 20:49:46 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0146.hostedemail.com [216.40.44.146]) by kanga.kvack.org (Postfix) with ESMTP id A8AF86B0272 for ; Sat, 30 Nov 2019 20:49:46 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 63883824999B for ; Sun, 1 Dec 2019 01:49:46 +0000 (UTC) X-FDA: 76214891172.18.whip57_8f7dbf43d0d5a X-Spam-Summary: 2,0,0,7ad30f7a6d080ebf,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:axboe@kernel.dk:jack@suse.cz:khlebnikov@yandex-team.ru::mm-commits@vger.kernel.org:torvalds@linux-foundation.org:viro@zeniv.linux.org.uk,RULES_HIT:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1535:1543:1711:1730:1747:1777:1792:1981:2194:2199:2393:2525:2553:2559:2563:2682:2685:2859:2899:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4605:5007:6121:6261:6653:7576:7875:7903:8599:9025:9207:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12114:12291:12296:12297:12438:12517:12519:12555:12679:12783:12986:13255:14181:14721:14849:14915:21080:21433:21451:21627:21939:30012:30054:30056:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:4, LUA_SUMM X-HE-Tag: whip57_8f7dbf43d0d5a X-Filterd-Recvd-Size: 5183 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:45 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C642320880; Sun, 1 Dec 2019 01:49:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164985; bh=7quBNIY5QEdm7U1wRpd8ZNFmxXwG/zO0Gad7eix4hp8=; h=Date:From:To:Subject:From; b=OI9FCHDu1Um+wfsh5wQ8bJ2I/XsedRpwaTxGwyEx9UGxgbZmfPRW/1XpEHy/H4hYv cbTbC7FNYtd5+HyR7Q0+s1Jdc9FCzakBDJ1qYGH244ds2lknKihfYJfi4KD6vBZLH0 H7khMthLb25a5ogi3JhyS2XRz4MW+yPxOWzoPAaE= Date: Sat, 30 Nov 2019 17:49:44 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, axboe@kernel.dk, jack@suse.cz, khlebnikov@yandex-team.ru, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 012/158] fs/direct-io.c: keep dio_warn_stale_pagecache() when CONFIG_BLOCK=n Message-ID: <20191201014944.cCRb-Aojv%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Konstantin Khlebnikov Subject: fs/direct-io.c: keep dio_warn_stale_pagecache() when CONFIG_BLOCK=n This helper prints warning if direct I/O write failed to invalidate cache, and set EIO at inode to warn usersapce about possible data corruption. See also commit 5a9d929d6e13 ("iomap: report collisions between directio and buffered writes to userspace"). Direct I/O is supported by non-disk filesystems, for example NFS. Thus generic code needs this even in kernel without CONFIG_BLOCK. Link: http://lkml.kernel.org/r/157270038074.4812.7980855544557488880.stgit@buzz Signed-off-by: Konstantin Khlebnikov Reviewed-by: Andrew Morton Reviewed-by: Jan Kara Cc: Jens Axboe Cc: Alexander Viro Signed-off-by: Andrew Morton --- fs/direct-io.c | 21 --------------------- include/linux/fs.h | 6 +++++- mm/filemap.c | 21 +++++++++++++++++++++ 3 files changed, 26 insertions(+), 22 deletions(-) --- a/fs/direct-io.c~fs-keep-dio_warn_stale_pagecache-when-config_block=n +++ a/fs/direct-io.c @@ -221,27 +221,6 @@ static inline struct page *dio_get_page( } /* - * Warn about a page cache invalidation failure during a direct io write. - */ -void dio_warn_stale_pagecache(struct file *filp) -{ - static DEFINE_RATELIMIT_STATE(_rs, 86400 * HZ, DEFAULT_RATELIMIT_BURST); - char pathname[128]; - struct inode *inode = file_inode(filp); - char *path; - - errseq_set(&inode->i_mapping->wb_err, -EIO); - if (__ratelimit(&_rs)) { - path = file_path(filp, pathname, sizeof(pathname)); - if (IS_ERR(path)) - path = "(unknown)"; - pr_crit("Page cache invalidation failure on direct I/O. Possible data corruption due to collision with buffered I/O!\n"); - pr_crit("File: %s PID: %d Comm: %.20s\n", path, current->pid, - current->comm); - } -} - -/* * dio_complete() - called when all DIO BIO I/O has been completed * * This drops i_dio_count, lets interested parties know that a DIO operation --- a/include/linux/fs.h~fs-keep-dio_warn_stale_pagecache-when-config_block=n +++ a/include/linux/fs.h @@ -3149,7 +3149,6 @@ enum { }; void dio_end_io(struct bio *bio); -void dio_warn_stale_pagecache(struct file *filp); ssize_t __blockdev_direct_IO(struct kiocb *iocb, struct inode *inode, struct block_device *bdev, struct iov_iter *iter, @@ -3194,6 +3193,11 @@ static inline void inode_dio_end(struct wake_up_bit(&inode->i_state, __I_DIO_WAKEUP); } +/* + * Warn about a page cache invalidation failure diring a direct I/O write. + */ +void dio_warn_stale_pagecache(struct file *filp); + extern void inode_set_flags(struct inode *inode, unsigned int flags, unsigned int mask); --- a/mm/filemap.c~fs-keep-dio_warn_stale_pagecache-when-config_block=n +++ a/mm/filemap.c @@ -3161,6 +3161,27 @@ int pagecache_write_end(struct file *fil } EXPORT_SYMBOL(pagecache_write_end); +/* + * Warn about a page cache invalidation failure during a direct I/O write. + */ +void dio_warn_stale_pagecache(struct file *filp) +{ + static DEFINE_RATELIMIT_STATE(_rs, 86400 * HZ, DEFAULT_RATELIMIT_BURST); + char pathname[128]; + struct inode *inode = file_inode(filp); + char *path; + + errseq_set(&inode->i_mapping->wb_err, -EIO); + if (__ratelimit(&_rs)) { + path = file_path(filp, pathname, sizeof(pathname)); + if (IS_ERR(path)) + path = "(unknown)"; + pr_crit("Page cache invalidation failure on direct I/O. Possible data corruption due to collision with buffered I/O!\n"); + pr_crit("File: %s PID: %d Comm: %.20s\n", path, current->pid, + current->comm); + } +} + ssize_t generic_file_direct_write(struct kiocb *iocb, struct iov_iter *from) { From patchwork Sun Dec 1 01:49:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268203 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 97578112B for ; Sun, 1 Dec 2019 01:49:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 64EAB208C3 for ; Sun, 1 Dec 2019 01:49:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="YXmdfDsi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 64EAB208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D4EF46B0274; Sat, 30 Nov 2019 20:49:49 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D27A56B0275; Sat, 30 Nov 2019 20:49:49 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C14A16B0276; Sat, 30 Nov 2019 20:49:49 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0100.hostedemail.com [216.40.44.100]) by kanga.kvack.org (Postfix) with ESMTP id A42D76B0274 for ; Sat, 30 Nov 2019 20:49:49 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 59E7C824999B for ; Sun, 1 Dec 2019 01:49:49 +0000 (UTC) X-FDA: 76214891298.03.tooth75_8febd47e18e13 X-Spam-Summary: 2,0,0,a2981d25d123f269,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:axboe@kernel.dk:jack@suse.cz:khlebnikov@yandex-team.ru::mm-commits@vger.kernel.org:torvalds@linux-foundation.org:viro@zeniv.linux.org.uk,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2194:2199:2393:2525:2553:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3867:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:7903:8599:8660:9025:9036:9207:9545:10004:10913:11026:11473:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12783:12986:13069:13148:13230:13255:13311:13357:14181:14384:14721:14849:14915:21067:21080:21451:21627:21939:30012:30054:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: tooth75_8febd47e18e13 X-Filterd-Recvd-Size: 3082 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:48 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id EABA52084E; Sun, 1 Dec 2019 01:49:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164988; bh=UAwlasgGfNqnfPY8d1A7JPTm2QF2si8jlDhqOnXo9mI=; h=Date:From:To:Subject:From; b=YXmdfDsi1tIgxzl+u9O7uh3ROoXYMn2egEfokDfZgnBtykw7XPz9yzDB6TmwY1Nmb mHdcId9ziXPyGmhXvrZO60GWR5HO9LIaS+X1n6Np494L7F1ZOiPiBGgSbVIH6BQ778 nW9TKGZ5tEEjGQmTLsFV7hD/1Euljl/j5NMKzS/o= Date: Sat, 30 Nov 2019 17:49:47 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, axboe@kernel.dk, jack@suse.cz, khlebnikov@yandex-team.ru, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 013/158] mm/filemap.c: warn if stale pagecache is left after direct write Message-ID: <20191201014947.ru2upF2br%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Konstantin Khlebnikov Subject: mm/filemap.c: warn if stale pagecache is left after direct write generic_file_direct_write() tries to invalidate pagecache after O_DIRECT write. Unlike to similar code in dio_complete() this silently ignores error returned from invalidate_inode_pages2_range(). According to comment this code here because not all filesystems call dio_complete() to do proper invalidation after O_DIRECT write. Noticeable example is a blkdev_direct_IO(). This patch calls dio_warn_stale_pagecache() if invalidation fails. Link: http://lkml.kernel.org/r/157270038294.4812.2238891109785106069.stgit@buzz Signed-off-by: Konstantin Khlebnikov Reviewed-by: Andrew Morton Reviewed-by: Jan Kara Cc: Jens Axboe Cc: Alexander Viro Signed-off-by: Andrew Morton --- mm/filemap.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) --- a/mm/filemap.c~fs-warn-if-stale-pagecache-is-left-after-direct-write +++ a/mm/filemap.c @@ -3241,11 +3241,13 @@ generic_file_direct_write(struct kiocb * * do not end up with dio_complete() being called, so let's not break * them by removing it completely. * + * Noticeable example is a blkdev_direct_IO(). + * * Skip invalidation for async writes or if mapping has no pages. */ - if (written > 0 && mapping->nrpages) - invalidate_inode_pages2_range(mapping, - pos >> PAGE_SHIFT, end); + if (written > 0 && mapping->nrpages && + invalidate_inode_pages2_range(mapping, pos >> PAGE_SHIFT, end)) + dio_warn_stale_pagecache(file); if (written > 0) { pos += written; From patchwork Sun Dec 1 01:49:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268205 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C81F6112B for ; Sun, 1 Dec 2019 01:49:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9658E2086A for ; Sun, 1 Dec 2019 01:49:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="c+2wDuLq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9658E2086A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 02FA66B0276; Sat, 30 Nov 2019 20:49:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F24E96B0277; Sat, 30 Nov 2019 20:49:52 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE99F6B0278; Sat, 30 Nov 2019 20:49:52 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0049.hostedemail.com [216.40.44.49]) by kanga.kvack.org (Postfix) with ESMTP id C14276B0276 for ; Sat, 30 Nov 2019 20:49:52 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 78EBE40C0 for ; Sun, 1 Dec 2019 01:49:52 +0000 (UTC) X-FDA: 76214891424.20.crime34_9062d5e7a151e X-Spam-Summary: 2,0,0,548079aa493183ce,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:ira.weiny@intel.com:jhubbard@nvidia.com::mm-commits@vger.kernel.org:torvalds@linux-foundation.org:vbabka@suse.cz:zhongjiang@huawei.com,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:3870:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6261:6653:7576:8531:9025:9391:9545:10004:10913:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13255:13846:14096:14181:14721:14849:21080:21212:21433:21451:21627:21819:21939:30054:30064:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: crime34_9062d5e7a151e X-Filterd-Recvd-Size: 3326 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:52 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 19AB4208E4; Sun, 1 Dec 2019 01:49:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164991; bh=7Sw3eEIus75nQFpmr1T1WuXKYQavg0WAU7PdxGnIHNI=; h=Date:From:To:Subject:From; b=c+2wDuLqXyx/oQRs9od9mJRCpZEVhlMfKjCsn7DxnhteWbT4TH6QQ+h7yFztW938b YFrYz39nO4mg4spI+YJ51kb6eHji+zS+xuatiZa6AIrg51m5zyLo6L8iLqRt32HYSg awJBCopw87sS9oHki5zJU82SoI1Wg+SRoPRHebKw= Date: Sat, 30 Nov 2019 17:49:50 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, ira.weiny@intel.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, zhongjiang@huawei.com Subject: [patch 014/158] mm/gup.c: allow CMA migration to propagate errors back to caller Message-ID: <20191201014950.8xby9TJPA%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: zhong jiang Subject: mm/gup.c: allow CMA migration to propagate errors back to caller check_and_migrate_cma_pages() was recording the result of __get_user_pages_locked() in an unsigned "nr_pages" variable. Because __get_user_pages_locked() returns a signed value that can include negative errno values, this had the effect of hiding errors. Change check_and_migrate_cma_pages() implementation so that it uses a signed variable instead, and propagates the results back to the caller just as other gup internal functions do. This was discovered with the help of unsigned_lesser_than_zero.cocci. Link: http://lkml.kernel.org/r/1571671030-58029-1-git-send-email-zhongjiang@huawei.com Signed-off-by: zhong jiang Suggested-by: John Hubbard Acked-by: Vlastimil Babka Reviewed-by: John Hubbard Reviewed-by: Ira Weiny Signed-off-by: Andrew Morton --- mm/gup.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) --- a/mm/gup.c~mm-gup-allow-cma-migration-to-propagate-errors-back-to-caller +++ a/mm/gup.c @@ -1443,6 +1443,7 @@ static long check_and_migrate_cma_pages( bool drain_allow = true; bool migrate_allow = true; LIST_HEAD(cma_page_list); + long ret = nr_pages; check_again: for (i = 0; i < nr_pages;) { @@ -1504,17 +1505,18 @@ check_again: * again migrating any new CMA pages which we failed to isolate * earlier. */ - nr_pages = __get_user_pages_locked(tsk, mm, start, nr_pages, + ret = __get_user_pages_locked(tsk, mm, start, nr_pages, pages, vmas, NULL, gup_flags); - if ((nr_pages > 0) && migrate_allow) { + if ((ret > 0) && migrate_allow) { + nr_pages = ret; drain_allow = true; goto check_again; } } - return nr_pages; + return ret; } #else static long check_and_migrate_cma_pages(struct task_struct *tsk, From patchwork Sun Dec 1 01:49:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268207 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B3343921 for ; Sun, 1 Dec 2019 01:49:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7B9EE208C3 for ; Sun, 1 Dec 2019 01:49:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="SwDju6Lh" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7B9EE208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 18F526B0278; Sat, 30 Nov 2019 20:49:56 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 119F36B0279; Sat, 30 Nov 2019 20:49:56 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EFCDB6B027A; Sat, 30 Nov 2019 20:49:55 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0035.hostedemail.com [216.40.44.35]) by kanga.kvack.org (Postfix) with ESMTP id D84E46B0278 for ; Sat, 30 Nov 2019 20:49:55 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 8D06440F1 for ; Sun, 1 Dec 2019 01:49:55 +0000 (UTC) X-FDA: 76214891550.10.edge37_90d477250a92b X-Spam-Summary: 2,0,0,0330ba0708cd697e,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:david@redhat.com:jhubbard@nvidia.com::liuxiang_1999@126.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2393:2525:2553:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3151:3355:3865:3866:3868:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4641:5007:6261:6653:7576:8660:9025:9545:10004:10226:10913:11026:11473:11658:11914:12043:12048:12114:12291:12295:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13148:13230:13255:13846:14096:14181:14721:14849:21080:21324:21433:21451:21600:21627:21819:21939:30034:30054:30064:30074:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFti me:3,LUA X-HE-Tag: edge37_90d477250a92b X-Filterd-Recvd-Size: 4289 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:55 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2B867215A5; Sun, 1 Dec 2019 01:49:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164994; bh=wPACwLULN6vw5l/eLSSEOzRjlJhLORKDVYnHUAmQ8dg=; h=Date:From:To:Subject:From; b=SwDju6Lhc1XcHkt1fQ+/e/tPAzyM6+n8lp+2AsHqIRNXhroEkjVpOv34vX6ipYWgU CdM4CkzGUeEl/dJjK7D2j8kwxrAVAL7tzelDWC40tG+0y+oHe7sD1AMuy9QzeQFKWA 4F2eRYekaokMF5x6546MvpG/5T70U7OaxZKTBC0Q= Date: Sat, 30 Nov 2019 17:49:53 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, david@redhat.com, jhubbard@nvidia.com, linux-mm@kvack.org, liuxiang_1999@126.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 015/158] mm/gup.c: fix comments of __get_user_pages() and get_user_pages_remote() Message-ID: <20191201014953.VXu3YPS4P%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Liu Xiang Subject: mm/gup.c: fix comments of __get_user_pages() and get_user_pages_remote() Fix comments of __get_user_pages() and get_user_pages_remote(), make them more clear. Link: http://lkml.kernel.org/r/1572443533-3118-1-git-send-email-liuxiang_1999@126.com Signed-off-by: Liu Xiang Suggested-by: John Hubbard Reviewed-by: David Hildenbrand Reviewed-by: John Hubbard Signed-off-by: Andrew Morton --- mm/gup.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-) --- a/mm/gup.c~mm-gup-fix-comments-of-__get_user_pages-and-get_user_pages_remote +++ a/mm/gup.c @@ -734,11 +734,17 @@ static int check_vma_flags(struct vm_are * Or NULL if the caller does not require them. * @nonblocking: whether waiting for disk IO or mmap_sem contention * - * Returns number of pages pinned. This may be fewer than the number - * requested. If nr_pages is 0 or negative, returns 0. If no pages - * were pinned, returns -errno. Each page returned must be released - * with a put_page() call when it is finished with. vmas will only - * remain valid while mmap_sem is held. + * Returns either number of pages pinned (which may be less than the + * number requested), or an error. Details about the return value: + * + * -- If nr_pages is 0, returns 0. + * -- If nr_pages is >0, but no pages were pinned, returns -errno. + * -- If nr_pages is >0, and some pages were pinned, returns the number of + * pages pinned. Again, this may be less than nr_pages. + * + * The caller is responsible for releasing returned @pages, via put_page(). + * + * @vmas are valid only as long as mmap_sem is held. * * Must be called with mmap_sem held. It may be released. See below. * @@ -1107,11 +1113,17 @@ static __always_inline long __get_user_p * subsequently whether VM_FAULT_RETRY functionality can be * utilised. Lock must initially be held. * - * Returns number of pages pinned. This may be fewer than the number - * requested. If nr_pages is 0 or negative, returns 0. If no pages - * were pinned, returns -errno. Each page returned must be released - * with a put_page() call when it is finished with. vmas will only - * remain valid while mmap_sem is held. + * Returns either number of pages pinned (which may be less than the + * number requested), or an error. Details about the return value: + * + * -- If nr_pages is 0, returns 0. + * -- If nr_pages is >0, but no pages were pinned, returns -errno. + * -- If nr_pages is >0, and some pages were pinned, returns the number of + * pages pinned. Again, this may be less than nr_pages. + * + * The caller is responsible for releasing returned @pages, via put_page(). + * + * @vmas are valid only as long as mmap_sem is held. * * Must be called with mmap_sem held for read or write. * From patchwork Sun Dec 1 01:49:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268209 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BD292112B for ; Sun, 1 Dec 2019 01:50:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7A50B215E5 for ; Sun, 1 Dec 2019 01:50:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="rZbwFGIW" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7A50B215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2528F6B027A; Sat, 30 Nov 2019 20:49:59 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 22BE46B027B; Sat, 30 Nov 2019 20:49:59 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1401E6B027C; Sat, 30 Nov 2019 20:49:59 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0252.hostedemail.com [216.40.44.252]) by kanga.kvack.org (Postfix) with ESMTP id EA1446B027A for ; Sat, 30 Nov 2019 20:49:58 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 95F3540FB for ; Sun, 1 Dec 2019 01:49:58 +0000 (UTC) X-FDA: 76214891676.13.cats60_9146b36ac3a00 X-Spam-Summary: 2,0,0,5a7616698a72441d,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hare@suse.de:hch@lst.de:hughd@google.com::mm-commits@vger.kernel.org:naohiro.aota@wdc.com:torvalds@linux-foundation.org:tytso@mit.edu:willy@infradead.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2731:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3868:3870:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6261:6653:7576:8599:9025:9545:10004:10913:11026:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12783:12986:13069:13161:13229:13311:13357:13846:14181:14384:14721:14849:21080:21451:21627:21740:21939:30034:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:6,LUA_SUMMARY:none X-HE-Tag: cats60_9146b36ac3a00 X-Filterd-Recvd-Size: 2886 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:49:58 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 48B902086A; Sun, 1 Dec 2019 01:49:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575164997; bh=A0GEc7IFh8b6AbmlxqjSNknz+r6JLlOHUayMLRhqlis=; h=Date:From:To:Subject:From; b=rZbwFGIWULkTZ6RQk+MTczDeNJWF8UCCzysKHPCA+NFB36O2PA2GZK6vw9wDIlftx P0cWJjd0TG5pdz2rdYWaj/+aud4hf4dLoTE2svcTM+yMCeJMPwLUg71G6VPy5xAtGW xyf2lCyLzWNjkHuTQU7awJRqwcF6bNUMUUUv1ZCY= Date: Sat, 30 Nov 2019 17:49:56 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hare@suse.de, hch@lst.de, hughd@google.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, naohiro.aota@wdc.com, torvalds@linux-foundation.org, tytso@mit.edu, willy@infradead.org Subject: [patch 016/158] mm, swap: disallow swapon() on zoned block devices Message-ID: <20191201014956.tXC37Qp6P%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naohiro Aota Subject: mm, swap: disallow swapon() on zoned block devices A zoned block device consists of a number of zones. Zones are either conventional and accepting random writes or sequential and requiring that writes be issued in LBA order from each zone write pointer position. For the write restriction, zoned block devices are not suitable for a swap device. Disallow swapon on them. [akpm@linux-foundation.org: reflow and reword comment, per Christoph] Link: http://lkml.kernel.org/r/20191015085814.637837-1-naohiro.aota@wdc.com Signed-off-by: Naohiro Aota Reviewed-by: Christoph Hellwig Cc: Matthew Wilcox Cc: "Theodore Y. Ts'o" Cc: Hannes Reinecke Cc: Hugh Dickins Signed-off-by: Andrew Morton --- mm/swapfile.c | 7 +++++++ 1 file changed, 7 insertions(+) --- a/mm/swapfile.c~mm-swap-disallow-swapon-on-zoned-block-devices +++ a/mm/swapfile.c @@ -2887,6 +2887,13 @@ static int claim_swapfile(struct swap_in error = set_blocksize(p->bdev, PAGE_SIZE); if (error < 0) return error; + /* + * Zoned block devices contain zones that have a sequential + * write only restriction. Hence zoned block devices are not + * suitable for swapping. Disallow them here. + */ + if (blk_queue_is_zoned(p->bdev->bd_queue)) + return -EINVAL; p->flags |= SWP_BLKDEV; } else if (S_ISREG(inode->i_mode)) { p->bdev = inode->i_sb->s_bdev; From patchwork Sun Dec 1 01:50:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268211 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D7F69921 for ; Sun, 1 Dec 2019 01:50:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A3CFA208E4 for ; Sun, 1 Dec 2019 01:50:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="N5ELWlwL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A3CFA208E4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 44CF66B027B; Sat, 30 Nov 2019 20:50:02 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3FE946B027C; Sat, 30 Nov 2019 20:50:02 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2EEB76B027D; Sat, 30 Nov 2019 20:50:02 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0030.hostedemail.com [216.40.44.30]) by kanga.kvack.org (Postfix) with ESMTP id 1824C6B027B for ; Sat, 30 Nov 2019 20:50:02 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id BD77A824999B for ; Sun, 1 Dec 2019 01:50:01 +0000 (UTC) X-FDA: 76214891802.18.note21_38c8f690032 X-Spam-Summary: 2,0,0,9255c50bcc8670d6,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:dave.hansen@intel.com:fengguang.wu@intel.com:jingqi.liu@intel.com::mm-commits@vger.kernel.org:torvalds@linux-foundation.org:vbabka@suse.cz,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2859:2902:2918:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:8599:8957:9010:9025:9545:10004:10913:11026:11232:11658:11914:12043:12048:12114:12297:12438:12517:12519:12555:12679:12783:12986:13069:13311:13357:14096:14181:14384:14721:14849:21080:21451:21627:21740:21939:30054:30064:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:22,LUA_SUMMARY:none X-HE-Tag: note21_38c8f690032 X-Filterd-Recvd-Size: 2975 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:01 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 72ED4208C3; Sun, 1 Dec 2019 01:50:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165000; bh=5x9g57RBH9Ag/aAvkMggeXZm54GrcQeY24Nh4/RaC9E=; h=Date:From:To:Subject:From; b=N5ELWlwLDZIwP2oCn6P5f/SEcy1fv1ISEZTwUxvpyDBxbA8RDDX1ZGEz0iSazHYk9 kdIoavtKra5CL8xBPKCgVg5NeUgIZEkh7T4QtwbS0CWxUdI9Dm6aFxiL3e/I769VA0 tqLtnndx22rGDO7rNsCp8QAh3IMWrQYbcGmYsyo0= Date: Sat, 30 Nov 2019 17:50:00 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, dave.hansen@intel.com, fengguang.wu@intel.com, jingqi.liu@intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 017/158] mm/swap.c: trivial mark_page_accessed() cleanup Message-ID: <20191201015000.LheRfuWNR%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Fengguang Wu Subject: mm/swap.c: trivial mark_page_accessed() cleanup This avoids duplicated PageReferenced() calls. No behavior change. Link: http://lkml.kernel.org/r/20191016225326.GB12497@wfg-t540p.sh.intel.com Signed-off-by: Fengguang Wu Reviewed-by: Andrew Morton Acked-by: Vlastimil Babka Cc: Dave Hansen Cc: Liu Jingqi Signed-off-by: Andrew Morton --- mm/swap.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) --- a/mm/swap.c~mm-trivial-mark_page_accessed-cleanup +++ a/mm/swap.c @@ -373,9 +373,16 @@ static void __lru_cache_activate_page(st void mark_page_accessed(struct page *page) { page = compound_head(page); - if (!PageActive(page) && !PageUnevictable(page) && - PageReferenced(page)) { + if (!PageReferenced(page)) { + SetPageReferenced(page); + } else if (PageUnevictable(page)) { + /* + * Unevictable pages are on the "LRU_UNEVICTABLE" list. But, + * this list is never rotated or maintained, so marking an + * evictable page accessed has no effect. + */ + } else if (!PageActive(page)) { /* * If the page is on the LRU, queue it for activation via * activate_page_pvecs. Otherwise, assume the page is on a @@ -389,8 +396,6 @@ void mark_page_accessed(struct page *pag ClearPageReferenced(page); if (page_is_file_cache(page)) workingset_activation(page); - } else if (!PageReferenced(page)) { - SetPageReferenced(page); } if (page_is_idle(page)) clear_page_idle(page); From patchwork Sun Dec 1 01:50:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268213 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 466B5921 for ; Sun, 1 Dec 2019 01:50:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 081FE2245C for ; Sun, 1 Dec 2019 01:50:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="Y4H4IYhd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 081FE2245C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9089C6B027D; Sat, 30 Nov 2019 20:50:05 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 892656B027E; Sat, 30 Nov 2019 20:50:05 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 781A76B027F; Sat, 30 Nov 2019 20:50:05 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0230.hostedemail.com [216.40.44.230]) by kanga.kvack.org (Postfix) with ESMTP id 4D3BA6B027D for ; Sat, 30 Nov 2019 20:50:05 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 03145180AD817 for ; Sun, 1 Dec 2019 01:50:05 +0000 (UTC) X-FDA: 76214891970.23.table69_aefde5f220c X-Spam-Summary: 2,0,0,4a969ebdc5c80e86,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hannes@cmpxchg.org:laoar.shao@gmail.com::mhocko@suse.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org:vdavydov.dev@gmail.com,RULES_HIT:41:355:379:800:960:967:973:981:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4605:5007:6261:6653:7514:7576:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:13846:14181:14721:14849:21080:21433:21451:21627:21740:21819:21939:30012:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:22,LUA_SUMMARY:none X-HE-Tag: table69_aefde5f220c X-Filterd-Recvd-Size: 3720 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:04 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 94F8C2086A; Sun, 1 Dec 2019 01:50:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165003; bh=jFVAizLGkSLXCcEzYLQr9RivpQZNZ+vQcGOdwNKCA/Y=; h=Date:From:To:Subject:From; b=Y4H4IYhdAjNQgQZUAGpJiphwPN9j4cuIscVLUsTsw0K1upa42RYbTo/tzqcVFzuJz DW8lSdy9hCT4b6zLm8rR0SuKM0/uyNmp2TNHwIMalzUFke3Wo4o0ZSKlc/FauhT4Cx fXFLlupA7GYxOC09IqDlImbFjggHeEHhuU5tQAgQ= Date: Sat, 30 Nov 2019 17:50:03 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hannes@cmpxchg.org, laoar.shao@gmail.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 018/158] mm, memcg: clean up reclaim iter array Message-ID: <20191201015003.iDPFULUlC%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yafang Shao Subject: mm, memcg: clean up reclaim iter array The mem_cgroup_reclaim_cookie is only used in memcg softlimit reclaim now, and the priority of the reclaim is always 0. We don't need to define the iter in struct mem_cgroup_per_node as an array any more. That could make the code more clear and save some space. Link: http://lkml.kernel.org/r/1569897728-1686-1-git-send-email-laoar.shao@gmail.com Signed-off-by: Yafang Shao Acked-by: Michal Hocko Cc: Johannes Weiner Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- include/linux/memcontrol.h | 3 +-- mm/memcontrol.c | 11 +++-------- 2 files changed, 4 insertions(+), 10 deletions(-) --- a/include/linux/memcontrol.h~mm-memcg-clean-up-reclaim-iter-array +++ a/include/linux/memcontrol.h @@ -58,7 +58,6 @@ enum mem_cgroup_protection { struct mem_cgroup_reclaim_cookie { pg_data_t *pgdat; - int priority; unsigned int generation; }; @@ -126,7 +125,7 @@ struct mem_cgroup_per_node { unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; - struct mem_cgroup_reclaim_iter iter[DEF_PRIORITY + 1]; + struct mem_cgroup_reclaim_iter iter; struct memcg_shrinker_map __rcu *shrinker_map; --- a/mm/memcontrol.c~mm-memcg-clean-up-reclaim-iter-array +++ a/mm/memcontrol.c @@ -1052,7 +1052,7 @@ struct mem_cgroup *mem_cgroup_iter(struc struct mem_cgroup_per_node *mz; mz = mem_cgroup_nodeinfo(root, reclaim->pgdat->node_id); - iter = &mz->iter[reclaim->priority]; + iter = &mz->iter; if (prev && reclaim->generation != iter->generation) goto out_unlock; @@ -1152,15 +1152,11 @@ static void __invalidate_reclaim_iterato struct mem_cgroup_reclaim_iter *iter; struct mem_cgroup_per_node *mz; int nid; - int i; for_each_node(nid) { mz = mem_cgroup_nodeinfo(from, nid); - for (i = 0; i <= DEF_PRIORITY; i++) { - iter = &mz->iter[i]; - cmpxchg(&iter->position, - dead_memcg, NULL); - } + iter = &mz->iter; + cmpxchg(&iter->position, dead_memcg, NULL); } } @@ -1705,7 +1701,6 @@ static int mem_cgroup_soft_reclaim(struc unsigned long nr_scanned; struct mem_cgroup_reclaim_cookie reclaim = { .pgdat = pgdat, - .priority = 0, }; excess = soft_limit_excess(root_memcg); From patchwork Sun Dec 1 01:50:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268215 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2F598112B for ; Sun, 1 Dec 2019 01:50:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EFB38215F1 for ; Sun, 1 Dec 2019 01:50:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="EIz0KBC8" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EFB38215F1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ADD676B027F; Sat, 30 Nov 2019 20:50:08 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A8F4C6B0280; Sat, 30 Nov 2019 20:50:08 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A55D6B0281; Sat, 30 Nov 2019 20:50:08 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0246.hostedemail.com [216.40.44.246]) by kanga.kvack.org (Postfix) with ESMTP id 800A66B027F for ; Sat, 30 Nov 2019 20:50:08 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 399C8181AEF0B for ; Sun, 1 Dec 2019 01:50:08 +0000 (UTC) X-FDA: 76214892096.15.vase83_1289503fee21 X-Spam-Summary: 2,0,0,b019047990ac312b,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org:vdavydov.dev@gmail.com,RULES_HIT:41:355:379:800:960:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3867:3870:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7514:7576:8599:9025:9545:10004:10913:11026:11658:11914:12043:12048:12297:12517:12519:12555:12679:12783:12986:13069:13311:13357:13846:14094:14181:14384:14721:14849:21080:21451:21627:21939:30012:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: vase83_1289503fee21 X-Filterd-Recvd-Size: 2311 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:07 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D1109215E5; Sun, 1 Dec 2019 01:50:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165007; bh=Ga4cwFarMb9t2jqcIiDZcPtyjyt7RrB9BmTqnuDRqCY=; h=Date:From:To:Subject:From; b=EIz0KBC86xUaAQD4xYAmfKeXHAphTl9kV5EhiONrCLe8S1INZobUmH+Tb+AIf5pHK STTKUv/GG/EN/zL83haRom9R8BGf/G8DZByVx1pIyKPxBq3o5AR8t/k5oDBl0j6lX1 mp0V1phgtwvTTCle1X1HgU6PqxX1H1vOo+WR0nVE= Date: Sat, 30 Nov 2019 17:50:06 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 019/158] mm: memcontrol: remove dead code from memory_max_write() Message-ID: <20191201015006.V5SCwvXVP%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: mm: memcontrol: remove dead code from memory_max_write() When the reclaim loop in memory_max_write() is ^C'd or similar, we set err to -EINTR. But we don't return err. Once the limit is set, we always return success (nbytes). Delete the dead code. Link: http://lkml.kernel.org/r/20191022201518.341216-1-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Acked-by: Michal Hocko Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- mm/memcontrol.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) --- a/mm/memcontrol.c~mm-memcontrol-remove-dead-code-from-memory_max_write +++ a/mm/memcontrol.c @@ -6139,10 +6139,8 @@ static ssize_t memory_max_write(struct k if (nr_pages <= max) break; - if (signal_pending(current)) { - err = -EINTR; + if (signal_pending(current)) break; - } if (!drained) { drain_all_stock(memcg); From patchwork Sun Dec 1 01:50:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268217 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 051F1112B for ; Sun, 1 Dec 2019 01:50:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C638C208C3 for ; Sun, 1 Dec 2019 01:50:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="j7iF259Q" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C638C208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B04096B0281; Sat, 30 Nov 2019 20:50:12 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AB5F66B0282; Sat, 30 Nov 2019 20:50:12 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F2056B0283; Sat, 30 Nov 2019 20:50:12 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0192.hostedemail.com [216.40.44.192]) by kanga.kvack.org (Postfix) with ESMTP id 878EB6B0281 for ; Sat, 30 Nov 2019 20:50:12 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id CD1AB180AD817 for ; Sun, 1 Dec 2019 01:50:11 +0000 (UTC) X-FDA: 76214892222.03.print17_1ab32dbe061b X-Spam-Summary: 2,0,0,4a96846aa46eab06,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org:vdavydov.dev@gmail.com,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1543:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2689:2734:2859:2898:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:5007:6119:6120:6261:6653:7514:7576:7875:7901:7903:8531:8599:8603:8660:9025:9545:10004:10913:11026:11233:11473:11658:11914:12043:12048:12291:12295:12296:12297:12438:12517:12519:12555:12664:12679:12683:12783:12986:13148:13184:13229:13230:13846:14040:14096:14181:14721:14849:21080:21451:21611:21627:21740:21795:21939:30012:30051:30054:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCa che:0,MS X-HE-Tag: print17_1ab32dbe061b X-Filterd-Recvd-Size: 4715 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:11 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2202D2231B; Sun, 1 Dec 2019 01:50:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165010; bh=5iqFBbv1M0wIcttA5X9Yld++NF+a6Oqez7X4q4ldNok=; h=Date:From:To:Subject:From; b=j7iF259QiUl7DPwWUK1sZl+0KEDa59trajwr8+zwGyNQkZI+iQYQwsyhqlup1He0W TASGCxdBdK6yIU0RXfyZACCC4BowUxixZ/u/7uYNvK1xCz+y8+BW4RA7mSAhXM2i3x PyoXkAIH1ZY8+/zia0K83jBssrpHaH41H2/rv8D4= Date: Sat, 30 Nov 2019 17:50:09 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 020/158] mm: memcontrol: try harder to set a new memory.high Message-ID: <20191201015009.wgTpjuNKo%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: mm: memcontrol: try harder to set a new memory.high Setting a memory.high limit below the usage makes almost no effort to shrink the cgroup to the new target size. While memory.high is a "soft" limit that isn't supposed to cause OOM situations, we should still try harder to meet a user request through persistent reclaim. For example, after setting a 10M memory.high on an 800M cgroup full of file cache, the usage shrinks to about 350M: + cat /cgroup/workingset/memory.current 841568256 + echo 10M + cat /cgroup/workingset/memory.current 355729408 This isn't exactly what the user would expect to happen. Setting the value a few more times eventually whittles the usage down to what we are asking for: + echo 10M + cat /cgroup/workingset/memory.current 104181760 + echo 10M + cat /cgroup/workingset/memory.current 31801344 + echo 10M + cat /cgroup/workingset/memory.current 10440704 To improve this, add reclaim retry loops to the memory.high write() callback, similar to what we do for memory.max, to make a reasonable effort that the usage meets the requested size after the call returns. Afterwards, a single write() to memory.high is enough in all but extreme cases: + cat /cgroup/workingset/memory.current 841609216 + echo 10M + cat /cgroup/workingset/memory.current 10182656 790M is not a reasonable reclaim target to ask of a single reclaim invocation. And it wouldn't be reasonable to optimize the reclaim code for it. So asking for the full size but retrying is not a bad choice here: we express our intent, and benefit if reclaim becomes better at handling larger requests, but we also acknowledge that some of the deltas we can encounter in memory_high_write() are just too ridiculously big for a single reclaim invocation to manage. Link: http://lkml.kernel.org/r/20191022201518.341216-2-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Acked-by: Michal Hocko Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- mm/memcontrol.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) --- a/mm/memcontrol.c~mm-memcontrol-try-harder-to-set-a-new-memoryhigh +++ a/mm/memcontrol.c @@ -6091,7 +6091,8 @@ static ssize_t memory_high_write(struct char *buf, size_t nbytes, loff_t off) { struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); - unsigned long nr_pages; + unsigned int nr_retries = MEM_CGROUP_RECLAIM_RETRIES; + bool drained = false; unsigned long high; int err; @@ -6102,12 +6103,29 @@ static ssize_t memory_high_write(struct memcg->high = high; - nr_pages = page_counter_read(&memcg->memory); - if (nr_pages > high) - try_to_free_mem_cgroup_pages(memcg, nr_pages - high, - GFP_KERNEL, true); + for (;;) { + unsigned long nr_pages = page_counter_read(&memcg->memory); + unsigned long reclaimed; + + if (nr_pages <= high) + break; + + if (signal_pending(current)) + break; + + if (!drained) { + drain_all_stock(memcg); + drained = true; + continue; + } + + reclaimed = try_to_free_mem_cgroup_pages(memcg, nr_pages - high, + GFP_KERNEL, true); + + if (!reclaimed && !nr_retries--) + break; + } - memcg_wb_domain_size_changed(memcg); return nbytes; } From patchwork Sun Dec 1 01:50:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268219 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C93E9921 for ; Sun, 1 Dec 2019 01:50:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8EA5420880 for ; Sun, 1 Dec 2019 01:50:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="G2QNf+Mt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8EA5420880 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 33DC96B0283; Sat, 30 Nov 2019 20:50:15 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2EC566B0284; Sat, 30 Nov 2019 20:50:15 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 203A36B0285; Sat, 30 Nov 2019 20:50:15 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0151.hostedemail.com [216.40.44.151]) by kanga.kvack.org (Postfix) with ESMTP id 06AEB6B0283 for ; Sat, 30 Nov 2019 20:50:15 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id B19F6181AEF0B for ; Sun, 1 Dec 2019 01:50:14 +0000 (UTC) X-FDA: 76214892348.15.fold89_219acedd6816 X-Spam-Summary: 2,0,0,0c5638911b6adb14,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:guro@fb.com:haolee.swjtu@gmail.com::mhocko@suse.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2892:2895:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:3870:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7514:7576:8599:9010:9025:9545:10004:10913:11026:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12696:12737:12783:12986:13069:13161:13229:13255:13311:13357:14096:14181:14384:14721:14849:21080:21451:21627:21939:21966:30054:30064:30075,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: fold89_219acedd6816 X-Filterd-Recvd-Size: 2656 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:14 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 613A92087F; Sun, 1 Dec 2019 01:50:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165013; bh=vW2C7Vq9JDU1nkWPI6hbJOUhIRFPKM0CnVDUsgT1jtQ=; h=Date:From:To:Subject:From; b=G2QNf+Mt/DzravMP2C1duKH7vrqoQrQd6vKC4aLPglCdEa37uS+KHz3JzmFC1SvJS SO4jL4ghzp+pkwHWEmbTVDZu3XlsHkRWegMbC8Yi9HnDS2HRaffyxY46MNzhH9AMhG ScjA/tjx6T7P1kUwjlCVUSGD1sKvFn0UxTBu9d0g= Date: Sat, 30 Nov 2019 17:50:12 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, guro@fb.com, haolee.swjtu@gmail.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 021/158] include/linux/memcontrol.h: fix comments based on per-node memcg Message-ID: <20191201015012.2gUYzhiAO%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Hao Lee Subject: include/linux/memcontrol.h: fix comments based on per-node memcg These comments should be updated as memcg limit enforcement has been moved from zones to nodes. Link: http://lkml.kernel.org/r/20191022150618.GA15519@haolee.github.io Signed-off-by: Hao Lee Acked-by: Roman Gushchin Acked-by: Michal Hocko Signed-off-by: Andrew Morton --- include/linux/memcontrol.h | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) --- a/include/linux/memcontrol.h~mm-fix-comments-based-on-per-node-memcg +++ a/include/linux/memcontrol.h @@ -111,7 +111,7 @@ struct memcg_shrinker_map { }; /* - * per-zone information in memory controller. + * per-node information in memory controller. */ struct mem_cgroup_per_node { struct lruvec lruvec; @@ -398,8 +398,7 @@ mem_cgroup_nodeinfo(struct mem_cgroup *m * @memcg: memcg of the wanted lruvec * * Returns the lru list vector holding pages for a given @node or a given - * @memcg and @zone. This can be the node lruvec, if the memory controller - * is disabled. + * @memcg. This can be the node lruvec, if the memory controller is disabled. */ static inline struct lruvec *mem_cgroup_lruvec(struct pglist_data *pgdat, struct mem_cgroup *memcg) From patchwork Sun Dec 1 01:50:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268221 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 508F3921 for ; Sun, 1 Dec 2019 01:50:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0EC5A215A5 for ; Sun, 1 Dec 2019 01:50:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="BVidYVpD" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0EC5A215A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D27A36B0006; Sat, 30 Nov 2019 20:50:18 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CD7B56B0284; Sat, 30 Nov 2019 20:50:18 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C147F6B0286; Sat, 30 Nov 2019 20:50:18 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0009.hostedemail.com [216.40.44.9]) by kanga.kvack.org (Postfix) with ESMTP id A79CE6B0006 for ; Sat, 30 Nov 2019 20:50:18 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 4B93B180AD817 for ; Sun, 1 Dec 2019 01:50:18 +0000 (UTC) X-FDA: 76214892516.14.tent34_29bdd6a2994a X-Spam-Summary: 2,0,0,991f6d1aaffdafb0,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:gthelen@google.com:guro@fb.com:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:shakeelb@google.com:torvalds@linux-foundation.org,RULES_HIT:2:41:69:355:379:617:800:960:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1535:1605:1730:1747:1777:1792:2196:2198:2199:2200:2380:2393:2525:2553:2559:2564:2682:2685:2693:2731:2859:2892:2895:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4051:4120:4250:4321:4385:4605:5007:6119:6261:6653:7576:7875:7903:8599:9010:9025:9121:9545:9592:10004:10913:11026:11232:11473:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13846:14877:21080:21222:21451:21627:21740:21795:21939:30045:30051:30054:30064:30070:30075:30080:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Ne tcheck:n X-HE-Tag: tent34_29bdd6a2994a X-Filterd-Recvd-Size: 9620 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:17 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id BA21F21774; Sun, 1 Dec 2019 01:50:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165017; bh=JiOaVEnrYcuXnrFvJmUnLtWxs10xkr03DufaU3JIW1s=; h=Date:From:To:Subject:From; b=BVidYVpDgcPgLqILvvTvzeO3Tb+9r7IRK0Y2ddMI3w76g7vk0zlzm/zjXl4OQ7ny0 W6Zf7mL4IQfo5Pq9ii4wi17ec3iBHivdxkS2Z14D8RTdkViND/KfheIG3nZDcjxDog SHH8/lSPVXClHFMTLXdJoWMQtKltdVOsWvD3qIv0= Date: Sat, 30 Nov 2019 17:50:16 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, gthelen@google.com, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 022/158] mm: vmscan: memcontrol: remove mem_cgroup_select_victim_node() Message-ID: <20191201015016.dy6En-Ltj%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Shakeel Butt Subject: mm: vmscan: memcontrol: remove mem_cgroup_select_victim_node() Since commit 1ba6fc9af35b ("mm: vmscan: do not share cgroup iteration between reclaimers"), the memcg reclaim does not bail out earlier based on sc->nr_reclaimed and will traverse all the nodes. All the reclaimable pages of the memcg on all the nodes will be scanned relative to the reclaim priority. So, there is no need to maintain state regarding which node to start the memcg reclaim from. This patch effectively reverts the commit 889976dbcb12 ("memcg: reclaim memory from nodes in round-robin order") and commit 453a9bf347f1 ("memcg: fix numa scan information update to be triggered by memory event"). [shakeelb@google.com: v2] Link: http://lkml.kernel.org/r/20191030204232.139424-1-shakeelb@google.com Link: http://lkml.kernel.org/r/20191029234753.224143-1-shakeelb@google.com Signed-off-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Michal Hocko Acked-by: Johannes Weiner Cc: Greg Thelen Signed-off-by: Andrew Morton --- include/linux/memcontrol.h | 8 -- mm/memcontrol.c | 112 ----------------------------------- mm/vmscan.c | 14 +--- 3 files changed, 5 insertions(+), 129 deletions(-) --- a/include/linux/memcontrol.h~mm-vmscan-memcontrol-remove-mem_cgroup_select_victim_node +++ a/include/linux/memcontrol.h @@ -80,7 +80,6 @@ struct mem_cgroup_id { enum mem_cgroup_events_target { MEM_CGROUP_TARGET_THRESH, MEM_CGROUP_TARGET_SOFTLIMIT, - MEM_CGROUP_TARGET_NUMAINFO, MEM_CGROUP_NTARGETS, }; @@ -312,13 +311,6 @@ struct mem_cgroup { struct list_head kmem_caches; #endif - int last_scanned_node; -#if MAX_NUMNODES > 1 - nodemask_t scan_nodes; - atomic_t numainfo_events; - atomic_t numainfo_updating; -#endif - #ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list; struct wb_domain cgwb_domain; --- a/mm/memcontrol.c~mm-vmscan-memcontrol-remove-mem_cgroup_select_victim_node +++ a/mm/memcontrol.c @@ -108,7 +108,6 @@ static const char *const mem_cgroup_lru_ #define THRESHOLDS_EVENTS_TARGET 128 #define SOFTLIMIT_EVENTS_TARGET 1024 -#define NUMAINFO_EVENTS_TARGET 1024 /* * Cgroups above their limits are maintained in a RB-Tree, independent of @@ -877,9 +876,6 @@ static bool mem_cgroup_event_ratelimit(s case MEM_CGROUP_TARGET_SOFTLIMIT: next = val + SOFTLIMIT_EVENTS_TARGET; break; - case MEM_CGROUP_TARGET_NUMAINFO: - next = val + NUMAINFO_EVENTS_TARGET; - break; default: break; } @@ -899,21 +895,12 @@ static void memcg_check_events(struct me if (unlikely(mem_cgroup_event_ratelimit(memcg, MEM_CGROUP_TARGET_THRESH))) { bool do_softlimit; - bool do_numainfo __maybe_unused; do_softlimit = mem_cgroup_event_ratelimit(memcg, MEM_CGROUP_TARGET_SOFTLIMIT); -#if MAX_NUMNODES > 1 - do_numainfo = mem_cgroup_event_ratelimit(memcg, - MEM_CGROUP_TARGET_NUMAINFO); -#endif mem_cgroup_threshold(memcg); if (unlikely(do_softlimit)) mem_cgroup_update_tree(memcg, page); -#if MAX_NUMNODES > 1 - if (unlikely(do_numainfo)) - atomic_inc(&memcg->numainfo_events); -#endif } } @@ -1591,104 +1578,6 @@ static bool mem_cgroup_out_of_memory(str return ret; } -#if MAX_NUMNODES > 1 - -/** - * test_mem_cgroup_node_reclaimable - * @memcg: the target memcg - * @nid: the node ID to be checked. - * @noswap : specify true here if the user wants flle only information. - * - * This function returns whether the specified memcg contains any - * reclaimable pages on a node. Returns true if there are any reclaimable - * pages in the node. - */ -static bool test_mem_cgroup_node_reclaimable(struct mem_cgroup *memcg, - int nid, bool noswap) -{ - struct lruvec *lruvec = mem_cgroup_lruvec(NODE_DATA(nid), memcg); - - if (lruvec_page_state(lruvec, NR_INACTIVE_FILE) || - lruvec_page_state(lruvec, NR_ACTIVE_FILE)) - return true; - if (noswap || !total_swap_pages) - return false; - if (lruvec_page_state(lruvec, NR_INACTIVE_ANON) || - lruvec_page_state(lruvec, NR_ACTIVE_ANON)) - return true; - return false; - -} - -/* - * Always updating the nodemask is not very good - even if we have an empty - * list or the wrong list here, we can start from some node and traverse all - * nodes based on the zonelist. So update the list loosely once per 10 secs. - * - */ -static void mem_cgroup_may_update_nodemask(struct mem_cgroup *memcg) -{ - int nid; - /* - * numainfo_events > 0 means there was at least NUMAINFO_EVENTS_TARGET - * pagein/pageout changes since the last update. - */ - if (!atomic_read(&memcg->numainfo_events)) - return; - if (atomic_inc_return(&memcg->numainfo_updating) > 1) - return; - - /* make a nodemask where this memcg uses memory from */ - memcg->scan_nodes = node_states[N_MEMORY]; - - for_each_node_mask(nid, node_states[N_MEMORY]) { - - if (!test_mem_cgroup_node_reclaimable(memcg, nid, false)) - node_clear(nid, memcg->scan_nodes); - } - - atomic_set(&memcg->numainfo_events, 0); - atomic_set(&memcg->numainfo_updating, 0); -} - -/* - * Selecting a node where we start reclaim from. Because what we need is just - * reducing usage counter, start from anywhere is O,K. Considering - * memory reclaim from current node, there are pros. and cons. - * - * Freeing memory from current node means freeing memory from a node which - * we'll use or we've used. So, it may make LRU bad. And if several threads - * hit limits, it will see a contention on a node. But freeing from remote - * node means more costs for memory reclaim because of memory latency. - * - * Now, we use round-robin. Better algorithm is welcomed. - */ -int mem_cgroup_select_victim_node(struct mem_cgroup *memcg) -{ - int node; - - mem_cgroup_may_update_nodemask(memcg); - node = memcg->last_scanned_node; - - node = next_node_in(node, memcg->scan_nodes); - /* - * mem_cgroup_may_update_nodemask might have seen no reclaimmable pages - * last time it really checked all the LRUs due to rate limiting. - * Fallback to the current node in that case for simplicity. - */ - if (unlikely(node == MAX_NUMNODES)) - node = numa_node_id(); - - memcg->last_scanned_node = node; - return node; -} -#else -int mem_cgroup_select_victim_node(struct mem_cgroup *memcg) -{ - return 0; -} -#endif - static int mem_cgroup_soft_reclaim(struct mem_cgroup *root_memcg, pg_data_t *pgdat, gfp_t gfp_mask, @@ -5073,7 +4962,6 @@ static struct mem_cgroup *mem_cgroup_all goto fail; INIT_WORK(&memcg->high_work, high_work_func); - memcg->last_scanned_node = MAX_NUMNODES; INIT_LIST_HEAD(&memcg->oom_notify); mutex_init(&memcg->thresholds_lock); spin_lock_init(&memcg->move_lock); --- a/mm/vmscan.c~mm-vmscan-memcontrol-remove-mem_cgroup_select_victim_node +++ a/mm/vmscan.c @@ -3348,10 +3348,8 @@ unsigned long try_to_free_mem_cgroup_pag gfp_t gfp_mask, bool may_swap) { - struct zonelist *zonelist; unsigned long nr_reclaimed; unsigned long pflags; - int nid; unsigned int noreclaim_flag; struct scan_control sc = { .nr_to_reclaim = max(nr_pages, SWAP_CLUSTER_MAX), @@ -3364,16 +3362,14 @@ unsigned long try_to_free_mem_cgroup_pag .may_unmap = 1, .may_swap = may_swap, }; - - set_task_reclaim_state(current, &sc.reclaim_state); /* - * Unlike direct reclaim via alloc_pages(), memcg's reclaim doesn't - * take care of from where we get pages. So the node where we start the - * scan does not need to be the current node. + * Traverse the ZONELIST_FALLBACK zonelist of the current node to put + * equal pressure on all the nodes. This is based on the assumption that + * the reclaim does not bail out early. */ - nid = mem_cgroup_select_victim_node(memcg); + struct zonelist *zonelist = node_zonelist(numa_node_id(), sc.gfp_mask); - zonelist = &NODE_DATA(nid)->node_zonelists[ZONELIST_FALLBACK]; + set_task_reclaim_state(current, &sc.reclaim_state); trace_mm_vmscan_memcg_reclaim_begin(0, sc.gfp_mask); From patchwork Sun Dec 1 01:50:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268223 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6CE8A112B for ; Sun, 1 Dec 2019 01:50:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 38C5D20880 for ; Sun, 1 Dec 2019 01:50:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="kqxS6kGs" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 38C5D20880 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0CB406B0007; Sat, 30 Nov 2019 20:50:22 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0A33C6B0284; Sat, 30 Nov 2019 20:50:22 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EFB2F6B0287; Sat, 30 Nov 2019 20:50:21 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0027.hostedemail.com [216.40.44.27]) by kanga.kvack.org (Postfix) with ESMTP id D6ACD6B0007 for ; Sat, 30 Nov 2019 20:50:21 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 7F9502C7C for ; Sun, 1 Dec 2019 01:50:21 +0000 (UTC) X-FDA: 76214892642.01.jewel10_318d9356a91e X-Spam-Summary: 2,0,0,d44c32e9c3abf5c5,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:chris@chrisdown.name:guro@fb.com:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:tj@kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:877:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2553:2559:2564:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4416:5007:6119:6120:6261:6653:7576:7875:7901:7903:8599:8784:9010:9025:9545:10004:10913:11658:11914:12043:12048:12297:12517:12519:12555:12679:12696:12737:12783:12986:13069:13149:13230:13255:13311:13357:13846:13869:14181:14384:14721:14849:21080:21451:21627:21939:30005:30034:30045:30054:30064:30066:30070:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DN SBL:neut X-HE-Tag: jewel10_318d9356a91e X-Filterd-Recvd-Size: 3113 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:21 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 022CB2084E; Sun, 1 Dec 2019 01:50:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165020; bh=PUF9phQWCxMRf+GLoQtVw0sC+myP+JJecwJoTf8yPiM=; h=Date:From:To:Subject:From; b=kqxS6kGs1L0lUgx1rIUFNPD9Jn3RTAeoTrjaFhu0rFWA0DwZbuE/HGHwlprhV16Zb jgu5r7HYUJFMv6Xk2wYt67G2Cajv+IDZHDmNzxBRk0uxjhzf30N/5resO4DrvC/uX9 apwdpfySWGO3Tse4hV7YuGkbS52JpDjhvhMc0d80= Date: Sat, 30 Nov 2019 17:50:19 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, chris@chrisdown.name, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, tj@kernel.org, torvalds@linux-foundation.org Subject: [patch 023/158] Documentation/admin-guide/cgroup-v2.rst: document why inactive_X + active_X may not equal X Message-ID: <20191201015019.FwTkdHMCh%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Chris Down Subject: Documentation/admin-guide/cgroup-v2.rst: document why inactive_X + active_X may not equal X This has confused a significant number of people using cgroups inside Facebook, and some of those outside as well judging by posts like this[0] (although it's not a problem unique to cgroup v2). If shmem handling in particular becomes more coherent at some point in the future -- although that seems unlikely now -- we can change the wording here. [0]: https://unix.stackexchange.com/q/525092/10762 Link: http://lkml.kernel.org/r/20191111144958.GA11914@chrisdown.name Signed-off-by: Chris Down Acked-by: Michal Hocko Cc: Johannes Weiner Cc: Tejun Heo Cc: Roman Gushchin Signed-off-by: Andrew Morton --- Documentation/admin-guide/cgroup-v2.rst | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) --- a/Documentation/admin-guide/cgroup-v2.rst~docs-cgroup-mm-document-why-inactive_x-active_x-may-not-equal-x +++ a/Documentation/admin-guide/cgroup-v2.rst @@ -1288,7 +1288,12 @@ PAGE_SIZE multiple when read back. inactive_anon, active_anon, inactive_file, active_file, unevictable Amount of memory, swap-backed and filesystem-backed, on the internal memory management lists used by the - page reclaim algorithm + page reclaim algorithm. + + As these represent internal list state (eg. shmem pages are on anon + memory management lists), inactive_foo + active_foo may not be equal to + the value for the foo counter, since the foo counter is type-based, not + list-based. slab_reclaimable Part of "slab" that might be reclaimed, such as From patchwork Sun Dec 1 01:50:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268225 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 241E8112B for ; Sun, 1 Dec 2019 01:50:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D8ACE20880 for ; Sun, 1 Dec 2019 01:50:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="U6Y2c2ph" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D8ACE20880 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A8E4C6B0284; Sat, 30 Nov 2019 20:50:25 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A17F86B0288; Sat, 30 Nov 2019 20:50:25 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 956626B0289; Sat, 30 Nov 2019 20:50:25 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0093.hostedemail.com [216.40.44.93]) by kanga.kvack.org (Postfix) with ESMTP id 7EA9E6B0284 for ; Sat, 30 Nov 2019 20:50:25 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 2469D2C7C for ; Sun, 1 Dec 2019 01:50:25 +0000 (UTC) X-FDA: 76214892810.04.boys05_39d043f18a49 X-Spam-Summary: 2,0,0,6041c52d889dffb1,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hannes@cmpxchg.org:hdanton@sina.com:hughd@google.com:josef@toxicpanda.com:kirill.shutemov@linux.intel.com::mm-commits@vger.kernel.org:torvalds@linux-foundation.org:willy@infradead.org,RULES_HIT:2:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1535:1605:1730:1747:1777:1792:2194:2199:2393:2525:2553:2559:2563:2682:2685:2689:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3151:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4049:4118:4250:4321:5007:6119:6261:6653:7576:7875:8599:9025:9545:9592:10004:10226:10913:11026:11473:11658:11914:12043:12048:12114:12291:12296:12297:12438:12517:12519:12555:12679:12683:12696:12737:12783:12986:13161:13229:13255:13846:14096:21080:21212:21324:21451:21600:21627:21939:30045:30054:30070:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,Dom ainCache X-HE-Tag: boys05_39d043f18a49 X-Filterd-Recvd-Size: 7713 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:24 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 6C8AA215F1; Sun, 1 Dec 2019 01:50:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165024; bh=NHzO46Gf37rIo80vMiJbO57r2FcV1wsRyqQ0gFtC+tE=; h=Date:From:To:Subject:From; b=U6Y2c2phtI+9lq/in8gn3LZXQqPBA3ZE4AzfLhnno+iH9OHsp9k6TUqHp2UQMKhKn 5nW6wkiFgPaJ2UMrBiSKFENqSaVhPvFvxIJ8YtvjVb73EFYa1/rxV4toSfJ1GlQKTg D58c+Zq1epXOxURLPM0CDIyyZq4XQPTUdbIbAmrs= Date: Sat, 30 Nov 2019 17:50:22 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hannes@cmpxchg.org, hdanton@sina.com, hughd@google.com, josef@toxicpanda.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 024/158] mm: drop mmap_sem before calling balance_dirty_pages() in write fault Message-ID: <20191201015022.dGmFTLZi2%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: mm: drop mmap_sem before calling balance_dirty_pages() in write fault One of our services is observing hanging ps/top/etc under heavy write IO, and the task states show this is an mmap_sem priority inversion: A write fault is holding the mmap_sem in read-mode and waiting for (heavily cgroup-limited) IO in balance_dirty_pages(): [<0>] balance_dirty_pages+0x724/0x905 [<0>] balance_dirty_pages_ratelimited+0x254/0x390 [<0>] fault_dirty_shared_page.isra.96+0x4a/0x90 [<0>] do_wp_page+0x33e/0x400 [<0>] __handle_mm_fault+0x6f0/0xfa0 [<0>] handle_mm_fault+0xe4/0x200 [<0>] __do_page_fault+0x22b/0x4a0 [<0>] page_fault+0x45/0x50 [<0>] 0xffffffffffffffff Somebody tries to change the address space, contending for the mmap_sem in write-mode: [<0>] call_rwsem_down_write_failed_killable+0x13/0x20 [<0>] do_mprotect_pkey+0xa8/0x330 [<0>] SyS_mprotect+0xf/0x20 [<0>] do_syscall_64+0x5b/0x100 [<0>] entry_SYSCALL_64_after_hwframe+0x3d/0xa2 [<0>] 0xffffffffffffffff The waiting writer locks out all subsequent readers to avoid lock starvation, and several threads can be seen hanging like this: [<0>] call_rwsem_down_read_failed+0x14/0x30 [<0>] proc_pid_cmdline_read+0xa0/0x480 [<0>] __vfs_read+0x23/0x140 [<0>] vfs_read+0x87/0x130 [<0>] SyS_read+0x42/0x90 [<0>] do_syscall_64+0x5b/0x100 [<0>] entry_SYSCALL_64_after_hwframe+0x3d/0xa2 [<0>] 0xffffffffffffffff To fix this, do what we do for cache read faults already: drop the mmap_sem before calling into anything IO bound, in this case the balance_dirty_pages() function, and return VM_FAULT_RETRY. Link: http://lkml.kernel.org/r/20190924194238.GA29030@cmpxchg.org Signed-off-by: Johannes Weiner Reviewed-by: Matthew Wilcox (Oracle) Acked-by: Kirill A. Shutemov Cc: Josef Bacik Cc: Hillf Danton Cc: Hugh Dickins Signed-off-by: Andrew Morton --- mm/filemap.c | 21 --------------------- mm/internal.h | 21 +++++++++++++++++++++ mm/memory.c | 38 +++++++++++++++++++++++++++----------- 3 files changed, 48 insertions(+), 32 deletions(-) --- a/mm/filemap.c~mm-drop-mmap_sem-before-calling-balance_dirty_pages-in-write-fault +++ a/mm/filemap.c @@ -2329,27 +2329,6 @@ EXPORT_SYMBOL(generic_file_read_iter); #ifdef CONFIG_MMU #define MMAP_LOTSAMISS (100) -static struct file *maybe_unlock_mmap_for_io(struct vm_fault *vmf, - struct file *fpin) -{ - int flags = vmf->flags; - - if (fpin) - return fpin; - - /* - * FAULT_FLAG_RETRY_NOWAIT means we don't want to wait on page locks or - * anything, so we only pin the file and drop the mmap_sem if only - * FAULT_FLAG_ALLOW_RETRY is set. - */ - if ((flags & (FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_RETRY_NOWAIT)) == - FAULT_FLAG_ALLOW_RETRY) { - fpin = get_file(vmf->vma->vm_file); - up_read(&vmf->vma->vm_mm->mmap_sem); - } - return fpin; -} - /* * lock_page_maybe_drop_mmap - lock the page, possibly dropping the mmap_sem * @vmf - the vm_fault for this fault. --- a/mm/internal.h~mm-drop-mmap_sem-before-calling-balance_dirty_pages-in-write-fault +++ a/mm/internal.h @@ -362,6 +362,27 @@ vma_address(struct page *page, struct vm return max(start, vma->vm_start); } +static inline struct file *maybe_unlock_mmap_for_io(struct vm_fault *vmf, + struct file *fpin) +{ + int flags = vmf->flags; + + if (fpin) + return fpin; + + /* + * FAULT_FLAG_RETRY_NOWAIT means we don't want to wait on page locks or + * anything, so we only pin the file and drop the mmap_sem if only + * FAULT_FLAG_ALLOW_RETRY is set. + */ + if ((flags & (FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_RETRY_NOWAIT)) == + FAULT_FLAG_ALLOW_RETRY) { + fpin = get_file(vmf->vma->vm_file); + up_read(&vmf->vma->vm_mm->mmap_sem); + } + return fpin; +} + #else /* !CONFIG_MMU */ static inline void clear_page_mlock(struct page *page) { } static inline void mlock_vma_page(struct page *page) { } --- a/mm/memory.c~mm-drop-mmap_sem-before-calling-balance_dirty_pages-in-write-fault +++ a/mm/memory.c @@ -2289,10 +2289,11 @@ static vm_fault_t do_page_mkwrite(struct * * The function expects the page to be locked and unlocks it. */ -static void fault_dirty_shared_page(struct vm_area_struct *vma, - struct page *page) +static vm_fault_t fault_dirty_shared_page(struct vm_fault *vmf) { + struct vm_area_struct *vma = vmf->vma; struct address_space *mapping; + struct page *page = vmf->page; bool dirtied; bool page_mkwrite = vma->vm_ops && vma->vm_ops->page_mkwrite; @@ -2307,16 +2308,30 @@ static void fault_dirty_shared_page(stru mapping = page_rmapping(page); unlock_page(page); + if (!page_mkwrite) + file_update_time(vma->vm_file); + + /* + * Throttle page dirtying rate down to writeback speed. + * + * mapping may be NULL here because some device drivers do not + * set page.mapping but still dirty their pages + * + * Drop the mmap_sem before waiting on IO, if we can. The file + * is pinning the mapping, as per above. + */ if ((dirtied || page_mkwrite) && mapping) { - /* - * Some device drivers do not set page.mapping - * but still dirty their pages - */ + struct file *fpin; + + fpin = maybe_unlock_mmap_for_io(vmf, NULL); balance_dirty_pages_ratelimited(mapping); + if (fpin) { + fput(fpin); + return VM_FAULT_RETRY; + } } - if (!page_mkwrite) - file_update_time(vma->vm_file); + return 0; } /* @@ -2571,6 +2586,7 @@ static vm_fault_t wp_page_shared(struct __releases(vmf->ptl) { struct vm_area_struct *vma = vmf->vma; + vm_fault_t ret = VM_FAULT_WRITE; get_page(vmf->page); @@ -2594,10 +2610,10 @@ static vm_fault_t wp_page_shared(struct wp_page_reuse(vmf); lock_page(vmf->page); } - fault_dirty_shared_page(vma, vmf->page); + ret |= fault_dirty_shared_page(vmf); put_page(vmf->page); - return VM_FAULT_WRITE; + return ret; } /* @@ -3641,7 +3657,7 @@ static vm_fault_t do_shared_fault(struct return ret; } - fault_dirty_shared_page(vma, vmf->page); + ret |= fault_dirty_shared_page(vmf); return ret; } From patchwork Sun Dec 1 01:50:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268227 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 53616921 for ; Sun, 1 Dec 2019 01:50:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1F0EF20880 for ; Sun, 1 Dec 2019 01:50:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="lErKpmLn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1F0EF20880 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E868F6B0288; Sat, 30 Nov 2019 20:50:28 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E10C16B028A; Sat, 30 Nov 2019 20:50:28 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CFEF56B028B; Sat, 30 Nov 2019 20:50:28 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0229.hostedemail.com [216.40.44.229]) by kanga.kvack.org (Postfix) with ESMTP id B6AD56B0288 for ; Sat, 30 Nov 2019 20:50:28 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 6964F2C8A for ; Sun, 1 Dec 2019 01:50:28 +0000 (UTC) X-FDA: 76214892936.23.fall52_41776fd16f1f X-Spam-Summary: 2,0,0,681871a11b86faed,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hannes@cmpxchg.org:hdanton@sina.com:hughd@google.com:josef@toxicpanda.com:kirill.shutemov@linux.intel.com::mm-commits@vger.kernel.org:torvalds@linux-foundation.org:willy@infradead.org,RULES_HIT:41:355:379:800:960:966:967:973:982:988:989:1260:1263:1345:1381:1431:1437:1534:1543:1711:1730:1747:1777:1792:2194:2196:2199:2200:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3151:3353:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:5007:6238:6261:6653:7576:7903:8599:8603:8660:9025:9163:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12740:12783:12895:12986:13148:13221:13229:13230:13870:14096:14181:14721:14849:21080:21324:21451:21600:21611:21627:21939:30029:30054:30056,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netch eck:none X-HE-Tag: fall52_41776fd16f1f X-Filterd-Recvd-Size: 4978 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:27 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0021A215A5; Sun, 1 Dec 2019 01:50:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165027; bh=AQOCa69yfz4567KAk4TyDRBGeVGEjGK0FkKtr02EMkQ=; h=Date:From:To:Subject:From; b=lErKpmLn57Tj5rWruo4FBGGXeFEUeuY9c9k7FNRclAXPtuc6WRjZeMtgWhbX0U3Tl WGzFpEYEkJsaOlEBkmbmyP3vuNjReBEc/1PNjObgRvSTl4o7CFxpy/+6XBNIQEYA8V mKVg8lsbe5fdCWybKOKJrXJtnWjb/ZCP47uz/wlI= Date: Sat, 30 Nov 2019 17:50:26 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hannes@cmpxchg.org, hdanton@sina.com, hughd@google.com, josef@toxicpanda.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 025/158] shmem: pin the file in shmem_fault() if mmap_sem is dropped Message-ID: <20191201015026.h8Xjo-Lbi%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Kirill A. Shutemov" Subject: shmem: pin the file in shmem_fault() if mmap_sem is dropped syzbot found the following crash: BUG: KASAN: use-after-free in perf_trace_lock_acquire+0x401/0x530 include/trace/events/lock.h:13 Read of size 8 at addr ffff8880a5cf2c50 by task syz-executor.0/26173 CPU: 0 PID: 26173 Comm: syz-executor.0 Not tainted 5.3.0-rc6 #146 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x172/0x1f0 lib/dump_stack.c:113 print_address_description.cold+0xd4/0x306 mm/kasan/report.c:351 __kasan_report.cold+0x1b/0x36 mm/kasan/report.c:482 kasan_report+0x12/0x17 mm/kasan/common.c:618 __asan_report_load8_noabort+0x14/0x20 mm/kasan/generic_report.c:132 perf_trace_lock_acquire+0x401/0x530 include/trace/events/lock.h:13 trace_lock_acquire include/trace/events/lock.h:13 [inline] lock_acquire+0x2de/0x410 kernel/locking/lockdep.c:4411 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline] _raw_spin_lock+0x2f/0x40 kernel/locking/spinlock.c:151 spin_lock include/linux/spinlock.h:338 [inline] shmem_fault+0x5ec/0x7b0 mm/shmem.c:2034 __do_fault+0x111/0x540 mm/memory.c:3083 do_shared_fault mm/memory.c:3535 [inline] do_fault mm/memory.c:3613 [inline] handle_pte_fault mm/memory.c:3840 [inline] __handle_mm_fault+0x2adf/0x3f20 mm/memory.c:3964 handle_mm_fault+0x1b5/0x6b0 mm/memory.c:4001 do_user_addr_fault arch/x86/mm/fault.c:1441 [inline] __do_page_fault+0x536/0xdd0 arch/x86/mm/fault.c:1506 do_page_fault+0x38/0x590 arch/x86/mm/fault.c:1530 page_fault+0x39/0x40 arch/x86/entry/entry_64.S:1202 It happens if the VMA got unmapped under us while we dropped mmap_sem and inode got freed. Pinning the file if we drop mmap_sem fixes the issue. Link: http://lkml.kernel.org/r/20190927083908.rhifa4mmaxefc24r@box Signed-off-by: Kirill A. Shutemov Reported-by: syzbot+03ee87124ee05af991bd@syzkaller.appspotmail.com Acked-by: Johannes Weiner Reviewed-by: Matthew Wilcox (Oracle) Cc: Hillf Danton Cc: Hugh Dickins Cc: Josef Bacik Signed-off-by: Andrew Morton --- mm/shmem.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) --- a/mm/shmem.c~shmem-pin-the-file-in-shmem_fault-if-mmap_sem-is-dropped +++ a/mm/shmem.c @@ -2022,16 +2022,14 @@ static vm_fault_t shmem_fault(struct vm_ shmem_falloc->waitq && vmf->pgoff >= shmem_falloc->start && vmf->pgoff < shmem_falloc->next) { + struct file *fpin; wait_queue_head_t *shmem_falloc_waitq; DEFINE_WAIT_FUNC(shmem_fault_wait, synchronous_wake_function); ret = VM_FAULT_NOPAGE; - if ((vmf->flags & FAULT_FLAG_ALLOW_RETRY) && - !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) { - /* It's polite to up mmap_sem if we can */ - up_read(&vma->vm_mm->mmap_sem); + fpin = maybe_unlock_mmap_for_io(vmf, NULL); + if (fpin) ret = VM_FAULT_RETRY; - } shmem_falloc_waitq = shmem_falloc->waitq; prepare_to_wait(shmem_falloc_waitq, &shmem_fault_wait, @@ -2049,6 +2047,9 @@ static vm_fault_t shmem_fault(struct vm_ spin_lock(&inode->i_lock); finish_wait(shmem_falloc_waitq, &shmem_fault_wait); spin_unlock(&inode->i_lock); + + if (fpin) + fput(fpin); return ret; } spin_unlock(&inode->i_lock); From patchwork Sun Dec 1 01:50:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268229 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 04162112B for ; Sun, 1 Dec 2019 01:50:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B69AD208C3 for ; Sun, 1 Dec 2019 01:50:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ns4G2WGi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B69AD208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 955F86B028A; Sat, 30 Nov 2019 20:50:32 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 906B96B028C; Sat, 30 Nov 2019 20:50:32 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 843DE6B028D; Sat, 30 Nov 2019 20:50:32 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0187.hostedemail.com [216.40.44.187]) by kanga.kvack.org (Postfix) with ESMTP id 6C4A96B028A for ; Sat, 30 Nov 2019 20:50:32 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 15A9C824999B for ; Sun, 1 Dec 2019 01:50:32 +0000 (UTC) X-FDA: 76214893104.21.chess76_49f0aa138557 X-Spam-Summary: 2,0,0,d829d4edbd26dbde,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:aneesh.kumar@linux.ibm.com:carmenjackson@google.com:dan.j.williams@intel.com:dancol@google.com:jglisse@redhat.com:joel@joelfernandes.org::mayankgupta@google.com:mhocko@suse.com:minchan@kernel.org:mm-commits@vger.kernel.org:rcampbell@nvidia.com:rostedt@goodmis.org:timmurray@google.com:torvalds@linux-foundation.org:vbabka@suse.cz:willy@infradead.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1434:1437:1535:1544:1711:1730:1747:1777:1792:2198:2199:2393:2525:2553:2559:2564:2682:2685:2859:2895:2897:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4117:4250:4321:4605:5007:6119:6120:6261:6653:6737:7576:7875:7901:7903:8599:8603:8957:9025:9545:10004:10913:11026:11232:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:1 2555:126 X-HE-Tag: chess76_49f0aa138557 X-Filterd-Recvd-Size: 6495 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:31 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 6D70120880; Sun, 1 Dec 2019 01:50:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165031; bh=130sR5n/8Kc+PG7rKwPor4U4sF0H6vS4B/ziG5/DT6I=; h=Date:From:To:Subject:From; b=Ns4G2WGihlGNjcrnzwp85mDsutUN7mlcEeUEhX13oEBiL5FCjZv+7d9dn9rRISGS2 d/Pr2jukMGX5RgE5lF2/J8o0DwKVQmFHUb8LQzzHIqpAKHjNxrmqT6fkeaxoqmLHMF TMVX9p5JpDpGJ03NnMBFFfGFdSVLnMfQ97Uu3zfc= Date: Sat, 30 Nov 2019 17:50:30 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, carmenjackson@google.com, dan.j.williams@intel.com, dancol@google.com, jglisse@redhat.com, joel@joelfernandes.org, linux-mm@kvack.org, mayankgupta@google.com, mhocko@suse.com, minchan@kernel.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, rostedt@goodmis.org, timmurray@google.com, torvalds@linux-foundation.org, vbabka@suse.cz, willy@infradead.org Subject: [patch 026/158] mm: emit tracepoint when RSS changes Message-ID: <20191201015030.MR-ux4mV1%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Joel Fernandes (Google)" Subject: mm: emit tracepoint when RSS changes Useful to track how RSS is changing per TGID to detect spikes in RSS and memory hogs. Several Android teams have been using this patch in various kernel trees for half a year now. Many reported to me it is really useful so I'm posting it upstream. Initial patch developed by Tim Murray. Changes I made from original patch: o Prevent any additional space consumed by mm_struct. Regarding the fact that the RSS may change too often thus flooding the traces - note that, there is some "hysterisis" with this already. That is - We update the counter only if we receive 64 page faults due to SPLIT_RSS_ACCOUNTING. However, during zapping or copying of pte range, the RSS is updated immediately which can become noisy/flooding. In a previous discussion, we agreed that BPF or ftrace can be used to rate limit the signal if this becomes an issue. Also note that I added wrappers to trace_rss_stat to prevent compiler errors where linux/mm.h is included from tracing code, causing errors such as: CC kernel/trace/power-traces.o In file included from ./include/trace/define_trace.h:102, from ./include/trace/events/kmem.h:342, from ./include/linux/mm.h:31, from ./include/linux/ring_buffer.h:5, from ./include/linux/trace_events.h:6, from ./include/trace/events/power.h:12, from kernel/trace/power-traces.c:15: ./include/trace/trace_events.h:113:22: error: field `ent' has incomplete type struct trace_entry ent; \ Link: http://lore.kernel.org/r/20190903200905.198642-1-joel@joelfernandes.org Link: http://lkml.kernel.org/r/20191001172817.234886-1-joel@joelfernandes.org Co-developed-by: Tim Murray Signed-off-by: Tim Murray Signed-off-by: Joel Fernandes (Google) Acked-by: Michal Hocko Cc: Carmen Jackson Cc: Mayank Gupta Cc: Daniel Colascione Cc: Steven Rostedt (VMware) Cc: Minchan Kim Cc: "Aneesh Kumar K.V" Cc: Dan Williams Cc: Jerome Glisse Cc: Matthew Wilcox Cc: Ralph Campbell Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- include/linux/mm.h | 14 +++++++++++--- include/trace/events/kmem.h | 21 +++++++++++++++++++++ mm/memory.c | 6 ++++++ 3 files changed, 38 insertions(+), 3 deletions(-) --- a/include/linux/mm.h~mm-emit-tracepoint-when-rss-changes +++ a/include/linux/mm.h @@ -1643,19 +1643,27 @@ static inline unsigned long get_mm_count return (unsigned long)val; } +void mm_trace_rss_stat(int member, long count); + static inline void add_mm_counter(struct mm_struct *mm, int member, long value) { - atomic_long_add(value, &mm->rss_stat.count[member]); + long count = atomic_long_add_return(value, &mm->rss_stat.count[member]); + + mm_trace_rss_stat(member, count); } static inline void inc_mm_counter(struct mm_struct *mm, int member) { - atomic_long_inc(&mm->rss_stat.count[member]); + long count = atomic_long_inc_return(&mm->rss_stat.count[member]); + + mm_trace_rss_stat(member, count); } static inline void dec_mm_counter(struct mm_struct *mm, int member) { - atomic_long_dec(&mm->rss_stat.count[member]); + long count = atomic_long_dec_return(&mm->rss_stat.count[member]); + + mm_trace_rss_stat(member, count); } /* Optimized variant when page is already known not to be PageAnon */ --- a/include/trace/events/kmem.h~mm-emit-tracepoint-when-rss-changes +++ a/include/trace/events/kmem.h @@ -316,6 +316,27 @@ TRACE_EVENT(mm_page_alloc_extfrag, __entry->change_ownership) ); +TRACE_EVENT(rss_stat, + + TP_PROTO(int member, + long count), + + TP_ARGS(member, count), + + TP_STRUCT__entry( + __field(int, member) + __field(long, size) + ), + + TP_fast_assign( + __entry->member = member; + __entry->size = (count << PAGE_SHIFT); + ), + + TP_printk("member=%d size=%ldB", + __entry->member, + __entry->size) + ); #endif /* _TRACE_KMEM_H */ /* This part must be outside protection */ --- a/mm/memory.c~mm-emit-tracepoint-when-rss-changes +++ a/mm/memory.c @@ -72,6 +72,8 @@ #include #include +#include + #include #include #include @@ -152,6 +154,10 @@ static int __init init_zero_pfn(void) } core_initcall(init_zero_pfn); +void mm_trace_rss_stat(int member, long count) +{ + trace_rss_stat(member, count); +} #if defined(SPLIT_RSS_COUNTING) From patchwork Sun Dec 1 01:50:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268231 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B667D112B for ; Sun, 1 Dec 2019 01:50:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 76A6E217AB for ; Sun, 1 Dec 2019 01:50:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="Cnq+ZFLS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 76A6E217AB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6E5B86B028C; Sat, 30 Nov 2019 20:50:36 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 696366B028E; Sat, 30 Nov 2019 20:50:36 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5AC966B028F; Sat, 30 Nov 2019 20:50:36 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0197.hostedemail.com [216.40.44.197]) by kanga.kvack.org (Postfix) with ESMTP id 42EF36B028C for ; Sat, 30 Nov 2019 20:50:36 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id E399640DD for ; Sun, 1 Dec 2019 01:50:35 +0000 (UTC) X-FDA: 76214893230.08.lamp77_52ca8aeb6309 X-Spam-Summary: 2,0,0,607bcf8ecae270ef,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:aneesh.kumar@linux.ibm.com:carmenjackson@google.com:dan.j.williams@intel.com:dancol@google.com:ilkos@google.com:jglisse@redhat.com:joel@joelfernandes.org::mayankgupta@google.com:mhocko@suse.com:minchan@kernel.org:mm-commits@vger.kernel.org:pmladek@suse.com:rcampbell@nvidia.com:rostedt@goodmis.org:timmurray@google.com:torvalds@linux-foundation.org:vbabka@suse.cz:willy@infradead.org,RULES_HIT:2:41:69:355:379:800:960:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1535:1605:1730:1747:1777:1792:2198:2199:2393:2525:2559:2565:2682:2685:2859:2892:2895:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4050:4120:4250:4321:4605:5007:6119:6120:6261:6653:6737:7576:7875:7901:7903:8568:8599:8603:8660:8957:9025:9545:10004:10913:11026:11232:11473:11658:11914:12043:12048:1229 1:12296: X-HE-Tag: lamp77_52ca8aeb6309 X-Filterd-Recvd-Size: 9369 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:35 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2669C208C3; Sun, 1 Dec 2019 01:50:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165034; bh=y7ntZQuT7q0/JIh2SYpCHrQiJDOQzIO0HD5EFHs+i6U=; h=Date:From:To:Subject:From; b=Cnq+ZFLS5bCQm69f5FPVezdckITKSZVQ8nWSlQih8PtTECSn/nmRVdyP5CHbVyQgC 17spmkjJFxYRtLCXnBDxpKFmWH8GMyZGfkzWgCHta45kuG6STwnT+ZF6hBxtgcRb7V FnQCP1/SPrPqIPBsQmZBvQGrYpShyN3f5afg6D4c= Date: Sat, 30 Nov 2019 17:50:33 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, carmenjackson@google.com, dan.j.williams@intel.com, dancol@google.com, ilkos@google.com, jglisse@redhat.com, joel@joelfernandes.org, linux-mm@kvack.org, mayankgupta@google.com, mhocko@suse.com, minchan@kernel.org, mm-commits@vger.kernel.org, pmladek@suse.com, rcampbell@nvidia.com, rostedt@goodmis.org, timmurray@google.com, torvalds@linux-foundation.org, vbabka@suse.cz, willy@infradead.org Subject: [patch 027/158] rss_stat: add support to detect RSS updates of external mm Message-ID: <20191201015033.5BEWpolbT%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Joel Fernandes (Google)" Subject: rss_stat: add support to detect RSS updates of external mm When a process updates the RSS of a different process, the rss_stat tracepoint appears in the context of the process doing the update. This can confuse userspace that the RSS of process doing the update is updated, while in reality a different process's RSS was updated. This issue happens in reclaim paths such as with direct reclaim or background reclaim. This patch adds more information to the tracepoint about whether the mm being updated belongs to the current process's context (curr field). We also include a hash of the mm pointer so that the process who the mm belongs to can be uniquely identified (mm_id field). Also vsprintf.c is refactored a bit to allow reuse of hashing code. [akpm@linux-foundation.org: remove unused local `str'] [joelaf@google.com: inline call to ptr_to_hashval] Link: http://lore.kernel.org/r/20191113153816.14b95acd@gandalf.local.home Link: http://lkml.kernel.org/r/20191114164622.GC233237@google.com Link: http://lkml.kernel.org/r/20191106024452.81923-1-joel@joelfernandes.org Signed-off-by: Joel Fernandes (Google) Reported-by: Ioannis Ilkos Acked-by: Petr Mladek [lib/vsprintf.c] Cc: Tim Murray Cc: Michal Hocko Cc: Carmen Jackson Cc: Mayank Gupta Cc: Daniel Colascione Cc: Steven Rostedt (VMware) Cc: Minchan Kim Cc: "Aneesh Kumar K.V" Cc: Dan Williams Cc: Jerome Glisse Cc: Matthew Wilcox Cc: Ralph Campbell Cc: Vlastimil Babka Cc: Steven Rostedt Signed-off-by: Andrew Morton --- include/linux/mm.h | 8 +++--- include/linux/string.h | 2 + include/trace/events/kmem.h | 32 ++++++++++++++++++++++++--- lib/vsprintf.c | 40 ++++++++++++++++++++++++---------- mm/memory.c | 4 +-- 5 files changed, 66 insertions(+), 20 deletions(-) --- a/include/linux/mm.h~rss_stat-add-support-to-detect-rss-updates-of-external-mm +++ a/include/linux/mm.h @@ -1643,27 +1643,27 @@ static inline unsigned long get_mm_count return (unsigned long)val; } -void mm_trace_rss_stat(int member, long count); +void mm_trace_rss_stat(struct mm_struct *mm, int member, long count); static inline void add_mm_counter(struct mm_struct *mm, int member, long value) { long count = atomic_long_add_return(value, &mm->rss_stat.count[member]); - mm_trace_rss_stat(member, count); + mm_trace_rss_stat(mm, member, count); } static inline void inc_mm_counter(struct mm_struct *mm, int member) { long count = atomic_long_inc_return(&mm->rss_stat.count[member]); - mm_trace_rss_stat(member, count); + mm_trace_rss_stat(mm, member, count); } static inline void dec_mm_counter(struct mm_struct *mm, int member) { long count = atomic_long_dec_return(&mm->rss_stat.count[member]); - mm_trace_rss_stat(member, count); + mm_trace_rss_stat(mm, member, count); } /* Optimized variant when page is already known not to be PageAnon */ --- a/include/linux/string.h~rss_stat-add-support-to-detect-rss-updates-of-external-mm +++ a/include/linux/string.h @@ -216,6 +216,8 @@ int bprintf(u32 *bin_buf, size_t size, c extern ssize_t memory_read_from_buffer(void *to, size_t count, loff_t *ppos, const void *from, size_t available); +int ptr_to_hashval(const void *ptr, unsigned long *hashval_out); + /** * strstarts - does @str start with @prefix? * @str: string to examine --- a/include/trace/events/kmem.h~rss_stat-add-support-to-detect-rss-updates-of-external-mm +++ a/include/trace/events/kmem.h @@ -316,24 +316,50 @@ TRACE_EVENT(mm_page_alloc_extfrag, __entry->change_ownership) ); +/* + * Required for uniquely and securely identifying mm in rss_stat tracepoint. + */ +#ifndef __PTR_TO_HASHVAL +static unsigned int __maybe_unused mm_ptr_to_hash(const void *ptr) +{ + int ret; + unsigned long hashval; + + ret = ptr_to_hashval(ptr, &hashval); + if (ret) + return 0; + + /* The hashed value is only 32-bit */ + return (unsigned int)hashval; +} +#define __PTR_TO_HASHVAL +#endif + TRACE_EVENT(rss_stat, - TP_PROTO(int member, + TP_PROTO(struct mm_struct *mm, + int member, long count), - TP_ARGS(member, count), + TP_ARGS(mm, member, count), TP_STRUCT__entry( + __field(unsigned int, mm_id) + __field(unsigned int, curr) __field(int, member) __field(long, size) ), TP_fast_assign( + __entry->mm_id = mm_ptr_to_hash(mm); + __entry->curr = !!(current->mm == mm); __entry->member = member; __entry->size = (count << PAGE_SHIFT); ), - TP_printk("member=%d size=%ldB", + TP_printk("mm_id=%u curr=%d member=%d size=%ldB", + __entry->mm_id, + __entry->curr, __entry->member, __entry->size) ); --- a/lib/vsprintf.c~rss_stat-add-support-to-detect-rss-updates-of-external-mm +++ a/lib/vsprintf.c @@ -761,11 +761,38 @@ static int __init initialize_ptr_random( early_initcall(initialize_ptr_random); /* Maps a pointer to a 32 bit unique identifier. */ +static inline int __ptr_to_hashval(const void *ptr, unsigned long *hashval_out) +{ + unsigned long hashval; + + if (static_branch_unlikely(¬_filled_random_ptr_key)) + return -EAGAIN; + +#ifdef CONFIG_64BIT + hashval = (unsigned long)siphash_1u64((u64)ptr, &ptr_key); + /* + * Mask off the first 32 bits, this makes explicit that we have + * modified the address (and 32 bits is plenty for a unique ID). + */ + hashval = hashval & 0xffffffff; +#else + hashval = (unsigned long)siphash_1u32((u32)ptr, &ptr_key); +#endif + *hashval_out = hashval; + return 0; +} + +int ptr_to_hashval(const void *ptr, unsigned long *hashval_out) +{ + return __ptr_to_hashval(ptr, hashval_out); +} + static char *ptr_to_id(char *buf, char *end, const void *ptr, struct printf_spec spec) { const char *str = sizeof(ptr) == 8 ? "(____ptrval____)" : "(ptrval)"; unsigned long hashval; + int ret; /* When debugging early boot use non-cryptographically secure hash. */ if (unlikely(debug_boot_weak_hash)) { @@ -773,22 +800,13 @@ static char *ptr_to_id(char *buf, char * return pointer_string(buf, end, (const void *)hashval, spec); } - if (static_branch_unlikely(¬_filled_random_ptr_key)) { + ret = __ptr_to_hashval(ptr, &hashval); + if (ret) { spec.field_width = 2 * sizeof(ptr); /* string length must be less than default_width */ return error_string(buf, end, str, spec); } -#ifdef CONFIG_64BIT - hashval = (unsigned long)siphash_1u64((u64)ptr, &ptr_key); - /* - * Mask off the first 32 bits, this makes explicit that we have - * modified the address (and 32 bits is plenty for a unique ID). - */ - hashval = hashval & 0xffffffff; -#else - hashval = (unsigned long)siphash_1u32((u32)ptr, &ptr_key); -#endif return pointer_string(buf, end, (const void *)hashval, spec); } --- a/mm/memory.c~rss_stat-add-support-to-detect-rss-updates-of-external-mm +++ a/mm/memory.c @@ -154,9 +154,9 @@ static int __init init_zero_pfn(void) } core_initcall(init_zero_pfn); -void mm_trace_rss_stat(int member, long count) +void mm_trace_rss_stat(struct mm_struct *mm, int member, long count) { - trace_rss_stat(member, count); + trace_rss_stat(mm, member, count); } #if defined(SPLIT_RSS_COUNTING) From patchwork Sun Dec 1 01:50:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268233 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C2BF2921 for ; Sun, 1 Dec 2019 01:50:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8498C2086A for ; Sun, 1 Dec 2019 01:50:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="D1AAJC2g" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8498C2086A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5AE5E6B028E; Sat, 30 Nov 2019 20:50:39 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 535D66B0290; Sat, 30 Nov 2019 20:50:39 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 42EC76B0291; Sat, 30 Nov 2019 20:50:39 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0178.hostedemail.com [216.40.44.178]) by kanga.kvack.org (Postfix) with ESMTP id 266BE6B028E for ; Sat, 30 Nov 2019 20:50:39 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id D9990181AEF0B for ; Sun, 1 Dec 2019 01:50:38 +0000 (UTC) X-FDA: 76214893356.10.hate97_59e89d6e0f3c X-Spam-Summary: 2,0,0,251e8eb50180363f,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org::mm-commits@vger.kernel.org:richardw.yang@linux.intel.com:torvalds@linux-foundation.org:vbabka@suse.cz:yang.shi@linux.alibaba.com,RULES_HIT:41:355:379:800:960:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3867:3870:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:8599:9025:9545:10004:10913:11026:11232:11473:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12783:12986:13069:13161:13229:13311:13357:14181:14384:14721:14849:21060:21080:21451:21627:21796:21939:30036:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: hate97_59e89d6e0f3c X-Filterd-Recvd-Size: 2575 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:38 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9E90B2464B; Sun, 1 Dec 2019 01:50:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165037; bh=ui8MrM2IQniacgnUJpA99LS3ki48Gr76dWDU96o1gpw=; h=Date:From:To:Subject:From; b=D1AAJC2gEj0rXI0cuqtKfyeYJr+mF+9ovLCVgxdQWjzYdDx1WNR+o5nJHLdp53S75 q8zy3LkNS4dsfye1yo9kcuVtv1clDuR6J+xcLviFudv2zzwP5wUS75LXbSBchgC4k2 xCHrqPUnLJm1uj11SnTXVOHSYEP24c/jN5ux/G2s= Date: Sat, 30 Nov 2019 17:50:37 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, richardw.yang@linux.intel.com, torvalds@linux-foundation.org, vbabka@suse.cz, yang.shi@linux.alibaba.com Subject: [patch 028/158] mm/mmap.c: remove a never-triggered warning in __vma_adjust() Message-ID: <20191201015037.xWc6F2ZFZ%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wei Yang Subject: mm/mmap.c: remove a never-triggered warning in __vma_adjust() The upper level of "if" makes sure (end >= next->vm_end), which means there are only two possibilities: 1) end == next->vm_end 2) end > next->vm_end remove_next is assigned to be (1 + end > next->vm_end). This means if remove_next is 1, end must equal to next->vm_end. The VM_WARN_ON will never trigger. Link: http://lkml.kernel.org/r/20190912063126.13250-1-richardw.yang@linux.intel.com Signed-off-by: Wei Yang Reviewed-by: Andrew Morton Cc: Vlastimil Babka Cc: Yang Shi Signed-off-by: Andrew Morton --- mm/mmap.c | 2 -- 1 file changed, 2 deletions(-) --- a/mm/mmap.c~mm-mmapc-remove-a-never-trigger-warning-in-__vma_adjust +++ a/mm/mmap.c @@ -769,8 +769,6 @@ int __vma_adjust(struct vm_area_struct * remove_next = 1 + (end > next->vm_end); VM_WARN_ON(remove_next == 2 && end != next->vm_next->vm_end); - VM_WARN_ON(remove_next == 1 && - end != next->vm_end); /* trim end to next, for case 6 first pass */ end = next->vm_end; } From patchwork Sun Dec 1 01:50:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268235 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CEA0F921 for ; Sun, 1 Dec 2019 01:50:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9AE902086A for ; Sun, 1 Dec 2019 01:50:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="tLxGAL04" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9AE902086A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9300B6B0290; Sat, 30 Nov 2019 20:50:42 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8E0AD6B0292; Sat, 30 Nov 2019 20:50:42 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F5FE6B0293; Sat, 30 Nov 2019 20:50:42 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0175.hostedemail.com [216.40.44.175]) by kanga.kvack.org (Postfix) with ESMTP id 666AD6B0290 for ; Sat, 30 Nov 2019 20:50:42 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 166DF181AEF10 for ; Sun, 1 Dec 2019 01:50:42 +0000 (UTC) X-FDA: 76214893524.22.war30_611dac271d5f X-Spam-Summary: 2,0,0,cea492b76236a477,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:khlebnikov@yandex-team.ru::mhocko@kernel.org:mm-commits@vger.kernel.org:torvalds@linux-foundation.org:willy@infradead.org,RULES_HIT:41:355:379:421:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1543:1711:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3165:3355:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4041:4321:4385:5007:6261:6653:7576:7903:8599:8660:9025:9207:9545:10004:10913:11026:11473:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12783:12986:13148:13161:13229:13230:13868:14096:14181:14721:14849:14915:21080:21212:21324:21451:21627:21740:21795:21939:30051:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custo m_rules: X-HE-Tag: war30_611dac271d5f X-Filterd-Recvd-Size: 4550 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:41 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id BC1F6215E5; Sun, 1 Dec 2019 01:50:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165041; bh=inL8AaU8URjjH+ziLF8n2QsFZRt6cCXPrM0YthOq7b4=; h=Date:From:To:Subject:From; b=tLxGAL04HRjP9I7SrVa60cTUUixT3genthbX5e81U5HVxOH/MI44rmYk0dAfCpYnu n6eed/dPzFSFmcAwQA3Pn9vwTaU9Z8nMyK3TZP0Y9ap2TArIf31vTytqrmyqJOOnLt BaICstVKYA4Z/pWc0OoMT453KWgrebVjBf/uG6CI= Date: Sat, 30 Nov 2019 17:50:40 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, khlebnikov@yandex-team.ru, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 029/158] mm/swap.c: piggyback lru_add_drain_all() calls Message-ID: <20191201015040.dGbXkKv8r%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Konstantin Khlebnikov Subject: mm/swap.c: piggyback lru_add_drain_all() calls This is a very slow operation. Right now POSIX_FADV_DONTNEED is the top user because it has to freeze page references when removing it from the cache. invalidate_bdev() calls it for the same reason. Both are triggered from userspace, so it's easy to generate a storm. mlock/mlockall no longer calls lru_add_drain_all - I've seen here serious slowdown on older kernels. There are some less obvious paths in memory migration/CMA/offlining which shouldn't call frequently. The worst case requires a non-trivial workload because lru_add_drain_all() skips cpus where vectors are empty. Something must constantly generate a flow of pages for each cpu. Also cpus must be busy to make scheduling per-cpu works slower. And the machine must be big enough (64+ cpus in our case). In our case that was a massive series of mlock calls in map-reduce while other tasks write logs (and generates flows of new pages in per-cpu vectors). Mlock calls were serialized by mutex and accumulated latency up to 10 seconds or more. The kernel does not call lru_add_drain_all on mlock paths since 4.15, but the same scenario could be triggered by fadvise(POSIX_FADV_DONTNEED) or any other remaining user. There is no reason to do the drain again if somebody else already drained all the per-cpu vectors while we waited for the lock. Piggyback on a drain starting and finishing while we wait for the lock: all pages pending at the time of our entry were drained from the vectors. Callers like POSIX_FADV_DONTNEED retry their operations once after draining per-cpu vectors when pages have unexpected references. Link: http://lkml.kernel.org/r/157019456205.3142.3369423180908482020.stgit@buzz Signed-off-by: Konstantin Khlebnikov Reviewed-by: Andrew Morton Cc: Michal Hocko Cc: Matthew Wilcox Signed-off-by: Andrew Morton --- mm/swap.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) --- a/mm/swap.c~mm-swap-piggyback-lru_add_drain_all-calls +++ a/mm/swap.c @@ -713,9 +713,10 @@ static void lru_add_drain_per_cpu(struct */ void lru_add_drain_all(void) { + static seqcount_t seqcount = SEQCNT_ZERO(seqcount); static DEFINE_MUTEX(lock); static struct cpumask has_work; - int cpu; + int cpu, seq; /* * Make sure nobody triggers this path before mm_percpu_wq is fully @@ -724,7 +725,19 @@ void lru_add_drain_all(void) if (WARN_ON(!mm_percpu_wq)) return; + seq = raw_read_seqcount_latch(&seqcount); + mutex_lock(&lock); + + /* + * Piggyback on drain started and finished while we waited for lock: + * all pages pended at the time of our enter were drained from vectors. + */ + if (__read_seqcount_retry(&seqcount, seq)) + goto done; + + raw_write_seqcount_latch(&seqcount); + cpumask_clear(&has_work); for_each_online_cpu(cpu) { @@ -745,6 +758,7 @@ void lru_add_drain_all(void) for_each_cpu(cpu, &has_work) flush_work(&per_cpu(lru_add_drain_work, cpu)); +done: mutex_unlock(&lock); } #else From patchwork Sun Dec 1 01:50:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268237 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 11E49921 for ; Sun, 1 Dec 2019 01:50:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C7273208E4 for ; Sun, 1 Dec 2019 01:50:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="A6d9++za" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C7273208E4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B79616B0292; Sat, 30 Nov 2019 20:50:45 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B28EE6B0294; Sat, 30 Nov 2019 20:50:45 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A668E6B0295; Sat, 30 Nov 2019 20:50:45 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0056.hostedemail.com [216.40.44.56]) by kanga.kvack.org (Postfix) with ESMTP id 8E3DB6B0292 for ; Sat, 30 Nov 2019 20:50:45 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 4D20D180AD817 for ; Sun, 1 Dec 2019 01:50:45 +0000 (UTC) X-FDA: 76214893650.13.spoon18_68f064652928 X-Spam-Summary: 2,0,0,6b296b6472f42619,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hch@infradead.org::mgorman@techsingularity.net:mm-commits@vger.kernel.org:osalvador@suse.de:richardw.yang@linux.intel.com:torvalds@linux-foundation.org:vbabka@suse.cz:willy@infradead.org,RULES_HIT:41:69:355:379:800:960:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2859:2901:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6119:6261:6653:7576:8599:9010:9025:9121:9545:10004:10913:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13161:13172:13229:14096:14181:14721:14849:21080:21451:21627:21790:21939:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0 ,LFtime: X-HE-Tag: spoon18_68f064652928 X-Filterd-Recvd-Size: 3895 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:44 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E28E72086A; Sun, 1 Dec 2019 01:50:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165044; bh=tRkNnM4/yqTlTEfqmxUfUHlryVaKPkSHBrI726c/yAE=; h=Date:From:To:Subject:From; b=A6d9++zauhm51fcmUa0IELIy88+CDL4lyR8+TYfjTwf5Tn6xxCF2u5OA7UFK8Qsje LaulgDTAXKh4CY5WYAZdEeJQ6KVmI1mJ0mCVvS8J0eUMXUOKjhUT91MyisdeqwZ967 94/s5hD0VptGeK3Tljq/CTF7CvVqGQn0l4LGD/LE= Date: Sat, 30 Nov 2019 17:50:43 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hch@infradead.org, linux-mm@kvack.org, mgorman@techsingularity.net, mm-commits@vger.kernel.org, osalvador@suse.de, richardw.yang@linux.intel.com, torvalds@linux-foundation.org, vbabka@suse.cz, willy@infradead.org Subject: [patch 030/158] mm/mmap.c: prev could be retrieved from vma->vm_prev Message-ID: <20191201015043.vxwibc4lm%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wei Yang Subject: mm/mmap.c: prev could be retrieved from vma->vm_prev Currently __vma_unlink_common handles two cases: * has_prev * or not When has_prev is false, it is obvious prev is calculated from vma->vm_prev in __vma_unlink_common. When has_prev is true, the prev is passed through from __vma_unlink_prev in __vma_adjust for non-case 8. And at the beginning next is calculated from vma->vm_next, which implies vma is next->vm_prev. The above statement sounds a little complicated, while to think in another point of view, no matter whether vma and next is swapped, the mmap link list still preserves its property. It is proper to access vma->vm_prev. Link: http://lkml.kernel.org/r/20191006012636.31521-1-richardw.yang@linux.intel.com Signed-off-by: Wei Yang Cc: Mel Gorman Cc: Vlastimil Babka Cc: Oscar Salvador Cc: Christoph Hellwig Cc: Matthew Wilcox (Oracle) Signed-off-by: Andrew Morton --- mm/mmap.c | 20 +++++++------------- 1 file changed, 7 insertions(+), 13 deletions(-) --- a/mm/mmap.c~mm-mmapc-prev-could-be-retrieved-from-vma-vm_prev +++ a/mm/mmap.c @@ -684,23 +684,17 @@ static void __insert_vm_struct(struct mm static __always_inline void __vma_unlink_common(struct mm_struct *mm, struct vm_area_struct *vma, - struct vm_area_struct *prev, - bool has_prev, struct vm_area_struct *ignore) { - struct vm_area_struct *next; + struct vm_area_struct *prev, *next; vma_rb_erase_ignore(vma, &mm->mm_rb, ignore); next = vma->vm_next; - if (has_prev) + prev = vma->vm_prev; + if (prev) prev->vm_next = next; - else { - prev = vma->vm_prev; - if (prev) - prev->vm_next = next; - else - mm->mmap = next; - } + else + mm->mmap = next; if (next) next->vm_prev = prev; @@ -712,7 +706,7 @@ static inline void __vma_unlink_prev(str struct vm_area_struct *vma, struct vm_area_struct *prev) { - __vma_unlink_common(mm, vma, prev, true, vma); + __vma_unlink_common(mm, vma, vma); } /* @@ -898,7 +892,7 @@ again: * "next" (which is stored in post-swap() * "vma"). */ - __vma_unlink_common(mm, next, NULL, false, vma); + __vma_unlink_common(mm, next, vma); if (file) __remove_shared_vm_struct(next, file, mapping); } else if (insert) { From patchwork Sun Dec 1 01:50:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268239 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4719A921 for ; Sun, 1 Dec 2019 01:50:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 133AA2086A for ; Sun, 1 Dec 2019 01:50:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="N+CZlBWK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 133AA2086A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DC2E86B0294; Sat, 30 Nov 2019 20:50:48 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D73426B0296; Sat, 30 Nov 2019 20:50:48 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C88DD6B0297; Sat, 30 Nov 2019 20:50:48 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0184.hostedemail.com [216.40.44.184]) by kanga.kvack.org (Postfix) with ESMTP id AF04A6B0294 for ; Sat, 30 Nov 2019 20:50:48 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 658E8180AD817 for ; Sun, 1 Dec 2019 01:50:48 +0000 (UTC) X-FDA: 76214893776.25.elbow50_701eec72f300 X-Spam-Summary: 2,0,0,b1ac14644b531569,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hch@infradead.org::mgorman@techsingularity.net:mm-commits@vger.kernel.org:osalvador@suse.de:richardw.yang@linux.intel.com:torvalds@linux-foundation.org:vbabka@suse.cz:willy@infradead.org,RULES_HIT:41:355:379:800:960:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3867:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:8599:9025:9545:9592:10004:10913:11026:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12783:12986:13069:13161:13229:13311:13357:14096:14181:14384:14721:14849:21063:21080:21451:21611:21627:21939:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFti me:3,LUA X-HE-Tag: elbow50_701eec72f300 X-Filterd-Recvd-Size: 2867 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:48 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1F2C8208E4; Sun, 1 Dec 2019 01:50:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165047; bh=WYPcX9zTkbpFT93PVsZfKpohbxeZdoKh1Ucj0kHGlvA=; h=Date:From:To:Subject:From; b=N+CZlBWKmkfy9mHUdkbC7p2qnGffNN/qbOy2dK3uzyUXlecyL1IBRdoT4PxSzp8O8 WMJBGzLU4djZn/hzibgxdm9SUIgPbWwdURu+bB4t7zwcKuVwzjVFakzCdeE010w7SW 4Nr1o1IZCTSWvFYoiu7l9rTbQizeyR7IcJbrtkpA= Date: Sat, 30 Nov 2019 17:50:46 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hch@infradead.org, linux-mm@kvack.org, mgorman@techsingularity.net, mm-commits@vger.kernel.org, osalvador@suse.de, richardw.yang@linux.intel.com, torvalds@linux-foundation.org, vbabka@suse.cz, willy@infradead.org Subject: [patch 031/158] mm/mmap.c: __vma_unlink_prev() is not necessary now Message-ID: <20191201015046.7koFRsYYb%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wei Yang Subject: mm/mmap.c: __vma_unlink_prev() is not necessary now The third parameter of __vma_unlink_common() could differentiate these two types. __vma_unlink_prev() is not necessary now. Link: http://lkml.kernel.org/r/20191006012636.31521-2-richardw.yang@linux.intel.com Signed-off-by: Wei Yang Cc: Christoph Hellwig Cc: Matthew Wilcox (Oracle) Cc: Mel Gorman Cc: Oscar Salvador Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- mm/mmap.c | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) --- a/mm/mmap.c~mm-mmapc-__vma_unlink_prev-is-not-necessary-now +++ a/mm/mmap.c @@ -702,13 +702,6 @@ static __always_inline void __vma_unlink vmacache_invalidate(mm); } -static inline void __vma_unlink_prev(struct mm_struct *mm, - struct vm_area_struct *vma, - struct vm_area_struct *prev) -{ - __vma_unlink_common(mm, vma, vma); -} - /* * We cannot adjust vm_start, vm_end, vm_pgoff fields of a vma that * is already present in an i_mmap tree without adjusting the tree. @@ -881,7 +874,7 @@ again: * us to remove next before dropping the locks. */ if (remove_next != 3) - __vma_unlink_prev(mm, next, vma); + __vma_unlink_common(mm, next, next); else /* * vma is not before next if they've been From patchwork Sun Dec 1 01:50:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268241 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8C91F921 for ; Sun, 1 Dec 2019 01:50:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5865620715 for ; Sun, 1 Dec 2019 01:50:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="r4Y6cg9/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5865620715 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 318C36B0296; Sat, 30 Nov 2019 20:50:52 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 29DD26B0298; Sat, 30 Nov 2019 20:50:52 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 18CDF6B0299; Sat, 30 Nov 2019 20:50:52 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 02AD56B0296 for ; Sat, 30 Nov 2019 20:50:51 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id A03B62C7C for ; Sun, 1 Dec 2019 01:50:51 +0000 (UTC) X-FDA: 76214893902.15.side56_77a2195c135e X-Spam-Summary: 2,0,0,319051465c52d6e6,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hch@infradead.org::mgorman@techsingularity.net:mm-commits@vger.kernel.org:osalvador@suse.de:richardw.yang@linux.intel.com:torvalds@linux-foundation.org:vbabka@suse.cz:willy@infradead.org,RULES_HIT:41:69:355:379:800:960:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3865:3867:3868:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:7903:8599:9010:9025:9545:9592:10004:10913:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13255:14181:14721:14849:21080:21451:21627:21939:30012:30054:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:4,LUA_SUMMARY:none X-HE-Tag: side56_77a2195c135e X-Filterd-Recvd-Size: 4312 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:51 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5370A2086A; Sun, 1 Dec 2019 01:50:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165050; bh=qelrs/S3JOt/lC9HVBfxAA60A4wAWAC9HBkcdS9kvu8=; h=Date:From:To:Subject:From; b=r4Y6cg9/GHjQQsHPjb20X4an4gvhqSqTN8Rw2uEfx2AeWGJohdFaLu4FnYu8s6LII JNj402E3CjfBwrx4GAzMDDHk5ldHr+Gish2m0AzVl+eN+hHdd1NjVXnf7VA4Dyqt3A DYu6V6Tl/FQOjRv2fSaBQg7fV347KyX8mmgeKYts= Date: Sat, 30 Nov 2019 17:50:49 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hch@infradead.org, linux-mm@kvack.org, mgorman@techsingularity.net, mm-commits@vger.kernel.org, osalvador@suse.de, richardw.yang@linux.intel.com, torvalds@linux-foundation.org, vbabka@suse.cz, willy@infradead.org Subject: [patch 032/158] mm/mmap.c: extract __vma_unlink_list() as counterpart for __vma_link_list() Message-ID: <20191201015049.AzHxsc5Ok%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wei Yang Subject: mm/mmap.c: extract __vma_unlink_list() as counterpart for __vma_link_list() Just make the code a little easier to read. Link: http://lkml.kernel.org/r/20191006012636.31521-3-richardw.yang@linux.intel.com Signed-off-by: Wei Yang Cc: Christoph Hellwig Cc: Matthew Wilcox (Oracle) Cc: Mel Gorman Cc: Oscar Salvador Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- mm/internal.h | 1 + mm/mmap.c | 12 +----------- mm/nommu.c | 8 +------- mm/util.c | 14 ++++++++++++++ 4 files changed, 17 insertions(+), 18 deletions(-) --- a/mm/internal.h~mm-mmapc-extract-__vma_unlink_list-as-counter-part-for-__vma_link_list +++ a/mm/internal.h @@ -291,6 +291,7 @@ static inline bool is_data_mapping(vm_fl /* mm/util.c */ void __vma_link_list(struct mm_struct *mm, struct vm_area_struct *vma, struct vm_area_struct *prev, struct rb_node *rb_parent); +void __vma_unlink_list(struct mm_struct *mm, struct vm_area_struct *vma); #ifdef CONFIG_MMU extern long populate_vma_page_range(struct vm_area_struct *vma, --- a/mm/mmap.c~mm-mmapc-extract-__vma_unlink_list-as-counter-part-for-__vma_link_list +++ a/mm/mmap.c @@ -686,18 +686,8 @@ static __always_inline void __vma_unlink struct vm_area_struct *vma, struct vm_area_struct *ignore) { - struct vm_area_struct *prev, *next; - vma_rb_erase_ignore(vma, &mm->mm_rb, ignore); - next = vma->vm_next; - prev = vma->vm_prev; - if (prev) - prev->vm_next = next; - else - mm->mmap = next; - if (next) - next->vm_prev = prev; - + __vma_unlink_list(mm, vma); /* Kill the cache */ vmacache_invalidate(mm); } --- a/mm/nommu.c~mm-mmapc-extract-__vma_unlink_list-as-counter-part-for-__vma_link_list +++ a/mm/nommu.c @@ -684,13 +684,7 @@ static void delete_vma_from_mm(struct vm /* remove from the MM's tree and list */ rb_erase(&vma->vm_rb, &mm->mm_rb); - if (vma->vm_prev) - vma->vm_prev->vm_next = vma->vm_next; - else - mm->mmap = vma->vm_next; - - if (vma->vm_next) - vma->vm_next->vm_prev = vma->vm_prev; + __vma_unlink_list(mm, vma); } /* --- a/mm/util.c~mm-mmapc-extract-__vma_unlink_list-as-counter-part-for-__vma_link_list +++ a/mm/util.c @@ -292,6 +292,20 @@ void __vma_link_list(struct mm_struct *m next->vm_prev = vma; } +void __vma_unlink_list(struct mm_struct *mm, struct vm_area_struct *vma) +{ + struct vm_area_struct *prev, *next; + + next = vma->vm_next; + prev = vma->vm_prev; + if (prev) + prev->vm_next = next; + else + mm->mmap = next; + if (next) + next->vm_prev = prev; +} + /* Check if the vma is being used as a stack by this task */ int vma_is_stack_for_current(struct vm_area_struct *vma) { From patchwork Sun Dec 1 01:50:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268243 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CFC54112B for ; Sun, 1 Dec 2019 01:50:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9C807206F0 for ; Sun, 1 Dec 2019 01:50:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="gYE+/12Y" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9C807206F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 780E76B0298; Sat, 30 Nov 2019 20:50:55 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 730F26B029A; Sat, 30 Nov 2019 20:50:55 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F9A06B029B; Sat, 30 Nov 2019 20:50:55 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0176.hostedemail.com [216.40.44.176]) by kanga.kvack.org (Postfix) with ESMTP id 47CA06B0298 for ; Sat, 30 Nov 2019 20:50:55 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 02843180AD817 for ; Sun, 1 Dec 2019 01:50:55 +0000 (UTC) X-FDA: 76214894070.27.verse90_7f64042a4058 X-Spam-Summary: 2,0,0,fa77ac76a5443632,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org::mgorman@techsingularity.net:mm-commits@vger.kernel.org:richardw.yang@linux.intel.com:torvalds@linux-foundation.org:vbabka@suse.cz:willy@infradead.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:8599:9010:9025:9545:10004:10913:11026:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12783:12986:13161:13229:14181:14721:14849:21080:21451:21627:21939:30012:30051:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: verse90_7f64042a4058 X-Filterd-Recvd-Size: 4144 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:54 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 855292073C; Sun, 1 Dec 2019 01:50:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165053; bh=dSnhQ39iU0lth5tK/Nzm9cXeYscgXQH+T9+A/REhXhc=; h=Date:From:To:Subject:From; b=gYE+/12YDsTB850cc6J7MSL7TNKOZyow6Dv0EwScbqSSnYM9e+T6Quf8Gs/HbjSQs tfTf06+ljuyD9JN3+Taq0nlWIKI83CRdIjgT5gL6uWrMEUv1BV7RfL5wHhVrMLzNbo VV1O+43yINyibfEMEdOznMeb3DUdbAuf3xhW1tps= Date: Sat, 30 Nov 2019 17:50:53 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, linux-mm@kvack.org, mgorman@techsingularity.net, mm-commits@vger.kernel.org, richardw.yang@linux.intel.com, torvalds@linux-foundation.org, vbabka@suse.cz, willy@infradead.org Subject: [patch 033/158] mm/mmap.c: rb_parent is not necessary in __vma_link_list() Message-ID: <20191201015053.yBGUELT9h%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wei Yang Subject: mm/mmap.c: rb_parent is not necessary in __vma_link_list() Now we use rb_parent to get next, while this is not necessary. When prev is NULL, this means vma should be the first element in the list. Then next should be current first one (mm->mmap), no matter whether we have parent or not. After removing it, the code shows the beauty of symmetry. Link: http://lkml.kernel.org/r/20190813032656.16625-1-richardw.yang@linux.intel.com Signed-off-by: Wei Yang Acked-by: Andrew Morton Cc: Mel Gorman Cc: Vlastimil Babka Cc: Matthew Wilcox Signed-off-by: Andrew Morton --- mm/internal.h | 2 +- mm/mmap.c | 2 +- mm/nommu.c | 2 +- mm/util.c | 8 ++------ 4 files changed, 5 insertions(+), 9 deletions(-) --- a/mm/internal.h~mm-mmapc-rb_parent-is-not-necessary-in-__vma_link_list +++ a/mm/internal.h @@ -290,7 +290,7 @@ static inline bool is_data_mapping(vm_fl /* mm/util.c */ void __vma_link_list(struct mm_struct *mm, struct vm_area_struct *vma, - struct vm_area_struct *prev, struct rb_node *rb_parent); + struct vm_area_struct *prev); void __vma_unlink_list(struct mm_struct *mm, struct vm_area_struct *vma); #ifdef CONFIG_MMU --- a/mm/mmap.c~mm-mmapc-rb_parent-is-not-necessary-in-__vma_link_list +++ a/mm/mmap.c @@ -641,7 +641,7 @@ __vma_link(struct mm_struct *mm, struct struct vm_area_struct *prev, struct rb_node **rb_link, struct rb_node *rb_parent) { - __vma_link_list(mm, vma, prev, rb_parent); + __vma_link_list(mm, vma, prev); __vma_link_rb(mm, vma, rb_link, rb_parent); } --- a/mm/nommu.c~mm-mmapc-rb_parent-is-not-necessary-in-__vma_link_list +++ a/mm/nommu.c @@ -648,7 +648,7 @@ static void add_vma_to_mm(struct mm_stru if (rb_prev) prev = rb_entry(rb_prev, struct vm_area_struct, vm_rb); - __vma_link_list(mm, vma, prev, parent); + __vma_link_list(mm, vma, prev); } /* --- a/mm/util.c~mm-mmapc-rb_parent-is-not-necessary-in-__vma_link_list +++ a/mm/util.c @@ -271,7 +271,7 @@ void *memdup_user_nul(const void __user EXPORT_SYMBOL(memdup_user_nul); void __vma_link_list(struct mm_struct *mm, struct vm_area_struct *vma, - struct vm_area_struct *prev, struct rb_node *rb_parent) + struct vm_area_struct *prev) { struct vm_area_struct *next; @@ -280,12 +280,8 @@ void __vma_link_list(struct mm_struct *m next = prev->vm_next; prev->vm_next = vma; } else { + next = mm->mmap; mm->mmap = vma; - if (rb_parent) - next = rb_entry(rb_parent, - struct vm_area_struct, vm_rb); - else - next = NULL; } vma->vm_next = next; if (next) From patchwork Sun Dec 1 01:50:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268245 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 327A2112B for ; Sun, 1 Dec 2019 01:51:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E9C83206F0 for ; Sun, 1 Dec 2019 01:50:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="dmIY41gK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E9C83206F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C89976B029A; Sat, 30 Nov 2019 20:50:58 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C3B6F6B029C; Sat, 30 Nov 2019 20:50:58 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B50736B029D; Sat, 30 Nov 2019 20:50:58 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0214.hostedemail.com [216.40.44.214]) by kanga.kvack.org (Postfix) with ESMTP id 9FFBA6B029A for ; Sat, 30 Nov 2019 20:50:58 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 68DBC824999B for ; Sun, 1 Dec 2019 01:50:58 +0000 (UTC) X-FDA: 76214894196.10.dock18_873737fd534c X-Spam-Summary: 2,0,0,7885f43804138749,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:cai@lca.pw:jglisse@redhat.com:khlebnikov@yandex-team.ru:kirill.shutemov@linux.intel.com::mike.kravetz@oracle.com:mm-commits@vger.kernel.org:richardw.yang@linux.intel.com:riel@surriel.com:shakeelb@google.com:torvalds@linux-foundation.org,RULES_HIT:41:152:355:379:800:960:967:973:988:989:1260:1263:1277:1311:1313:1314:1345:1381:1431:1437:1513:1515:1516:1518:1521:1535:1543:1593:1594:1605:1711:1730:1747:1777:1792:1969:2195:2198:2199:2200:2393:2525:2553:2559:2563:2682:2685:2689:2693:2859:2899:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6119:6261:6653:6737:7576:7903:8599:8660:8666:9025:9207:9545:10004:10400:10913:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13148:13151:13228:13230:14096:14097:14181:14721: 14849:14 X-HE-Tag: dock18_873737fd534c X-Filterd-Recvd-Size: 5226 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:50:57 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D59B6206E1; Sun, 1 Dec 2019 01:50:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165057; bh=1ps3QRFuKzUBB60qg0tSnlO8FbdgG53ScwlY77rMywA=; h=Date:From:To:Subject:From; b=dmIY41gKhIYvL6bwLxCqvBNuPPGtqg14BHdMyANK/LHnl0XwPLaLZqvj9ZeIh1Dhn yBj3qKjmA4jWewWzOgcu5CZKtC5MvrfBhDfIbxSnJGMvXSAjTHXrFmFqxsZrW97z40 GcZphKg0IT9VqCGb3HoPmOp97EavZ6qh7CSDpAtA= Date: Sat, 30 Nov 2019 17:50:56 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, cai@lca.pw, jglisse@redhat.com, khlebnikov@yandex-team.ru, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, richardw.yang@linux.intel.com, riel@surriel.com, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 034/158] mm/rmap.c: don't reuse anon_vma if we just want a copy Message-ID: <20191201015056.UHNvLWuJ8%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wei Yang Subject: mm/rmap.c: don't reuse anon_vma if we just want a copy Before commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma hierarchy"), anon_vma_clone() doesn't change dst->anon_vma. While after this commit, anon_vma_clone() will try to reuse an exist one on forking. But this commit go a little bit further for the case not forking. anon_vma_clone() is called from __vma_split(), __split_vma(), copy_vma() and anon_vma_fork(). For the first three places, the purpose here is get a copy of src and we don't expect to touch dst->anon_vma even it is NULL. While after that commit, it is possible to reuse an anon_vma when dst->anon_vma is NULL. This is not we intend to have. This patch stops reuse of anon_vma for non-fork cases. Link: http://lkml.kernel.org/r/20191011072256.16275-1-richardw.yang@linux.intel.com Fixes: 7a3ef208e662 ("mm: prevent endless growth of anon_vma hierarchy") Signed-off-by: Wei Yang Acked-by: Konstantin Khlebnikov Cc: Kirill A. Shutemov Cc: "Jérôme Glisse" Cc: Mike Kravetz Cc: Rik van Riel Cc: Qian Cai Cc: Shakeel Butt Signed-off-by: Andrew Morton --- mm/rmap.c | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) --- a/mm/rmap.c~mm-rmapc-dont-reuse-anon_vma-if-we-just-want-a-copy +++ a/mm/rmap.c @@ -251,13 +251,19 @@ static inline void unlock_anon_vma_root( * Attach the anon_vmas from src to dst. * Returns 0 on success, -ENOMEM on failure. * - * If dst->anon_vma is NULL this function tries to find and reuse existing - * anon_vma which has no vmas and only one child anon_vma. This prevents - * degradation of anon_vma hierarchy to endless linear chain in case of - * constantly forking task. On the other hand, an anon_vma with more than one - * child isn't reused even if there was no alive vma, thus rmap walker has a - * good chance of avoiding scanning the whole hierarchy when it searches where - * page is mapped. + * anon_vma_clone() is called by __vma_split(), __split_vma(), copy_vma() and + * anon_vma_fork(). The first three want an exact copy of src, while the last + * one, anon_vma_fork(), may try to reuse an existing anon_vma to prevent + * endless growth of anon_vma. Since dst->anon_vma is set to NULL before call, + * we can identify this case by checking (!dst->anon_vma && src->anon_vma). + * + * If (!dst->anon_vma && src->anon_vma) is true, this function tries to find + * and reuse existing anon_vma which has no vmas and only one child anon_vma. + * This prevents degradation of anon_vma hierarchy to endless linear chain in + * case of constantly forking task. On the other hand, an anon_vma with more + * than one child isn't reused even if there was no alive vma, thus rmap + * walker has a good chance of avoiding scanning the whole hierarchy when it + * searches where page is mapped. */ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src) { @@ -287,8 +293,8 @@ int anon_vma_clone(struct vm_area_struct * will always reuse it. Root anon_vma is never reused: * it has self-parent reference and at least one child. */ - if (!dst->anon_vma && anon_vma != src->anon_vma && - anon_vma->degree < 2) + if (!dst->anon_vma && src->anon_vma && + anon_vma != src->anon_vma && anon_vma->degree < 2) dst->anon_vma = anon_vma; } if (dst->anon_vma) From patchwork Sun Dec 1 01:50:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268247 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A378D921 for ; Sun, 1 Dec 2019 01:51:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6E8C1206E1 for ; Sun, 1 Dec 2019 01:51:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="bKRZdq74" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6E8C1206E1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 510306B029C; Sat, 30 Nov 2019 20:51:02 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4C1406B029E; Sat, 30 Nov 2019 20:51:02 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 387786B029F; Sat, 30 Nov 2019 20:51:02 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0057.hostedemail.com [216.40.44.57]) by kanga.kvack.org (Postfix) with ESMTP id 216066B029C for ; Sat, 30 Nov 2019 20:51:02 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id C9D7A2C8A for ; Sun, 1 Dec 2019 01:51:01 +0000 (UTC) X-FDA: 76214894322.20.songs61_8f276f79b230 X-Spam-Summary: 2,0,0,e5f2093291f7512b,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:cai@lca.pw:jglisse@redhat.com:khlebnikov@yandex-team.ru:kirill.shutemov@linux.intel.com::mike.kravetz@oracle.com:mm-commits@vger.kernel.org:richardw.yang@linux.intel.com:riel@surriel.com:shakeelb@google.com:torvalds@linux-foundation.org,RULES_HIT:41:152:355:379:800:960:967:973:988:989:1260:1263:1277:1311:1313:1314:1345:1381:1431:1437:1513:1515:1516:1518:1521:1534:1542:1593:1594:1711:1730:1747:1777:1792:1969:2195:2199:2393:2525:2553:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6119:6261:6653:6737:7576:7903:8599:9025:9207:9545:10004:10394:10400:10913:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:14181:14721:14849:14915:21080:21222:21324:21451:21627:21789:21795:21939:30051:3005 4:30064: X-HE-Tag: songs61_8f276f79b230 X-Filterd-Recvd-Size: 4331 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:01 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 632C6206F0; Sun, 1 Dec 2019 01:51:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165060; bh=JPgyu+DkpPaPHfSVib9rQetv+Rugns2LUUxAYmLW97Y=; h=Date:From:To:Subject:From; b=bKRZdq74uuaAzF1ihQhx4TQimw7wcqJZT32lcFZTQofyNICAWFPcDsWkIntBE97jU 8OTfSkmiGCxCp9M7zkptZ17BMlQBcgp/ZX3LYL+Xp6VzikwOdtl2ng4DFpieN10Qg3 eNm56TV2ZgbhAePaG/zILclDoXe6MOWMwoFHhXps= Date: Sat, 30 Nov 2019 17:50:59 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, cai@lca.pw, jglisse@redhat.com, khlebnikov@yandex-team.ru, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, richardw.yang@linux.intel.com, riel@surriel.com, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 035/158] mm/rmap.c: reuse mergeable anon_vma as parent when fork Message-ID: <20191201015059.JAkRF-SAj%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wei Yang Subject: mm/rmap.c: reuse mergeable anon_vma as parent when fork In __anon_vma_prepare(), we will try to find anon_vma if it is possible to reuse it. While on fork, the logic is different. Since commit 5beb49305251 ("mm: change anon_vma linking to fix multi-process server scalability issue"), function anon_vma_clone() tries to allocate new anon_vma for child process. But the logic here will allocate a new anon_vma for each vma, even in parent this vma is mergeable and share the same anon_vma with its sibling. This may do better for scalability issue, while it is not necessary to do so especially after interval tree is used. Commit 7a3ef208e662 ("mm: prevent endless growth of anon_vma hierarchy") tries to reuse some anon_vma by counting child anon_vma and attached vmas. While for those mergeable anon_vmas, we can just reuse it and not necessary to go through the logic. After this change, kernel build test reduces 20% anon_vma allocation. Do the same kernel build test, it shows run time in sys reduced 11.6%. Origin: real 2m50.467s user 17m52.002s sys 1m51.953s real 2m48.662s user 17m55.464s sys 1m50.553s real 2m51.143s user 17m59.687s sys 1m53.600s Patched: real 2m39.933s user 17m1.835s sys 1m38.802s real 2m39.321s user 17m1.634s sys 1m39.206s real 2m39.575s user 17m1.420s sys 1m38.845s Link: http://lkml.kernel.org/r/20191011072256.16275-2-richardw.yang@linux.intel.com Signed-off-by: Wei Yang Acked-by: Konstantin Khlebnikov Cc: Kirill A. Shutemov Cc: "Jérôme Glisse" Cc: Mike Kravetz Cc: Rik van Riel Cc: Qian Cai Cc: Shakeel Butt Signed-off-by: Andrew Morton --- mm/rmap.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) --- a/mm/rmap.c~mm-rmapc-reuse-mergeable-anon_vma-as-parent-when-fork +++ a/mm/rmap.c @@ -269,6 +269,19 @@ int anon_vma_clone(struct vm_area_struct { struct anon_vma_chain *avc, *pavc; struct anon_vma *root = NULL; + struct vm_area_struct *prev = dst->vm_prev, *pprev = src->vm_prev; + + /* + * If parent share anon_vma with its vm_prev, keep this sharing in in + * child. + * + * 1. Parent has vm_prev, which implies we have vm_prev. + * 2. Parent and its vm_prev have the same anon_vma. + */ + if (!dst->anon_vma && src->anon_vma && + pprev && pprev->anon_vma == src->anon_vma) + dst->anon_vma = prev->anon_vma; + list_for_each_entry_reverse(pavc, &src->anon_vma_chain, same_vma) { struct anon_vma *anon_vma; From patchwork Sun Dec 1 01:51:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268249 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EBE6C112B for ; Sun, 1 Dec 2019 01:51:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B817D206F0 for ; Sun, 1 Dec 2019 01:51:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="prLbjhyY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B817D206F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B788F6B029E; Sat, 30 Nov 2019 20:51:05 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B50256B02A0; Sat, 30 Nov 2019 20:51:05 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A8CB66B02A1; Sat, 30 Nov 2019 20:51:05 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0005.hostedemail.com [216.40.44.5]) by kanga.kvack.org (Postfix) with ESMTP id 912806B029E for ; Sat, 30 Nov 2019 20:51:05 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 4631A2A87 for ; Sun, 1 Dec 2019 01:51:05 +0000 (UTC) X-FDA: 76214894490.13.skirt61_971a12c38052 X-Spam-Summary: 2,0,0,a68f7230201e72be,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:cai@lca.pw:jglisse@redhat.com:khlebnikov@yandex-team.ru:kirill.shutemov@linux.intel.com::mhocko@suse.com:mike.kravetz@oracle.com:mm-commits@vger.kernel.org:pugaowei@gmail.com:richardw.yang@linux.intel.com:riel@surriel.com:shakeelb@google.com:torvalds@linux-foundation.org:vbabka@suse.cz,RULES_HIT:41:152:355:379:800:960:967:973:988:989:1260:1263:1277:1311:1313:1314:1345:1381:1431:1437:1513:1515:1516:1518:1521:1535:1543:1593:1594:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2859:2899:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3608:3865:3866:3867:3868:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4605:5007:6119:6261:6653:6737:7514:7576:7875:7903:8599:9025:9036:9207:9545:10004:10400:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13255:13846:14181 :14721:1 X-HE-Tag: skirt61_971a12c38052 X-Filterd-Recvd-Size: 5270 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:04 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A8CB3206E1; Sun, 1 Dec 2019 01:51:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165064; bh=Wgq/am9Z3fF8Hjbp64KlkRUoi5mtSEkhiDtvj6Kvpvs=; h=Date:From:To:Subject:From; b=prLbjhyYnFunYc5g6QkwzI8EL4Yyf69euaHiyYDWyjf7n9e6EpM86pead5nsC0fXC TDZtJtWODL0iyGp6jNY8mDNiNipvlK/38d0EtucAZakuTSbxJfH+EoKvCvzaxNxZwN gGIMg9fp9WFCxnjQyKtcsbY8ZBnXsaU3k+ceVomM= Date: Sat, 30 Nov 2019 17:51:03 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, cai@lca.pw, jglisse@redhat.com, khlebnikov@yandex-team.ru, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, pugaowei@gmail.com, richardw.yang@linux.intel.com, riel@surriel.com, shakeelb@google.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 036/158] mm/mmap.c: use IS_ERR_VALUE to check return value of get_unmapped_area Message-ID: <20191201015103.QxO3Ou2PX%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Gaowei Pu Subject: mm/mmap.c: use IS_ERR_VALUE to check return value of get_unmapped_area get_unmapped_area() returns an address or -errno on failure. Historically we have checked for the failure by offset_in_page() which is correct but quite hard to read. Newer code started using IS_ERR_VALUE which is much easier to read. Convert remaining users of offset_in_page as well. [mhocko@suse.com: rewrite changelog] [mhocko@kernel.org: fix mremap.c and uprobes.c sites also] Link: http://lkml.kernel.org/r/20191012102512.28051-1-pugaowei@gmail.com Signed-off-by: Gaowei Pu Reviewed-by: Andrew Morton Acked-by: Michal Hocko Cc: Vlastimil Babka Cc: Wei Yang Cc: Konstantin Khlebnikov Cc: Kirill A. Shutemov Cc: "Jérôme Glisse" Cc: Mike Kravetz Cc: Rik van Riel Cc: Qian Cai Cc: Shakeel Butt Signed-off-by: Andrew Morton --- kernel/events/uprobes.c | 2 +- mm/mmap.c | 9 +++++---- mm/mremap.c | 4 ++-- 3 files changed, 8 insertions(+), 7 deletions(-) --- a/kernel/events/uprobes.c~mm-mmapc-use-is_err_value-to-check-return-value-of-get_unmapped_area +++ a/kernel/events/uprobes.c @@ -1457,7 +1457,7 @@ static int xol_add_vma(struct mm_struct /* Try to map as high as possible, this is only a hint. */ area->vaddr = get_unmapped_area(NULL, TASK_SIZE - PAGE_SIZE, PAGE_SIZE, 0, 0); - if (area->vaddr & ~PAGE_MASK) { + if (IS_ERR_VALUE(area->vaddr)) { ret = area->vaddr; goto fail; } --- a/mm/mmap.c~mm-mmapc-use-is_err_value-to-check-return-value-of-get_unmapped_area +++ a/mm/mmap.c @@ -1417,7 +1417,7 @@ unsigned long do_mmap(struct file *file, * that it represents a valid section of the address space. */ addr = get_unmapped_area(file, addr, len, pgoff, flags); - if (offset_in_page(addr)) + if (IS_ERR_VALUE(addr)) return addr; if (flags & MAP_FIXED_NOREPLACE) { @@ -2981,15 +2981,16 @@ static int do_brk_flags(unsigned long ad struct rb_node **rb_link, *rb_parent; pgoff_t pgoff = addr >> PAGE_SHIFT; int error; + unsigned long mapped_addr; /* Until we need other flags, refuse anything except VM_EXEC. */ if ((flags & (~VM_EXEC)) != 0) return -EINVAL; flags |= VM_DATA_DEFAULT_FLAGS | VM_ACCOUNT | mm->def_flags; - error = get_unmapped_area(NULL, addr, len, 0, MAP_FIXED); - if (offset_in_page(error)) - return error; + mapped_addr = get_unmapped_area(NULL, addr, len, 0, MAP_FIXED); + if (IS_ERR_VALUE(mapped_addr)) + return mapped_addr; error = mlock_future_check(mm, mm->def_flags, len); if (error) --- a/mm/mremap.c~mm-mmapc-use-is_err_value-to-check-return-value-of-get_unmapped_area +++ a/mm/mremap.c @@ -558,7 +558,7 @@ static unsigned long mremap_to(unsigned ret = get_unmapped_area(vma->vm_file, new_addr, new_len, vma->vm_pgoff + ((addr - vma->vm_start) >> PAGE_SHIFT), map_flags); - if (offset_in_page(ret)) + if (IS_ERR_VALUE(ret)) goto out1; ret = move_vma(vma, addr, old_len, new_len, new_addr, locked, uf, @@ -706,7 +706,7 @@ SYSCALL_DEFINE5(mremap, unsigned long, a vma->vm_pgoff + ((addr - vma->vm_start) >> PAGE_SHIFT), map_flags); - if (offset_in_page(new_addr)) { + if (IS_ERR_VALUE(new_addr)) { ret = new_addr; goto out; } From patchwork Sun Dec 1 01:51:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268251 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 88F8D112B for ; Sun, 1 Dec 2019 01:51:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 550242073C for ; Sun, 1 Dec 2019 01:51:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="q8e4UkqV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 550242073C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 473636B02A0; Sat, 30 Nov 2019 20:51:09 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4250B6B02A2; Sat, 30 Nov 2019 20:51:09 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 33A066B02A3; Sat, 30 Nov 2019 20:51:09 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0018.hostedemail.com [216.40.44.18]) by kanga.kvack.org (Postfix) with ESMTP id 1D2236B02A0 for ; Sat, 30 Nov 2019 20:51:09 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id C9331180AD817 for ; Sun, 1 Dec 2019 01:51:08 +0000 (UTC) X-FDA: 76214894616.19.magic00_9f3cffaf8827 X-Spam-Summary: 2,0,0,c79f624be24a0d3e,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:aneesh.kumar@linux.ibm.com:arnd@arndb.de:kirill.shutemov@linux.intel.com::mm-commits@vger.kernel.org:npiggin@gmail.com:peterz@infradead.org:torvalds@linux-foundation.org:vgupta@synopsys.com:vineet.gupta1@synopsys.com:will@kernel.org,RULES_HIT:41:355:379:800:960:966:967:968:973:988:989:1042:1260:1263:1345:1381:1431:1437:1535:1544:1711:1730:1747:1777:1792:1801:1981:2194:2196:2198:2199:2200:2201:2393:2525:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3866:3867:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4605:5007:6119:6261:6653:6737:7208:7514:7576:7903:7904:8599:8784:9025:9121:9545:10004:10913:11026:11473:11658:11914:12043:12048:12114:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:13846:14181:14721:14849:21080:21324:21451:21627:21772:21939:30003:30012:3005 4:30070, X-HE-Tag: magic00_9f3cffaf8827 X-Filterd-Recvd-Size: 5553 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:08 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2A418206F0; Sun, 1 Dec 2019 01:51:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165067; bh=so4E6KVNKsv4vHKf6ha+T7+bLs2yXVSwPa1w4H0AdtY=; h=Date:From:To:Subject:From; b=q8e4UkqV3LQpAIAXnwQ25dNPa5Jo5mtcizzmCqOx3EEoZZeNAPSGenFU448lg8PM1 2YNq8RffUy3I3KYVl2fMxSSysR0TRezkWlTDlzSqUl+6C3TPPrkqUYND9RaxONUs0i mIrjcwmL8Gww9XtCIjJCjfZtJkY8t+Hy4p/YkHYs= Date: Sat, 30 Nov 2019 17:51:06 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, arnd@arndb.de, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, npiggin@gmail.com, peterz@infradead.org, torvalds@linux-foundation.org, vgupta@synopsys.com, Vineet.Gupta1@synopsys.com, will@kernel.org Subject: [patch 037/158] ARC: mm: remove __ARCH_USE_5LEVEL_HACK Message-ID: <20191201015106.CD4zNxmtL%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vineet Gupta Subject: ARC: mm: remove __ARCH_USE_5LEVEL_HACK Patch series "elide extraneous generated code for folded p4d/pud/pmd", v3. This series came out of seemingly benign excursion into understanding/removing __ARCH_USE_5LEVEL_HACK from ARC port showing some extraneous code being generated despite folded p4d/pud/pmd | bloat-o-meter2 vmlinux-[AB]* | add/remove: 0/0 grow/shrink: 3/0 up/down: 130/0 (130) | function old new delta | free_pgd_range 548 660 +112 | p4d_clear_bad 2 20 +18 The patches here address that | bloat-o-meter2 vmlinux-[BF]* | add/remove: 0/2 grow/shrink: 0/1 up/down: 0/-386 (-386) | function old new delta | pud_clear_bad 20 - -20 | p4d_clear_bad 20 - -20 | free_pgd_range 660 314 -346 The code savings are not a whole lot, but still worthwhile IMHO. This patch (of 5): With paging code made 5-level compliant, this is no longer needed. ARC has software page walker with 2 lookup levels (pgd -> pte) This was expected to be non functional change but ended with slight code bloat due to needless inclusions of p*d_free_tlb() macros which will be addressed in further patches. | bloat-o-meter2 vmlinux-[AB]* | add/remove: 0/0 grow/shrink: 2/0 up/down: 128/0 (128) | function old new delta | free_pgd_range 546 656 +110 | p4d_clear_bad 2 20 +18 | Total: Before=4137148, After=4137276, chg 0.000000% Link: http://lkml.kernel.org/r/20191016162400.14796-2-vgupta@synopsys.com Signed-off-by: Vineet Gupta Acked-by: Kirill A. Shutemov Cc: "Aneesh Kumar K . V" Cc: Arnd Bergmann Cc: Linus Torvalds Cc: Nick Piggin Cc: Peter Zijlstra Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arc/include/asm/pgtable.h | 1 - arch/arc/mm/fault.c | 10 ++++++++-- arch/arc/mm/highmem.c | 4 +++- 3 files changed, 11 insertions(+), 4 deletions(-) --- a/arch/arc/include/asm/pgtable.h~arc-mm-remove-__arch_use_5level_hack +++ a/arch/arc/include/asm/pgtable.h @@ -33,7 +33,6 @@ #define _ASM_ARC_PGTABLE_H #include -#define __ARCH_USE_5LEVEL_HACK #include #include #include /* to propagate CONFIG_ARC_MMU_VER */ --- a/arch/arc/mm/fault.c~arc-mm-remove-__arch_use_5level_hack +++ a/arch/arc/mm/fault.c @@ -30,6 +30,7 @@ noinline static int handle_kernel_vaddr_ * with the 'reference' page table. */ pgd_t *pgd, *pgd_k; + p4d_t *p4d, *p4d_k; pud_t *pud, *pud_k; pmd_t *pmd, *pmd_k; @@ -39,8 +40,13 @@ noinline static int handle_kernel_vaddr_ if (!pgd_present(*pgd_k)) goto bad_area; - pud = pud_offset(pgd, address); - pud_k = pud_offset(pgd_k, address); + p4d = p4d_offset(pgd, address); + p4d_k = p4d_offset(pgd_k, address); + if (!p4d_present(*p4d_k)) + goto bad_area; + + pud = pud_offset(p4d, address); + pud_k = pud_offset(p4d_k, address); if (!pud_present(*pud_k)) goto bad_area; --- a/arch/arc/mm/highmem.c~arc-mm-remove-__arch_use_5level_hack +++ a/arch/arc/mm/highmem.c @@ -111,12 +111,14 @@ EXPORT_SYMBOL(__kunmap_atomic); static noinline pte_t * __init alloc_kmap_pgtable(unsigned long kvaddr) { pgd_t *pgd_k; + p4d_t *p4d_k; pud_t *pud_k; pmd_t *pmd_k; pte_t *pte_k; pgd_k = pgd_offset_k(kvaddr); - pud_k = pud_offset(pgd_k, kvaddr); + p4d_k = p4d_offset(pgd_k, kvaddr); + pud_k = pud_offset(p4d_k, kvaddr); pmd_k = pmd_offset(pud_k, kvaddr); pte_k = (pte_t *)memblock_alloc_low(PAGE_SIZE, PAGE_SIZE); From patchwork Sun Dec 1 01:51:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268253 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CBE18921 for ; Sun, 1 Dec 2019 01:51:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 98B942086A for ; Sun, 1 Dec 2019 01:51:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="UtJs7prH" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 98B942086A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 70BF76B02A2; Sat, 30 Nov 2019 20:51:12 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6BF7A6B02A4; Sat, 30 Nov 2019 20:51:12 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A9FC6B02A5; Sat, 30 Nov 2019 20:51:12 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0085.hostedemail.com [216.40.44.85]) by kanga.kvack.org (Postfix) with ESMTP id 426F56B02A2 for ; Sat, 30 Nov 2019 20:51:12 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id EF40440F1 for ; Sun, 1 Dec 2019 01:51:11 +0000 (UTC) X-FDA: 76214894784.13.dress26_a6d458f63e06 X-Spam-Summary: 2,0,0,7df301843f74c870,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:aneesh.kumar@linux.ibm.com:arnd@arndb.de:kirill.shutemov@linux.intel.com::mm-commits@vger.kernel.org:npiggin@gmail.com:peterz@infradead.org:torvalds@linux-foundation.org:vgupta@synopsys.com:vineet.gupta1@synopsys.com:will@kernel.org,RULES_HIT:41:355:379:800:960:965:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:1801:1978:1981:2194:2196:2199:2200:2393:2525:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4390:4395:4605:5007:6119:6261:6653:6737:7208:7514:7576:7903:7904:8599:8784:9025:9113:9121:9545:10004:10913:11026:11473:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12783:12986:13180:13229:13846:14181:14721:14849:21080:21451:21627:21939:30003:30012:30054:30070,0,RBL:error,Cach eIP:none X-HE-Tag: dress26_a6d458f63e06 X-Filterd-Recvd-Size: 4301 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:11 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 8B7922073C; Sun, 1 Dec 2019 01:51:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165071; bh=n3XN128wYSp2DeTLpenC+GS3TnYwMVjIWyZxyosMMQc=; h=Date:From:To:Subject:From; b=UtJs7prHsw6XQiQmQfM3lhh0BQcsPO3mFh4oIbiKtc2mQrd5/clSHNxK/Thoe7dXD Hlbuson3CFTD676Wgda3KaUqSBZ426UsATJ3cQa3uX1WBC2uj/im9LlsehqiPcGQyI Z0AtrYrVT6c6L0UqDrkEDPDItmNbIusO0c230QCs= Date: Sat, 30 Nov 2019 17:51:10 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, arnd@arndb.de, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, npiggin@gmail.com, peterz@infradead.org, torvalds@linux-foundation.org, vgupta@synopsys.com, Vineet.Gupta1@synopsys.com, will@kernel.org Subject: [patch 038/158] asm-generic/tlb: stub out pud_free_tlb() if nopud ... Message-ID: <20191201015110.crZ6tJxJL%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vineet Gupta Subject: asm-generic/tlb: stub out pud_free_tlb() if nopud ... ... independent of __ARCH_HAS_4LEVEL_HACK This came up when removing __ARCH_HAS_5LEVEL_HACK for ARC as code bloat. With this patch we see the following code reduction | bloat-o-meter2 vmlinux-B-elide-ARCH_USE_5LEVEL_HACK vmlinux-C-elide-pud_free_tlb | add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-104 (-104) | function old new delta | free_pgd_range 656 552 -104 | Total: Before=4137276, After=4137172, chg -1.000000% Note: The primary change is alternate defintion for pud_free_tlb() but while there also removed empty stubs for __pud_free_tlb, which is anyhow called only from pud_free_tlb() Link: http://lkml.kernel.org/r/20191016162400.14796-3-vgupta@synopsys.com Signed-off-by: Vineet Gupta Acked-by: Kirill A. Shutemov Acked-by: Linus Torvalds Cc: "Aneesh Kumar K . V" Cc: Arnd Bergmann Cc: Nick Piggin Cc: Peter Zijlstra Cc: Will Deacon Signed-off-by: Andrew Morton --- include/asm-generic/4level-fixup.h | 1 - include/asm-generic/pgtable-nopud.h | 2 +- include/asm-generic/tlb.h | 2 -- 3 files changed, 1 insertion(+), 4 deletions(-) --- a/include/asm-generic/4level-fixup.h~asm-generic-tlb-stub-out-pud_free_tlb-if-nopud +++ a/include/asm-generic/4level-fixup.h @@ -30,7 +30,6 @@ #undef pud_free_tlb #define pud_free_tlb(tlb, x, addr) do { } while (0) #define pud_free(mm, x) do { } while (0) -#define __pud_free_tlb(tlb, x, addr) do { } while (0) #undef pud_addr_end #define pud_addr_end(addr, end) (end) --- a/include/asm-generic/pgtable-nopud.h~asm-generic-tlb-stub-out-pud_free_tlb-if-nopud +++ a/include/asm-generic/pgtable-nopud.h @@ -59,7 +59,7 @@ static inline pud_t *pud_offset(p4d_t *p */ #define pud_alloc_one(mm, address) NULL #define pud_free(mm, x) do { } while (0) -#define __pud_free_tlb(tlb, x, a) do { } while (0) +#define pud_free_tlb(tlb, x, a) do { } while (0) #undef pud_addr_end #define pud_addr_end(addr, end) (end) --- a/include/asm-generic/tlb.h~asm-generic-tlb-stub-out-pud_free_tlb-if-nopud +++ a/include/asm-generic/tlb.h @@ -584,7 +584,6 @@ static inline void tlb_end_vma(struct mm } while (0) #endif -#ifndef __ARCH_HAS_4LEVEL_HACK #ifndef pud_free_tlb #define pud_free_tlb(tlb, pudp, address) \ do { \ @@ -594,7 +593,6 @@ static inline void tlb_end_vma(struct mm __pud_free_tlb(tlb, pudp, address); \ } while (0) #endif -#endif #ifndef __ARCH_HAS_5LEVEL_HACK #ifndef p4d_free_tlb From patchwork Sun Dec 1 01:51:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268255 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A003921 for ; Sun, 1 Dec 2019 01:51:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 06BB92073C for ; Sun, 1 Dec 2019 01:51:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="JSyge2pL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 06BB92073C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B790F6B02A4; Sat, 30 Nov 2019 20:51:15 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B286C6B02A6; Sat, 30 Nov 2019 20:51:15 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3ED46B02A7; Sat, 30 Nov 2019 20:51:15 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0224.hostedemail.com [216.40.44.224]) by kanga.kvack.org (Postfix) with ESMTP id 8AF1C6B02A4 for ; Sat, 30 Nov 2019 20:51:15 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 3902140DD for ; Sun, 1 Dec 2019 01:51:15 +0000 (UTC) X-FDA: 76214894910.02.smash91_ae92cbf47524 X-Spam-Summary: 2,0,0,d5232544a0b056c1,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:aneesh.kumar@linux.ibm.com:arnd@arndb.de:kirill.shutemov@linux.intel.com::mm-commits@vger.kernel.org:npiggin@gmail.com:peterz@infradead.org:torvalds@linux-foundation.org:vgupta@synopsys.com:vineet.gupta1@synopsys.com:will@kernel.org,RULES_HIT:41:355:379:800:960:965:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:1801:1978:1981:2194:2196:2199:2200:2393:2525:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4390:4395:4605:5007:6119:6261:6653:6737:7208:7514:7576:7903:7904:8599:8784:9025:9113:9121:9545:10004:10913:11026:11473:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12783:12986:13846:14181:14721:14849:21080:21451:21611:21627:21939:30003:30012:30054,0,RBL:error,CacheIP:none,Bayesian :0.5,0.5 X-HE-Tag: smash91_ae92cbf47524 X-Filterd-Recvd-Size: 4119 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:14 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D5E38206F0; Sun, 1 Dec 2019 01:51:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165074; bh=BLW52AgT78+G0osewpmmQ4znGH+hK5/pwnU/Lx/9ynU=; h=Date:From:To:Subject:From; b=JSyge2pLAIKuOpkIdX57n560o8dbzm9MCYqTjgZMinC/XDf2vbt/MeyslTOsJX6Ts JlILvz5ciEqO/XWoT2q7zC+VODnuOAjHOc3QomswAV2n1aF58qgxIoAY/N5bnq48yx 9gmJ83wJLx7g8HuHtvWB+ZLlPciId1pQWaSSS8Z4= Date: Sat, 30 Nov 2019 17:51:13 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, arnd@arndb.de, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, npiggin@gmail.com, peterz@infradead.org, torvalds@linux-foundation.org, vgupta@synopsys.com, Vineet.Gupta1@synopsys.com, will@kernel.org Subject: [patch 039/158] asm-generic/tlb: stub out p4d_free_tlb() if nop4d ... Message-ID: <20191201015113.4sakLx4B5%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vineet Gupta Subject: asm-generic/tlb: stub out p4d_free_tlb() if nop4d ... ... independent of __ARCH_HAS_5LEVEL_HACK This came up when removing __ARCH_HAS_5LEVEL_HACK for ARC as code bloat. With this patch we see the following code reduction | bloat-o-meter2 vmlinux-C-elide-pud_free_tlb vmlinux-D-elide-p4d_free_tlb | add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-104 (-104) | function old new delta | free_pgd_range 552 422 -130 | Total: Before=4137172, After=4137042, chg -1.000000% Link: http://lkml.kernel.org/r/20191016162400.14796-4-vgupta@synopsys.com Signed-off-by: Vineet Gupta Acked-by: Kirill A. Shutemov Acked-by: Linus Torvalds Cc: "Aneesh Kumar K . V" Cc: Arnd Bergmann Cc: Nick Piggin Cc: Peter Zijlstra Cc: Will Deacon Signed-off-by: Andrew Morton --- include/asm-generic/5level-fixup.h | 1 - include/asm-generic/pgtable-nop4d.h | 2 +- include/asm-generic/tlb.h | 2 -- 3 files changed, 1 insertion(+), 4 deletions(-) --- a/include/asm-generic/5level-fixup.h~asm-generic-tlb-stub-out-p4d_free_tlb-if-nop4d +++ a/include/asm-generic/5level-fixup.h @@ -51,7 +51,6 @@ static inline int p4d_present(p4d_t p4d) #undef p4d_free_tlb #define p4d_free_tlb(tlb, x, addr) do { } while (0) #define p4d_free(mm, x) do { } while (0) -#define __p4d_free_tlb(tlb, x, addr) do { } while (0) #undef p4d_addr_end #define p4d_addr_end(addr, end) (end) --- a/include/asm-generic/pgtable-nop4d.h~asm-generic-tlb-stub-out-p4d_free_tlb-if-nop4d +++ a/include/asm-generic/pgtable-nop4d.h @@ -50,7 +50,7 @@ static inline p4d_t *p4d_offset(pgd_t *p */ #define p4d_alloc_one(mm, address) NULL #define p4d_free(mm, x) do { } while (0) -#define __p4d_free_tlb(tlb, x, a) do { } while (0) +#define p4d_free_tlb(tlb, x, a) do { } while (0) #undef p4d_addr_end #define p4d_addr_end(addr, end) (end) --- a/include/asm-generic/tlb.h~asm-generic-tlb-stub-out-p4d_free_tlb-if-nop4d +++ a/include/asm-generic/tlb.h @@ -594,7 +594,6 @@ static inline void tlb_end_vma(struct mm } while (0) #endif -#ifndef __ARCH_HAS_5LEVEL_HACK #ifndef p4d_free_tlb #define p4d_free_tlb(tlb, pudp, address) \ do { \ @@ -603,7 +602,6 @@ static inline void tlb_end_vma(struct mm __p4d_free_tlb(tlb, pudp, address); \ } while (0) #endif -#endif #endif /* CONFIG_MMU */ From patchwork Sun Dec 1 01:51:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268257 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A01CB112B for ; Sun, 1 Dec 2019 01:51:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6351B206F0 for ; Sun, 1 Dec 2019 01:51:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="a0DZ/QBU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6351B206F0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 07B9F6B02A6; Sat, 30 Nov 2019 20:51:19 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 02F1C6B02A8; Sat, 30 Nov 2019 20:51:18 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E84206B02A9; Sat, 30 Nov 2019 20:51:18 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0219.hostedemail.com [216.40.44.219]) by kanga.kvack.org (Postfix) with ESMTP id D06266B02A6 for ; Sat, 30 Nov 2019 20:51:18 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 8736982499A8 for ; Sun, 1 Dec 2019 01:51:18 +0000 (UTC) X-FDA: 76214895036.20.meal48_b637481b222e X-Spam-Summary: 2,0,0,c50538c8e73da046,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:aneesh.kumar@linux.ibm.com:arnd@arndb.de:kirill.shutemov@linux.intel.com::mm-commits@vger.kernel.org:npiggin@gmail.com:peterz@infradead.org:torvalds@linux-foundation.org:vgupta@synopsys.com:vineet.gupta1@synopsys.com:will@kernel.org,RULES_HIT:41:355:379:800:960:965:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:1801:1978:1981:2194:2196:2199:2200:2393:2525:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4390:4395:4605:5007:6119:6261:6653:6737:7208:7514:7576:7903:7904:8599:9025:9113:9121:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13069:13180:13223:13229:13311:13357:13846:14181:14384:14581:14721:14777:14849:21080: 21451:21 X-HE-Tag: meal48_b637481b222e X-Filterd-Recvd-Size: 3231 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:18 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1384B208E4; Sun, 1 Dec 2019 01:51:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165077; bh=EXWI8BKrEfmXJcJx0HqZC2HE4TaIKMbc/zYRhrwY/yk=; h=Date:From:To:Subject:From; b=a0DZ/QBU9NwmiKHNso7ZRww/x1LSXZ4DMEZtGkJ/GRu7ZYRsMO4h/EdrLpxdDaMbZ a6pVku4plcj8J4B53eW88hcVTnGSFYHZx4qaZlDu9N5XNQ3DRvOdMRqBuwp3RcC7/b N2Y7Zr/MSORIZS33WmDjuudjbnrVEiAdSJnIuUy4= Date: Sat, 30 Nov 2019 17:51:16 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, arnd@arndb.de, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, npiggin@gmail.com, peterz@infradead.org, torvalds@linux-foundation.org, vgupta@synopsys.com, Vineet.Gupta1@synopsys.com, will@kernel.org Subject: [patch 040/158] asm-generic/tlb: stub out pmd_free_tlb() if nopmd Message-ID: <20191201015116.wee7OfhTE%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vineet Gupta Subject: asm-generic/tlb: stub out pmd_free_tlb() if nopmd This came up when removing __ARCH_HAS_5LEVEL_HACK for ARC as code bloat. With this patch we see the following code reduction. | bloat-o-meter2 vmlinux-E-elide-p?d_clear_bad vmlinux-F-elide-pmd_free_tlb | add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-112 (-112) | function old new delta | free_pgd_range 422 310 -112 | Total: Before=4137042, After=4136930, chg -1.000000% Note that pmd folding can be tricky: In 2-level setup (where pmd is conceptually folded) most pmd routines are valid and refer to upper levels. In this patch we can, but see next patch for example where we can't Link: http://lkml.kernel.org/r/20191016162400.14796-5-vgupta@synopsys.com Signed-off-by: Vineet Gupta Acked-by: Kirill A. Shutemov Cc: "Aneesh Kumar K . V" Cc: Arnd Bergmann Cc: Linus Torvalds Cc: Nick Piggin Cc: Peter Zijlstra Cc: Will Deacon Signed-off-by: Andrew Morton --- include/asm-generic/pgtable-nopmd.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/include/asm-generic/pgtable-nopmd.h~asm-generic-tlb-stub-out-pmd_free_tlb-if-nopmd +++ a/include/asm-generic/pgtable-nopmd.h @@ -60,7 +60,7 @@ static inline pmd_t * pmd_offset(pud_t * static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd) { } -#define __pmd_free_tlb(tlb, x, a) do { } while (0) +#define pmd_free_tlb(tlb, x, a) do { } while (0) #undef pmd_addr_end #define pmd_addr_end(addr, end) (end) From patchwork Sun Dec 1 01:51:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268259 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 09D30112B for ; Sun, 1 Dec 2019 01:51:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C0F2B215E5 for ; Sun, 1 Dec 2019 01:51:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZLxG2cNW" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C0F2B215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 585036B02A8; Sat, 30 Nov 2019 20:51:22 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 55DC26B02AA; Sat, 30 Nov 2019 20:51:22 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 424906B02AB; Sat, 30 Nov 2019 20:51:22 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0119.hostedemail.com [216.40.44.119]) by kanga.kvack.org (Postfix) with ESMTP id 2C3706B02A8 for ; Sat, 30 Nov 2019 20:51:22 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id DA3B2824999B for ; Sun, 1 Dec 2019 01:51:21 +0000 (UTC) X-FDA: 76214895162.20.twig06_bdf81d019330 X-Spam-Summary: 2,0,0,4bc6524e3ecaf1be,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:aneesh.kumar@linux.ibm.com:arnd@arndb.de:kirill.shutemov@linux.intel.com::mm-commits@vger.kernel.org:npiggin@gmail.com:peterz@infradead.org:torvalds@linux-foundation.org:vgupta@synopsys.com:vineet.gupta1@synopsys.com:will@kernel.org,RULES_HIT:41:355:379:800:960:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:1801:1978:1981:2194:2196:2199:2200:2393:2525:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4605:5007:6119:6261:6653:6737:7208:7514:7576:7903:7904:7974:8599:8784:9025:9113:9121:9545:10004:10913:11026:11473:11658:11914:12043:12048:12114:12295:12297:12438:12517:12519:12555:12679:12783:12986:13255:13846:14181:14721:14849:21080:21451:21627:21939:30012:30054,0,RBL:error,Cach eIP:none X-HE-Tag: twig06_bdf81d019330 X-Filterd-Recvd-Size: 4114 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf28.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:21 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 6E4B82084E; Sun, 1 Dec 2019 01:51:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165080; bh=oy0j8etxcASYVIOiIT9WcHyDlXpOfrWG08HrLcdEfl0=; h=Date:From:To:Subject:From; b=ZLxG2cNW/vR0sw+tdHBEsJj5aHzpDR88+pZ8TAMjx74KyAZBKGn1j/tmC1d67NyOI qokxlMqpcjB12tJc+FjRIlHgUQx8Tf6wZgJHA85OrQwjnW6qPvpCNPupIWAxHLmfwC AliGdJBy7CQpIcK3fFaY8Wmh7NOpMXaHjzDoiaIA= Date: Sat, 30 Nov 2019 17:51:20 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, arnd@arndb.de, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, npiggin@gmail.com, peterz@infradead.org, torvalds@linux-foundation.org, vgupta@synopsys.com, Vineet.Gupta1@synopsys.com, will@kernel.org Subject: [patch 041/158] asm-generic/mm: stub out p{4,u}d_clear_bad() if __PAGETABLE_P{4,U}D_FOLDED Message-ID: <20191201015120.r8GhpSk1Y%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vineet Gupta Subject: asm-generic/mm: stub out p{4,u}d_clear_bad() if __PAGETABLE_P{4,U}D_FOLDED This came up when removing __ARCH_HAS_5LEVEL_HACK for ARC as code bloat. With this patch we see the following code reduction. | bloat-o-meter2 vmlinux-D-elide-p4d_free_tlb vmlinux-E-elide-p?d_clear_bad | add/remove: 0/2 grow/shrink: 0/0 up/down: 0/-40 (-40) | function old new delta | pud_clear_bad 20 - -20 | p4d_clear_bad 20 - -20 | Total: Before=4136930, After=4136890, chg -1.000000% Link: http://lkml.kernel.org/r/20191016162400.14796-6-vgupta@synopsys.com Signed-off-by: Vineet Gupta Acked-by: Kirill A. Shutemov Cc: Arnd Bergmann Cc: Will Deacon Cc: "Aneesh Kumar K . V" Cc: Nick Piggin Cc: Peter Zijlstra Cc: Linus Torvalds Signed-off-by: Andrew Morton --- include/asm-generic/pgtable.h | 11 +++++++++++ mm/pgtable-generic.c | 9 +++++++++ 2 files changed, 20 insertions(+) --- a/include/asm-generic/pgtable.h~asm-generic-mm-stub-out-p4ud_clear_bad-if-__pagetable_p4ud_folded +++ a/include/asm-generic/pgtable.h @@ -558,8 +558,19 @@ static inline pgprot_t pgprot_modify(pgp * Do the tests inline, but report and clear the bad entry in mm/memory.c. */ void pgd_clear_bad(pgd_t *); + +#ifndef __PAGETABLE_P4D_FOLDED void p4d_clear_bad(p4d_t *); +#else +#define p4d_clear_bad(p4d) do { } while (0) +#endif + +#ifndef __PAGETABLE_PUD_FOLDED void pud_clear_bad(pud_t *); +#else +#define pud_clear_bad(p4d) do { } while (0) +#endif + void pmd_clear_bad(pmd_t *); static inline int pgd_none_or_clear_bad(pgd_t *pgd) --- a/mm/pgtable-generic.c~asm-generic-mm-stub-out-p4ud_clear_bad-if-__pagetable_p4ud_folded +++ a/mm/pgtable-generic.c @@ -24,18 +24,27 @@ void pgd_clear_bad(pgd_t *pgd) pgd_clear(pgd); } +#ifndef __PAGETABLE_P4D_FOLDED void p4d_clear_bad(p4d_t *p4d) { p4d_ERROR(*p4d); p4d_clear(p4d); } +#endif +#ifndef __PAGETABLE_PUD_FOLDED void pud_clear_bad(pud_t *pud) { pud_ERROR(*pud); pud_clear(pud); } +#endif +/* + * Note that the pmd variant below can't be stub'ed out just as for p4d/pud + * above. pmd folding is special and typically pmd_* macros refer to upper + * level even when folded + */ void pmd_clear_bad(pmd_t *pmd) { pmd_ERROR(*pmd); From patchwork Sun Dec 1 01:51:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268261 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 17FE7921 for ; Sun, 1 Dec 2019 01:51:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D8B15208E4 for ; Sun, 1 Dec 2019 01:51:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="brdrJhuw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D8B15208E4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 389F86B02AA; Sat, 30 Nov 2019 20:51:25 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 33F7A6B02AC; Sat, 30 Nov 2019 20:51:25 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 278726B02AD; Sat, 30 Nov 2019 20:51:25 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0125.hostedemail.com [216.40.44.125]) by kanga.kvack.org (Postfix) with ESMTP id 09BAC6B02AA for ; Sat, 30 Nov 2019 20:51:25 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id C9F5740DD for ; Sun, 1 Dec 2019 01:51:24 +0000 (UTC) X-FDA: 76214895288.17.fruit82_c4cfb581a213 X-Spam-Summary: 2,0,0,3acfc8ef57041861,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org::miles.chen@mediatek.com:mm-commits@vger.kernel.org:paulmck@kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2194:2199:2393:2525:2559:2563:2682:2685:2859:2902:2915:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:7903:8599:8957:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13069:13311:13357:13846:14181:14384:14721:14849:21080:21451:21627:21939:30041:30054:30069,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0 ,LFtime: X-HE-Tag: fruit82_c4cfb581a213 X-Filterd-Recvd-Size: 2681 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:24 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9CF4620880; Sun, 1 Dec 2019 01:51:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165083; bh=iWBHcdA+bdpU8QXhtXLYQVnDQR43HVBDwD3MwX7XDnk=; h=Date:From:To:Subject:From; b=brdrJhuwpM9T63q9Y4lnrg76k18jI7by8UT64EAXwCwQtDun72yysBdZJrDCDM9GL 5ngYLb7bOXVWQn4jZhb95Rs7Nm7uAeZmQmlpMb94/4PT6TH37ZPU5F0UxcMKrTG7U2 TuSWPPob3cY/KCA/wSFBgqrC3NOMGTq3ZiLl6YGI= Date: Sat, 30 Nov 2019 17:51:23 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, linux-mm@kvack.org, miles.chen@mediatek.com, mm-commits@vger.kernel.org, paulmck@kernel.org, torvalds@linux-foundation.org Subject: [patch 042/158] mm/rmap.c: fix outdated comment in page_get_anon_vma() Message-ID: <20191201015123.RcoCgMv9e%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miles Chen Subject: mm/rmap.c: fix outdated comment in page_get_anon_vma() Replace DESTROY_BY_RCU with SLAB_TYPESAFE_BY_RCU because SLAB_DESTROY_BY_RCU has been renamed to SLAB_TYPESAFE_BY_RCU by commit 5f0d5a3ae7cf ("mm: Rename SLAB_DESTROY_BY_RCU to SLAB_TYPESAFE_BY_RCU") Link: http://lkml.kernel.org/r/20191017093554.22562-1-miles.chen@mediatek.com Signed-off-by: Miles Chen Cc: Paul E. McKenney Signed-off-by: Andrew Morton --- mm/rmap.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) --- a/mm/rmap.c~mm-fix-outdated-comment-in-page_get_anon_vma +++ a/mm/rmap.c @@ -477,9 +477,10 @@ void __init anon_vma_init(void) * chain and verify that the page in question is indeed mapped in it * [ something equivalent to page_mapped_in_vma() ]. * - * Since anon_vma's slab is DESTROY_BY_RCU and we know from page_remove_rmap() - * that the anon_vma pointer from page->mapping is valid if there is a - * mapcount, we can dereference the anon_vma after observing those. + * Since anon_vma's slab is SLAB_TYPESAFE_BY_RCU and we know from + * page_remove_rmap() that the anon_vma pointer from page->mapping is valid + * if there is a mapcount, we can dereference the anon_vma after observing + * those. */ struct anon_vma *page_get_anon_vma(struct page *page) { From patchwork Sun Dec 1 01:51:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268263 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1F812112B for ; Sun, 1 Dec 2019 01:51:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DDDB32084D for ; Sun, 1 Dec 2019 01:51:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="oI5ShyzO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DDDB32084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 38D4C6B02AC; Sat, 30 Nov 2019 20:51:28 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 33A8F6B02AE; Sat, 30 Nov 2019 20:51:28 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 252CB6B02AF; Sat, 30 Nov 2019 20:51:28 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0029.hostedemail.com [216.40.44.29]) by kanga.kvack.org (Postfix) with ESMTP id 0DD8A6B02AC for ; Sat, 30 Nov 2019 20:51:28 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id B9EB52C7C for ; Sun, 1 Dec 2019 01:51:27 +0000 (UTC) X-FDA: 76214895414.26.foot92_cbc14fc33335 X-Spam-Summary: 2,0,0,0a28a8a1d8ddf5aa,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:kirill.shutemov@linux.intel.com::mm-commits@vger.kernel.org:torvalds@linux-foundation.org:yang.shi@linux.alibaba.com,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3867:3868:3870:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4605:5007:6261:6653:7576:7808:7903:8660:9025:9545:10004:10913:11026:11232:11473:11658:11914:12043:12048:12114:12219:12297:12438:12517:12519:12555:12679:12783:12986:13069:13148:13161:13229:13230:13311:13357:14096:14181:14384:14721:14777:14849:21060:21080:21433:21451:21627:21819:21939:30003:30054:30074,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:n one X-HE-Tag: foot92_cbc14fc33335 X-Filterd-Recvd-Size: 2816 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf28.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:27 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 8B8512073C; Sun, 1 Dec 2019 01:51:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165086; bh=o0nGCfGrsxFwVvw8uha8izyjktMSf2bMS72f8ZGXQrw=; h=Date:From:To:Subject:From; b=oI5ShyzOqolqAA1VDHZHqI0/Ve1Y7fEcuTuJYfdEUYDBV9YrI6zqzpJsdhgw+U00l VvK+trH6wMSTrjr7f8qLVvOnIYZ4dwKD16w8KT4WAxpZ3oFWk+eFM6XO1buH0vqAoR O//LVxPOpkX7S8LcsxhJlqWhBFEq9gCAN4hlolXk= Date: Sat, 30 Nov 2019 17:51:26 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, yang.shi@linux.alibaba.com Subject: [patch 043/158] mm/rmap.c: use VM_BUG_ON_PAGE() in __page_check_anon_rmap() Message-ID: <20191201015126.m8w2PZPj7%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Shi Subject: mm/rmap.c: use VM_BUG_ON_PAGE() in __page_check_anon_rmap() The __page_check_anon_rmap() just calls two BUG_ON()s protected by CONFIG_DEBUG_VM, the #ifdef could be eliminated by using VM_BUG_ON_PAGE(). Link: http://lkml.kernel.org/r/1573157346-111316-1-git-send-email-yang.shi@linux.alibaba.com Signed-off-by: Yang Shi Acked-by: Kirill A. Shutemov Signed-off-by: Andrew Morton --- mm/rmap.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) --- a/mm/rmap.c~mm-rmap-use-vm_bug_on_page-in-__page_check_anon_rmap +++ a/mm/rmap.c @@ -1075,7 +1075,6 @@ static void __page_set_anon_rmap(struct static void __page_check_anon_rmap(struct page *page, struct vm_area_struct *vma, unsigned long address) { -#ifdef CONFIG_DEBUG_VM /* * The page's anon-rmap details (mapping and index) are guaranteed to * be set up correctly at this point. @@ -1088,9 +1087,9 @@ static void __page_check_anon_rmap(struc * are initially only visible via the pagetables, and the pte is locked * over the call to page_add_new_anon_rmap. */ - BUG_ON(page_anon_vma(page)->root != vma->anon_vma->root); - BUG_ON(page_to_pgoff(page) != linear_page_index(vma, address)); -#endif + VM_BUG_ON_PAGE(page_anon_vma(page)->root != vma->anon_vma->root, page); + VM_BUG_ON_PAGE(page_to_pgoff(page) != linear_page_index(vma, address), + page); } /** From patchwork Sun Dec 1 01:51:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268265 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B8CE3112B for ; Sun, 1 Dec 2019 01:51:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 79CAF2084D for ; Sun, 1 Dec 2019 01:51:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="YHBQzEXM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 79CAF2084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1B5C26B02AE; Sat, 30 Nov 2019 20:51:31 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 13E616B02B0; Sat, 30 Nov 2019 20:51:31 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 07D7B6B02B1; Sat, 30 Nov 2019 20:51:31 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0146.hostedemail.com [216.40.44.146]) by kanga.kvack.org (Postfix) with ESMTP id E3DA26B02AE for ; Sat, 30 Nov 2019 20:51:30 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id AF5322A87 for ; Sun, 1 Dec 2019 01:51:30 +0000 (UTC) X-FDA: 76214895540.12.help90_d2a44a10c330 X-Spam-Summary: 2,0,0,8dfc2beb533b72d3,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:arnd@arndb.de:kirill.shutemov@linux.intel.com::mm-commits@vger.kernel.org:thellstrom@vmware.com:torvalds@linux-foundation.org:willy@infradead.org,RULES_HIT:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:1978:1981:2194:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4605:5007:6261:6653:7576:8599:9025:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:13255:13846:14181:14721:14849:21063:21080:21450:21451:21627:21939:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: help90_d2a44a10c330 X-Filterd-Recvd-Size: 3502 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:30 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 8468F20880; Sun, 1 Dec 2019 01:51:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165089; bh=bUXwF6eV5LIVskZv5Ch3GcZqCraW5n5E51px2umeUKI=; h=Date:From:To:Subject:From; b=YHBQzEXMlmRM4E0NBhrPr+kom8j/aX1r5dZiFBFDrcuZggB/kHHXalu86Uq/kLli3 kHeiCqwUWIdx8uxYSuaqUHmBDaWlmL8nNPqI/4hSseTvMBGLvLrOCw+VOzbfVAQAyq Ok9wQhUcyLzbL5qJWYt/kTs/MXvAqzmx5yQjAzZw= Date: Sat, 30 Nov 2019 17:51:29 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, arnd@arndb.de, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, thellstrom@vmware.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 044/158] mm: move the backup x_devmap() functions to asm-generic/pgtable.h Message-ID: <20191201015129.jcMD1y0DQ%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Thomas Hellstrom Subject: mm: move the backup x_devmap() functions to asm-generic/pgtable.h The asm-generic/pgtable.h include file appears to be the correct place for the backup x_devmap() inline functions. Moving them here is also necessary if we want to include x_devmap() in the [pmd|pud]_unstable functions. So move the x_devmap() functions to asm-generic/pgtable.h Link: http://lkml.kernel.org/r/20191115115808.21181-1-thomas_os@shipmail.org Signed-off-by: Thomas Hellstrom Cc: Arnd Bergmann Cc: "Kirill A. Shutemov" Cc: Matthew Wilcox Signed-off-by: Andrew Morton --- include/asm-generic/pgtable.h | 15 +++++++++++++++ include/linux/mm.h | 15 --------------- 2 files changed, 15 insertions(+), 15 deletions(-) --- a/include/asm-generic/pgtable.h~mm-move-the-backup-x_devmap-functions-to-asm-generic-pgtableh +++ a/include/asm-generic/pgtable.h @@ -914,6 +914,21 @@ static inline int pud_write(pud_t pud) } #endif /* pud_write */ +#if !defined(CONFIG_ARCH_HAS_PTE_DEVMAP) || !defined(CONFIG_TRANSPARENT_HUGEPAGE) +static inline int pmd_devmap(pmd_t pmd) +{ + return 0; +} +static inline int pud_devmap(pud_t pud) +{ + return 0; +} +static inline int pgd_devmap(pgd_t pgd) +{ + return 0; +} +#endif + #if !defined(CONFIG_TRANSPARENT_HUGEPAGE) || \ (defined(CONFIG_TRANSPARENT_HUGEPAGE) && \ !defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD)) --- a/include/linux/mm.h~mm-move-the-backup-x_devmap-functions-to-asm-generic-pgtableh +++ a/include/linux/mm.h @@ -564,21 +564,6 @@ int vma_is_stack_for_current(struct vm_a struct mmu_gather; struct inode; -#if !defined(CONFIG_ARCH_HAS_PTE_DEVMAP) || !defined(CONFIG_TRANSPARENT_HUGEPAGE) -static inline int pmd_devmap(pmd_t pmd) -{ - return 0; -} -static inline int pud_devmap(pud_t pud) -{ - return 0; -} -static inline int pgd_devmap(pgd_t pgd) -{ - return 0; -} -#endif - /* * FIXME: take this include out, include page-flags.h in * files which need it (119 of them) From patchwork Sun Dec 1 01:51:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268267 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E7C0B921 for ; Sun, 1 Dec 2019 01:51:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AA7102084D for ; Sun, 1 Dec 2019 01:51:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="THT/rLMx" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AA7102084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 759A56B02B0; Sat, 30 Nov 2019 20:51:34 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7095E6B02B2; Sat, 30 Nov 2019 20:51:34 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6240A6B02B3; Sat, 30 Nov 2019 20:51:34 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0226.hostedemail.com [216.40.44.226]) by kanga.kvack.org (Postfix) with ESMTP id 4C7AD6B02B0 for ; Sat, 30 Nov 2019 20:51:34 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 05A8F181AEF0B for ; Sun, 1 Dec 2019 01:51:34 +0000 (UTC) X-FDA: 76214895708.24.mouth29_da4590faae21 X-Spam-Summary: 2,0,0,236cb11625a3ebfe,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:arnd@arndb.de:kirill.shutemov@linux.intel.com::mm-commits@vger.kernel.org:stable@vger.kernel.org:thellstrom@vmware.com:torvalds@linux-foundation.org:willy@infradead.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1543:1711:1730:1747:1777:1792:1801:1981:2194:2199:2393:2525:2559:2563:2682:2685:2691:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3865:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4605:5007:6119:6261:6653:7576:7903:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12291:12297:12438:12517:12519:12555:12679:12783:12986:13180:13229:13846:14039:14093:14096:14181:14721:14849:21063:21080:21450:21451:21627:21795:21939:30003:30012:30054:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,M SBL:0,DN X-HE-Tag: mouth29_da4590faae21 X-Filterd-Recvd-Size: 4840 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:33 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B0102215A5; Sun, 1 Dec 2019 01:51:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165093; bh=5YG5SJgJYRkpy/Jj3Zis4wmBxE1XZBfNejKoHe/VckI=; h=Date:From:To:Subject:From; b=THT/rLMxIS3YchSgfxZz5Oh8h0BG/75EblcSjLXA2wQcK/Us5+xX8aJngbmv6uxAp YKZx7+OCYsQnOacpLv7wud6vZjmFBNfaaBjUAp9gIr0uacqeIwuwadh61cyOXtjfvq VBIS+WypFFnQ8FHZVtOODQXuCHgWMZKSfClKfChw= Date: Sat, 30 Nov 2019 17:51:32 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, arnd@arndb.de, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, stable@vger.kernel.org, thellstrom@vmware.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 045/158] mm/memory.c: fix a huge pud insertion race during faulting Message-ID: <20191201015132.LxbEPLRfg%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Thomas Hellstrom Subject: mm/memory.c: fix a huge pud insertion race during faulting A huge pud page can theoretically be faulted in racing with pmd_alloc() in __handle_mm_fault(). That will lead to pmd_alloc() returning an invalid pmd pointer. Fix this by adding a pud_trans_unstable() function similar to pmd_trans_unstable() and check whether the pud is really stable before using the pmd pointer. Race: Thread 1: Thread 2: Comment create_huge_pud() Fallback - not taken. create_huge_pud() Taken. pmd_alloc() Returns an invalid pointer. This will result in user-visible huge page data corruption. Note that this was caught during a code audit rather than a real experienced problem. It looks to me like the only implementation that currently creates huge pud pagetable entries is dev_dax_huge_fault() which doesn't appear to care much about private (COW) mappings or write-tracking which is, I believe, a prerequisite for create_huge_pud() falling back on thread 1, but not in thread 2. Link: http://lkml.kernel.org/r/20191115115808.21181-2-thomas_os@shipmail.org Fixes: a00cc7d9dd93 ("mm, x86: add support for PUD-sized transparent hugepages") Signed-off-by: Thomas Hellstrom Acked-by: Kirill A. Shutemov Cc: Arnd Bergmann Cc: Matthew Wilcox Cc: Signed-off-by: Andrew Morton --- include/asm-generic/pgtable.h | 25 +++++++++++++++++++++++++ mm/memory.c | 6 ++++++ 2 files changed, 31 insertions(+) --- a/include/asm-generic/pgtable.h~mm-fix-a-huge-pud-insertion-race-during-faulting +++ a/include/asm-generic/pgtable.h @@ -938,6 +938,31 @@ static inline int pud_trans_huge(pud_t p } #endif +/* See pmd_none_or_trans_huge_or_clear_bad for discussion. */ +static inline int pud_none_or_trans_huge_or_dev_or_clear_bad(pud_t *pud) +{ + pud_t pudval = READ_ONCE(*pud); + + if (pud_none(pudval) || pud_trans_huge(pudval) || pud_devmap(pudval)) + return 1; + if (unlikely(pud_bad(pudval))) { + pud_clear_bad(pud); + return 1; + } + return 0; +} + +/* See pmd_trans_unstable for discussion. */ +static inline int pud_trans_unstable(pud_t *pud) +{ +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && \ + defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) + return pud_none_or_trans_huge_or_dev_or_clear_bad(pud); +#else + return 0; +#endif +} + #ifndef pmd_read_atomic static inline pmd_t pmd_read_atomic(pmd_t *pmdp) { --- a/mm/memory.c~mm-fix-a-huge-pud-insertion-race-during-faulting +++ a/mm/memory.c @@ -4010,6 +4010,7 @@ static vm_fault_t __handle_mm_fault(stru vmf.pud = pud_alloc(mm, p4d, address); if (!vmf.pud) return VM_FAULT_OOM; +retry_pud: if (pud_none(*vmf.pud) && __transparent_hugepage_enabled(vma)) { ret = create_huge_pud(&vmf); if (!(ret & VM_FAULT_FALLBACK)) @@ -4036,6 +4037,11 @@ static vm_fault_t __handle_mm_fault(stru vmf.pmd = pmd_alloc(mm, vmf.pud, address); if (!vmf.pmd) return VM_FAULT_OOM; + + /* Huge pud page fault raced with pmd_alloc? */ + if (pud_trans_unstable(vmf.pud)) + goto retry_pud; + if (pmd_none(*vmf.pmd) && __transparent_hugepage_enabled(vma)) { ret = create_huge_pmd(&vmf); if (!(ret & VM_FAULT_FALLBACK)) From patchwork Sun Dec 1 01:51:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268269 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 05172921 for ; Sun, 1 Dec 2019 01:51:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B6F76208C3 for ; Sun, 1 Dec 2019 01:51:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="hkgxusBf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B6F76208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9CC2C6B02B2; Sat, 30 Nov 2019 20:51:39 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9A1FD6B02B4; Sat, 30 Nov 2019 20:51:39 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DF396B02B5; Sat, 30 Nov 2019 20:51:39 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0056.hostedemail.com [216.40.44.56]) by kanga.kvack.org (Postfix) with ESMTP id 760EC6B02B2 for ; Sat, 30 Nov 2019 20:51:39 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 28787180AD817 for ; Sun, 1 Dec 2019 01:51:39 +0000 (UTC) X-FDA: 76214895918.16.able57_e61d3d83f405 X-Spam-Summary: 2,0,0,f10024382c9e4107,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:2:41:355:379:800:96 0:967:97 X-HE-Tag: able57_e61d3d83f405 X-Filterd-Recvd-Size: 7370 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:38 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 804442084D; Sun, 1 Dec 2019 01:51:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165098; bh=PGTiczPxnurPRNULcbLumtCE012mMbkW6yRFj/lyTe4=; h=Date:From:To:Subject:From; b=hkgxusBf80udsFDn0eQxN4ZO152RgG1lNI2E3bvxQpHblvku0a3SzpkKP0xRWK7R8 ptBUnmOf61owFKqp1C+5H58DqCed17iznNxzxhPiu4SLWLPUD/St13U0QRBPMaKsg/ u4iMatYUkbzYtdO8g7PVPYppKyI43XcJvNPWpmI8= Date: Sat, 30 Nov 2019 17:51:36 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, Mark.Rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 046/158] mm: add generic p?d_leaf() macros Message-ID: <20191201015136.hM2KwyVgH%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: mm: add generic p?d_leaf() macros Patch series "Generic page walk and ptdump", v15. Many architectures current have a debugfs file for dumping the kernel page tables. Currently each architecture has to implement custom functions for this because the details of walking the page tables used by the kernel are different between architectures. This series extends the capabilities of walk_page_range() so that it can deal with the page tables of the kernel (which have no VMAs and can contain larger huge pages than exist for user space). A generic PTDUMP implementation is the implemented making use of the new functionality of walk_page_range() and finally arm64 and x86 are switch to using it, removing the custom table walkers. To enable a generic page table walker to walk the unusual mappings of the kernel we need to implement a set of functions which let us know when the walker has reached the leaf entry. After a suggestion from Will Deacon I've chosen the name p?d_leaf() as this (hopefully) describes the purpose (and is a new name so has no historic baggage). Some architectures have p?d_large macros but this is easily confused with "large pages". This series ends with a generic PTDUMP implemention for arm64 and x86. Mostly this is a clean up and there should be very little functional change. The exceptions are: * arm64 PTDUMP debugfs now displays pages which aren't present (patch 22). * arm64 has the ability to efficiently process KASAN pages (which previously only x86 implemented). This means that the combination of KASAN and DEBUG_WX is now useable. This patch (of 23): Exposing the pud/pgd levels of the page tables to walk_page_range() means we may come across the exotic large mappings that come with large areas of contiguous memory (such as the kernel's linear map). For architectures that don't provide all p?d_leaf() macros, provide generic do nothing default that are suitable where there cannot be leaf pages at that level. Futher patches will add implementations for individual architectures. The name p?d_leaf() is chosen to minimize the confusion with existing uses of "large" pages and "huge" pages which do not necessary mean that the entry is a leaf (for example it may be a set of contiguous entries that only take 1 TLB slot). For the purpose of walking the page tables we don't need to know how it will be represented in the TLB, but we do need to know for sure if it is a leaf of the tree. Link: http://lkml.kernel.org/r/20191028135910.33253-2-steven.price@arm.com Signed-off-by: Steven Price Acked-by: Mark Rutland Cc: Albert Ou Cc: Alexandre Ghiti Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Michael Ellerman Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Cc: Zong Li Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Ingo Molnar Cc: Matthew Wilcox Cc: Naoya Horiguchi Cc: Shiraz Hashim Signed-off-by: Andrew Morton --- include/asm-generic/pgtable.h | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) --- a/include/asm-generic/pgtable.h~mm-add-generic-pd_leaf-macros +++ a/include/asm-generic/pgtable.h @@ -1238,4 +1238,24 @@ static inline bool arch_has_pfn_modify_c #define mm_pmd_folded(mm) __is_defined(__PAGETABLE_PMD_FOLDED) #endif +/* + * p?d_leaf() - true if this entry is a final mapping to a physical address. + * This differs from p?d_huge() by the fact that they are always available (if + * the architecture supports large pages at the appropriate level) even + * if CONFIG_HUGETLB_PAGE is not defined. + * Only meaningful when called on a valid entry. + */ +#ifndef pgd_leaf +#define pgd_leaf(x) 0 +#endif +#ifndef p4d_leaf +#define p4d_leaf(x) 0 +#endif +#ifndef pud_leaf +#define pud_leaf(x) 0 +#endif +#ifndef pmd_leaf +#define pmd_leaf(x) 0 +#endif + #endif /* _ASM_GENERIC_PGTABLE_H */ From patchwork Sun Dec 1 01:51:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268271 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BF960112B for ; Sun, 1 Dec 2019 01:51:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8D5C6215A5 for ; Sun, 1 Dec 2019 01:51:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="lA2R3AFt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8D5C6215A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 892B96B02B4; Sat, 30 Nov 2019 20:51:44 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 868DF6B02B6; Sat, 30 Nov 2019 20:51:44 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A6E06B02B7; Sat, 30 Nov 2019 20:51:44 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0068.hostedemail.com [216.40.44.68]) by kanga.kvack.org (Postfix) with ESMTP id 654EF6B02B4 for ; Sat, 30 Nov 2019 20:51:44 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 1DFCB2C7C for ; Sun, 1 Dec 2019 01:51:44 +0000 (UTC) X-FDA: 76214896128.04.burn37_f1b1a17ecd62 X-Spam-Summary: 2,0,0,108ceef6416619f7,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:41:355:379:800:960: 967:973: X-HE-Tag: burn37_f1b1a17ecd62 X-Filterd-Recvd-Size: 4871 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:43 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 92706208C3; Sun, 1 Dec 2019 01:51:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165103; bh=lFumLNgbtZMUa/Sg36gg+en8R9dqTD3qsc3yKiIH1wc=; h=Date:From:To:Subject:From; b=lA2R3AFt0bFij6rCZCfcMHq/AVU3ykOy6o6ZT5lGPx1flAZ5S02/j8bE5mXI8jJ/T rU4G5aOaoKcUWijyOiyhTTNwQndKxMsnmzK4syru4eeTEqAXB5eh+/crpkvXfX77QQ mHBYSxOLDB7uSRBwdpokFRpJHSAVAOJB/rieMu8Y= Date: Sat, 30 Nov 2019 17:51:41 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 047/158] arc: mm: add p?d_leaf() definitions Message-ID: <20191201015141.gKSb9TXsh%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: arc: mm: add p?d_leaf() definitions walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information will be provided by the p?d_leaf() functions/macros. For arc, we only have two levels, so only pmd_leaf() is needed. Link: http://lkml.kernel.org/r/20191028135910.33253-3-steven.price@arm.com Signed-off-by: Steven Price Acked-by: Vineet Gupta Cc: Vineet Gupta Cc: linux-snps-arc@lists.infradead.org Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Will Deacon Cc: Zong Li Signed-off-by: Andrew Morton --- arch/arc/include/asm/pgtable.h | 1 + 1 file changed, 1 insertion(+) --- a/arch/arc/include/asm/pgtable.h~arc-mm-add-pd_leaf-definitions +++ a/arch/arc/include/asm/pgtable.h @@ -273,6 +273,7 @@ static inline void pmd_set(pmd_t *pmdp, #define pmd_none(x) (!pmd_val(x)) #define pmd_bad(x) ((pmd_val(x) & ~PAGE_MASK)) #define pmd_present(x) (pmd_val(x)) +#define pmd_leaf(x) (pmd_val(x) & _PAGE_HW_SZ) #define pmd_clear(xp) do { pmd_val(*(xp)) = 0; } while (0) #define pte_page(pte) pfn_to_page(pte_pfn(pte)) From patchwork Sun Dec 1 01:51:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268273 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F053C112B for ; Sun, 1 Dec 2019 01:51:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BDA47215E5 for ; Sun, 1 Dec 2019 01:51:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="ADbNAVRp" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BDA47215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B02F66B02B6; Sat, 30 Nov 2019 20:51:49 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AB3B26B02B8; Sat, 30 Nov 2019 20:51:49 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F1076B02B9; Sat, 30 Nov 2019 20:51:49 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0146.hostedemail.com [216.40.44.146]) by kanga.kvack.org (Postfix) with ESMTP id 890646B02B6 for ; Sat, 30 Nov 2019 20:51:49 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 44B6E824999B for ; Sun, 1 Dec 2019 01:51:49 +0000 (UTC) X-FDA: 76214896338.20.beam29_fcf50031a04d X-Spam-Summary: 2,0,0,41ac4bcca380a2ee,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:41:355:379:800:960: 967:973: X-HE-Tag: beam29_fcf50031a04d X-Filterd-Recvd-Size: 5160 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:48 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 70694208E4; Sun, 1 Dec 2019 01:51:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165107; bh=Mr4LHEdtiKHOdFKx9nuumKPhm0SaAjaOuE1KjxxLWNk=; h=Date:From:To:Subject:From; b=ADbNAVRpAd209xtIg3zQ4gU73FkgND0fDlw4/PTf90U7/D2P9uLcis7ms0majftQC XEGQS0XrKpO2ad1rJhIhLcoVHIRN2MFzoI0o9WQyGzmBj7Rscs7kB5VZv0bUMupUtp lD+gEGZO0S4bqPf4hy5FhB/MK0RJ7cY17F0AJAHE= Date: Sat, 30 Nov 2019 17:51:46 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 048/158] arm: mm: add p?d_leaf() definitions Message-ID: <20191201015146.IL3mB9ych%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: arm: mm: add p?d_leaf() definitions walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information is provided by the p?d_leaf() functions/macros. For arm pmd_large() already exists and does what we want. So simply provide the generic pmd_leaf() name. Link: http://lkml.kernel.org/r/20191028135910.33253-4-steven.price@arm.com Signed-off-by: Steven Price Cc: Russell King Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: Ingo Molnar <.com> Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Cc: Zong Li Signed-off-by: Andrew Morton --- arch/arm/include/asm/pgtable-2level.h | 1 + arch/arm/include/asm/pgtable-3level.h | 1 + 2 files changed, 2 insertions(+) --- a/arch/arm/include/asm/pgtable-2level.h~arm-mm-add-pd_leaf-definitions +++ a/arch/arm/include/asm/pgtable-2level.h @@ -189,6 +189,7 @@ static inline pmd_t *pmd_offset(pud_t *p } #define pmd_large(pmd) (pmd_val(pmd) & 2) +#define pmd_leaf(pmd) (pmd_val(pmd) & 2) #define pmd_bad(pmd) (pmd_val(pmd) & 2) #define pmd_present(pmd) (pmd_val(pmd)) --- a/arch/arm/include/asm/pgtable-3level.h~arm-mm-add-pd_leaf-definitions +++ a/arch/arm/include/asm/pgtable-3level.h @@ -134,6 +134,7 @@ #define pmd_sect(pmd) ((pmd_val(pmd) & PMD_TYPE_MASK) == \ PMD_TYPE_SECT) #define pmd_large(pmd) pmd_sect(pmd) +#define pmd_leaf(pmd) pmd_sect(pmd) #define pud_clear(pudp) \ do { \ From patchwork Sun Dec 1 01:51:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268275 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9C5A3921 for ; Sun, 1 Dec 2019 01:51:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5E83F21774 for ; Sun, 1 Dec 2019 01:51:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="kb1aWg63" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5E83F21774 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 29E5E6B02B8; Sat, 30 Nov 2019 20:51:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 275266B02BA; Sat, 30 Nov 2019 20:51:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1B3376B02BB; Sat, 30 Nov 2019 20:51:54 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0101.hostedemail.com [216.40.44.101]) by kanga.kvack.org (Postfix) with ESMTP id 065196B02B8 for ; Sat, 30 Nov 2019 20:51:54 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id A513340DD for ; Sun, 1 Dec 2019 01:51:53 +0000 (UTC) X-FDA: 76214896506.11.pear00_107d94c60ab3b X-Spam-Summary: 2,0,0,43323b214dfb355f,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:41:355:379:800:960: 967:973: X-HE-Tag: pear00_107d94c60ab3b X-Filterd-Recvd-Size: 5616 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:53 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2B208215E5; Sun, 1 Dec 2019 01:51:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165112; bh=+rWStZHOQUaKVd1HMBjbB8DIVfN9At0UnY87VRFj5WA=; h=Date:From:To:Subject:From; b=kb1aWg63gof8YHxsqwDf5Pi8TKgw/mXS+7/cCfGi0milgLprJWuSdEuKGMvqoEt8e I5eqDRnBRfTbMsMM9Xwas7NuW/wuI2U6qa1pAUzovgYSvJvoSJfijhifKn8oRlNSsl pDGwlxkgpWfAZB7uQzA90zHobgyF2NaUhHTKjeO0= Date: Sat, 30 Nov 2019 17:51:50 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 049/158] arm64: mm: add p?d_leaf() definitions Message-ID: <20191201015150.ndjW9yxI4%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: arm64: mm: add p?d_leaf() definitions walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information will be provided by the p?d_leaf() functions/macros. For arm64, we already have p?d_sect() macros which we can reuse for p?d_leaf(). pud_sect() is defined as a dummy function when CONFIG_PGTABLE_LEVELS < 3 or CONFIG_ARM64_64K_PAGES is defined. However when the kernel is configured this way then architecturally it isn't allowed to have a large page at this level, and any code using these page walking macros is implicitly relying on the page size/number of levels being the same as the kernel. So it is safe to reuse this for p?d_leaf() as it is an architectural restriction. Link: http://lkml.kernel.org/r/20191028135910.33253-5-steven.price@arm.com Signed-off-by: Steven Price Acked-by: Catalin Marinas Cc: Will Deacon Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Zong Li Signed-off-by: Andrew Morton --- arch/arm64/include/asm/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) --- a/arch/arm64/include/asm/pgtable.h~arm64-mm-add-pd_leaf-definitions +++ a/arch/arm64/include/asm/pgtable.h @@ -445,6 +445,7 @@ extern pgprot_t phys_mem_access_prot(str PMD_TYPE_TABLE) #define pmd_sect(pmd) ((pmd_val(pmd) & PMD_TYPE_MASK) == \ PMD_TYPE_SECT) +#define pmd_leaf(pmd) pmd_sect(pmd) #if defined(CONFIG_ARM64_64K_PAGES) || CONFIG_PGTABLE_LEVELS < 3 static inline bool pud_sect(pud_t pud) { return false; } @@ -529,6 +530,7 @@ static inline void pte_unmap(pte_t *pte) #define pud_none(pud) (!pud_val(pud)) #define pud_bad(pud) (!(pud_val(pud) & PUD_TABLE_BIT)) #define pud_present(pud) pte_present(pud_pte(pud)) +#define pud_leaf(pud) pud_sect(pud) #define pud_valid(pud) pte_valid(pud_pte(pud)) static inline void set_pud(pud_t *pudp, pud_t pud) From patchwork Sun Dec 1 01:51:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268277 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A8803112B for ; Sun, 1 Dec 2019 01:52:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6A41A20880 for ; Sun, 1 Dec 2019 01:52:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="tFmAadwe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6A41A20880 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 362C56B02BA; Sat, 30 Nov 2019 20:51:59 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 338CE6B02BC; Sat, 30 Nov 2019 20:51:59 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24F2C6B02BD; Sat, 30 Nov 2019 20:51:59 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0180.hostedemail.com [216.40.44.180]) by kanga.kvack.org (Postfix) with ESMTP id 0E9406B02BA for ; Sat, 30 Nov 2019 20:51:59 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id B665B180AD817 for ; Sun, 1 Dec 2019 01:51:58 +0000 (UTC) X-FDA: 76214896716.22.smash07_113b5f4dcb617 X-Spam-Summary: 2,0,0,486c929b0d659d4c,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:41:355:379:800:960: 967:973: X-HE-Tag: smash07_113b5f4dcb617 X-Filterd-Recvd-Size: 4998 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:51:58 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 02266206E1; Sun, 1 Dec 2019 01:51:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165117; bh=oeL9/OLucOo0B18VzNVkAnYOw9gXgpKLMsHtWDpyPRs=; h=Date:From:To:Subject:From; b=tFmAadweW4Nq/KqU0Q/xYVLZZc+lHCYloxFszVPSJJYQvOn2OFbm+sl2MDSJ7fngX ZOeaVa12PkQcrfrahhFcWXBuVs/Hv5tZLnnIguqOl6zvKHOZjXy8nbMORZx4YnGFXD S75ff0Gnn9VMx52oqFM/PWhm6VML24bFn5Q9D4sM= Date: Sat, 30 Nov 2019 17:51:55 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 050/158] mips: mm: add p?d_leaf() definitions Message-ID: <20191201015155.Ks4HHk5Gg%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: mips: mm: add p?d_leaf() definitions walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information is provided by the p?d_leaf() functions/macros. If _PAGE_HUGE is defined we can simply look for it. When not defined we can be confident that there are no leaf pages in existence and fall back on the generic implementation (added in a later patch) which returns 0. Link: http://lkml.kernel.org/r/20191028135910.33253-6-steven.price@arm.com Signed-off-by: Steven Price Acked-by: Paul Burton Cc: Ralf Baechle Cc: Paul Burton Cc: James Hogan Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Russell King Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Cc: Zong Li Signed-off-by: Andrew Morton --- arch/mips/include/asm/pgtable.h | 5 +++++ 1 file changed, 5 insertions(+) --- a/arch/mips/include/asm/pgtable.h~mips-mm-add-pd_leaf-definitions +++ a/arch/mips/include/asm/pgtable.h @@ -639,6 +639,11 @@ static inline pmd_t pmdp_huge_get_and_cl #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#ifdef _PAGE_HUGE +#define pmd_leaf(pmd) ((pmd_val(pmd) & _PAGE_HUGE) != 0) +#define pud_leaf(pud) ((pud_val(pud) & _PAGE_HUGE) != 0) +#endif + #define gup_fast_permitted(start, end) (!cpu_has_dc_aliases) #include From patchwork Sun Dec 1 01:52:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268279 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60081112B for ; Sun, 1 Dec 2019 01:52:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 237EC2084D for ; Sun, 1 Dec 2019 01:52:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="s5+NqW38" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 237EC2084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D4DE46B02BC; Sat, 30 Nov 2019 20:52:03 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D24576B02BE; Sat, 30 Nov 2019 20:52:03 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C60F86B02BF; Sat, 30 Nov 2019 20:52:03 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0082.hostedemail.com [216.40.44.82]) by kanga.kvack.org (Postfix) with ESMTP id B076B6B02BC for ; Sat, 30 Nov 2019 20:52:03 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 5FFD6181AEF0B for ; Sun, 1 Dec 2019 01:52:03 +0000 (UTC) X-FDA: 76214896926.03.fear39_11e584e188433 X-Spam-Summary: 2,0,0,e2fc65db9849f918,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:41:355:379:800:960: 967:973: X-HE-Tag: fear39_11e584e188433 X-Filterd-Recvd-Size: 6347 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:52:02 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E122E20880; Sun, 1 Dec 2019 01:52:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165122; bh=Rj1aqpycMgmyKuhbCSNqmh+Dg0UXqi9ILuoogIJ9lfk=; h=Date:From:To:Subject:From; b=s5+NqW38qyUHsWwv1gtOjOEViqg0+uIsMC4uikhNmjyQO25JGV2aWmsVqCwu5v9P+ 1m13RU6GtAfkxEFAC8b+9zbMzOqRLsMzvx0U3xfHUodIwoubsxlx86cWtUd0nBJIkN 6Pu3RWd1Jxc03cshmInMW95XvcsgliRNIyV22gVg= Date: Sat, 30 Nov 2019 17:52:00 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 051/158] powerpc: mm: add p?d_leaf() definitions Message-ID: <20191201015200.6JoiLDYzh%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: powerpc: mm: add p?d_leaf() definitions walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information is provided by the p?d_leaf() functions/macros. For powerpc pmd_large() already exists and does what we want, so hoist it out of the CONFIG_TRANSPARENT_HUGEPAGE condition and implement the other levels. Macros are used to provide the generic p?d_leaf() names. Link: http://lkml.kernel.org/r/20191028135910.33253-7-steven.price@arm.com Signed-off-by: Steven Price Cc: Benjamin Herrenschmidt Cc: Paul Mackerras Cc: Michael Ellerman Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Cc: Zong Li Signed-off-by: Andrew Morton --- arch/powerpc/include/asm/book3s/64/pgtable.h | 30 +++++++++++------ 1 file changed, 21 insertions(+), 9 deletions(-) --- a/arch/powerpc/include/asm/book3s/64/pgtable.h~powerpc-mm-add-pd_leaf-definitions +++ a/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -923,6 +923,12 @@ static inline int pud_present(pud_t pud) return !!(pud_raw(pud) & cpu_to_be64(_PAGE_PRESENT)); } +#define pud_leaf pud_large +static inline int pud_large(pud_t pud) +{ + return !!(pud_raw(pud) & cpu_to_be64(_PAGE_PTE)); +} + extern struct page *pud_page(pud_t pud); extern struct page *pmd_page(pmd_t pmd); static inline pte_t pud_pte(pud_t pud) @@ -966,6 +972,12 @@ static inline int pgd_present(pgd_t pgd) return !!(pgd_raw(pgd) & cpu_to_be64(_PAGE_PRESENT)); } +#define pgd_leaf pgd_large +static inline int pgd_large(pgd_t pgd) +{ + return !!(pgd_raw(pgd) & cpu_to_be64(_PAGE_PTE)); +} + static inline pte_t pgd_pte(pgd_t pgd) { return __pte_raw(pgd_raw(pgd)); @@ -1133,6 +1145,15 @@ static inline bool pmd_access_permitted( return pte_access_permitted(pmd_pte(pmd), write); } +#define pmd_leaf pmd_large +/* + * returns true for pmd migration entries, THP, devmap, hugetlb + */ +static inline int pmd_large(pmd_t pmd) +{ + return !!(pmd_raw(pmd) & cpu_to_be64(_PAGE_PTE)); +} + #ifdef CONFIG_TRANSPARENT_HUGEPAGE extern pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot); extern pmd_t mk_pmd(struct page *page, pgprot_t pgprot); @@ -1159,15 +1180,6 @@ pmd_hugepage_update(struct mm_struct *mm return hash__pmd_hugepage_update(mm, addr, pmdp, clr, set); } -/* - * returns true for pmd migration entries, THP, devmap, hugetlb - * But compile time dependent on THP config - */ -static inline int pmd_large(pmd_t pmd) -{ - return !!(pmd_raw(pmd) & cpu_to_be64(_PAGE_PTE)); -} - static inline pmd_t pmd_mknotpresent(pmd_t pmd) { return __pmd(pmd_val(pmd) & ~_PAGE_PRESENT); From patchwork Sun Dec 1 01:52:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268281 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0A75B921 for ; Sun, 1 Dec 2019 01:52:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CBFDD206E1 for ; Sun, 1 Dec 2019 01:52:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="woZhK8yi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CBFDD206E1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A3E856B02BE; Sat, 30 Nov 2019 20:52:08 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9F0DE6B02C0; Sat, 30 Nov 2019 20:52:08 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92D0D6B02C1; Sat, 30 Nov 2019 20:52:08 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0111.hostedemail.com [216.40.44.111]) by kanga.kvack.org (Postfix) with ESMTP id 751596B02BE for ; Sat, 30 Nov 2019 20:52:08 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 23911824999B for ; Sun, 1 Dec 2019 01:52:08 +0000 (UTC) X-FDA: 76214897136.06.dirt02_129a7e48bec49 X-Spam-Summary: 2,0,0,5b4af1c9f0c5e812,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:41:355:379:800:960: 967:973: X-HE-Tag: dirt02_129a7e48bec49 X-Filterd-Recvd-Size: 5364 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:52:07 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A32882084D; Sun, 1 Dec 2019 01:52:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165127; bh=VWFx12Xia1fe8FujdzrojNGOBomOjgGN/U2NcDB5o4s=; h=Date:From:To:Subject:From; b=woZhK8yiLwJSRD9PTrYBcVJz28cInv+Cgd5g7vWxNr9xSWuF4MERToAs3kuDuw5c0 Stos++bhMxgA4Hx0Y7C7aU3lJT+rH73Qge9IpAwK3Pr7EoLZc1ha5eI38B1rd9am/f JPaY4mNViwzbZSb2skRNnTHO0Hz6LIY5KV3ytNpY= Date: Sat, 30 Nov 2019 17:52:05 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 052/158] riscv: mm: add p?d_leaf() definitions Message-ID: <20191201015205.W9IBNhBj_%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: riscv: mm: add p?d_leaf() definitions walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information is provided by the p?d_leaf() functions/macros. For riscv a page is a leaf page when it has a read, write or execute bit set on it. Link: http://lkml.kernel.org/r/20191028135910.33253-8-steven.price@arm.com Signed-off-by: Steven Price Reviewed-by: Alexandre Ghiti Reviewed-by: Zong Li Acked-by: Paul Walmsley [arch/riscv] Cc: Palmer Dabbelt Cc: Albert Ou Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Paul Burton Cc: Paul Mackerras Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/riscv/include/asm/pgtable-64.h | 7 +++++++ arch/riscv/include/asm/pgtable.h | 7 +++++++ 2 files changed, 14 insertions(+) --- a/arch/riscv/include/asm/pgtable-64.h~riscv-mm-add-pd_leaf-definitions +++ a/arch/riscv/include/asm/pgtable-64.h @@ -43,6 +43,13 @@ static inline int pud_bad(pud_t pud) return !pud_present(pud); } +#define pud_leaf pud_leaf +static inline int pud_leaf(pud_t pud) +{ + return pud_present(pud) && + (pud_val(pud) & (_PAGE_READ | _PAGE_WRITE | _PAGE_EXEC)); +} + static inline void set_pud(pud_t *pudp, pud_t pud) { *pudp = pud; --- a/arch/riscv/include/asm/pgtable.h~riscv-mm-add-pd_leaf-definitions +++ a/arch/riscv/include/asm/pgtable.h @@ -105,6 +105,13 @@ static inline int pmd_bad(pmd_t pmd) return !pmd_present(pmd); } +#define pmd_leaf pmd_leaf +static inline int pmd_leaf(pmd_t pmd) +{ + return pmd_present(pmd) && + (pmd_val(pmd) & (_PAGE_READ | _PAGE_WRITE | _PAGE_EXEC)); +} + static inline void set_pmd(pmd_t *pmdp, pmd_t pmd) { *pmdp = pmd; From patchwork Sun Dec 1 01:52:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268283 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F3ACA112B for ; Sun, 1 Dec 2019 01:52:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BFA40215E5 for ; Sun, 1 Dec 2019 01:52:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="EKxw3JDW" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BFA40215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 780776B02C0; Sat, 30 Nov 2019 20:52:13 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 757B36B02C2; Sat, 30 Nov 2019 20:52:13 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 695706B02C3; Sat, 30 Nov 2019 20:52:13 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0045.hostedemail.com [216.40.44.45]) by kanga.kvack.org (Postfix) with ESMTP id 50CC16B02C0 for ; Sat, 30 Nov 2019 20:52:13 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 01B90181AEF10 for ; Sun, 1 Dec 2019 01:52:13 +0000 (UTC) X-FDA: 76214897304.07.way29_134f2402f5021 X-Spam-Summary: 2,0,0,4c57eb17cf099bd9,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:41:355:379:800:960: 967:973: X-HE-Tag: way29_134f2402f5021 X-Filterd-Recvd-Size: 5040 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:52:12 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9CEA9206E1; Sun, 1 Dec 2019 01:52:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165132; bh=hg2ioVqjRk9AfqqPl2ctHXljgV7rZOvOBD+AtGdC1sE=; h=Date:From:To:Subject:From; b=EKxw3JDWyC7+EWTQBZTjyHG+pPirHJONoI0z3he3rUtjW/Rlpn05QquiQ05fqeY4o 3lvMdKyVg1mCVyuFCMeO0+6cqU3N5TZA/GNWy4KiTwO4E6grRcgTPfKEg+rYxEh3tI stiiIx1bHp1cINCkU7ck367KWQj6aYq84Tvbu/lk= Date: Sat, 30 Nov 2019 17:52:10 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 053/158] s390: mm: add p?d_leaf() definitions Message-ID: <20191201015210.3ZEhfXt41%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: s390: mm: add p?d_leaf() definitions walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information is provided by the p?d_leaf() functions/macros. For s390, pud_large() and pmd_large() are already implemented as static inline functions. Add a macro to provide the p?d_leaf names for the generic code to use. Link: http://lkml.kernel.org/r/20191028135910.33253-9-steven.price@arm.com Signed-off-by: Steven Price Cc: Heiko Carstens Cc: Vasily Gorbik Cc: Christian Borntraeger Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vineet Gupta Cc: Will Deacon Cc: Zong Li Signed-off-by: Andrew Morton --- arch/s390/include/asm/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) --- a/arch/s390/include/asm/pgtable.h~s390-mm-add-pd_leaf-definitions +++ a/arch/s390/include/asm/pgtable.h @@ -673,6 +673,7 @@ static inline int pud_none(pud_t pud) return pud_val(pud) == _REGION3_ENTRY_EMPTY; } +#define pud_leaf pud_large static inline int pud_large(pud_t pud) { if ((pud_val(pud) & _REGION_ENTRY_TYPE_MASK) != _REGION_ENTRY_TYPE_R3) @@ -690,6 +691,7 @@ static inline unsigned long pud_pfn(pud_ return (pud_val(pud) & origin_mask) >> PAGE_SHIFT; } +#define pmd_leaf pmd_large static inline int pmd_large(pmd_t pmd) { return (pmd_val(pmd) & _SEGMENT_ENTRY_LARGE) != 0; From patchwork Sun Dec 1 01:52:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268285 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E5368112B for ; Sun, 1 Dec 2019 01:52:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B23C72084D for ; Sun, 1 Dec 2019 01:52:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="D5XGx9s/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B23C72084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 70AB36B02C2; Sat, 30 Nov 2019 20:52:18 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6BB076B02C4; Sat, 30 Nov 2019 20:52:18 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F78F6B02C5; Sat, 30 Nov 2019 20:52:18 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0123.hostedemail.com [216.40.44.123]) by kanga.kvack.org (Postfix) with ESMTP id 4409C6B02C2 for ; Sat, 30 Nov 2019 20:52:18 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id EA57D180AD817 for ; Sun, 1 Dec 2019 01:52:17 +0000 (UTC) X-FDA: 76214897514.04.horn63_1406303f85807 X-Spam-Summary: 2,0,0,d333cef69eec3469,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:41:355:379:800:960: 967:973: X-HE-Tag: horn63_1406303f85807 X-Filterd-Recvd-Size: 5052 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:52:17 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 3ED78217AB; Sun, 1 Dec 2019 01:52:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165136; bh=yggEMZK3Ud9wFxPRNNnrIdqbf2FlATsEoHAKWLgn0nY=; h=Date:From:To:Subject:From; b=D5XGx9s/cpUhuyt97CEMIUCVOHx4BgwqXYfvlOaxdQMe2ZXLJOpR0EzuJ3ohifJnY 9WT/d4+8CnI5tnkrCWzjrdfRZOWr9lj/xko2OynsrL2cWkDV4ef8B5dpfyG573KdhU NJnU21C4RKJ3443as+CeL3f6a2qVcYKqnopOaMS0= Date: Sat, 30 Nov 2019 17:52:14 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 054/158] sparc: mm: add p?d_leaf() definitions Message-ID: <20191201015214.jkpFXFabz%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: sparc: mm: add p?d_leaf() definitions walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information is provided by the p?d_leaf() functions/macros. For sparc 64 bit, pmd_large() and pud_large() are already provided, so add macros to provide the p?d_leaf names required by the generic code. Link: http://lkml.kernel.org/r/20191028135910.33253-10-steven.price@arm.com Signed-off-by: Steven Price Acked-by: David S. Miller Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Cc: Zong Li Signed-off-by: Andrew Morton --- arch/sparc/include/asm/pgtable_64.h | 2 ++ 1 file changed, 2 insertions(+) --- a/arch/sparc/include/asm/pgtable_64.h~sparc-mm-add-pd_leaf-definitions +++ a/arch/sparc/include/asm/pgtable_64.h @@ -683,6 +683,7 @@ static inline unsigned long pte_special( return pte_val(pte) & _PAGE_SPECIAL; } +#define pmd_leaf pmd_large static inline unsigned long pmd_large(pmd_t pmd) { pte_t pte = __pte(pmd_val(pmd)); @@ -867,6 +868,7 @@ static inline unsigned long pud_page_vad /* only used by the stubbed out hugetlb gup code, should never be called */ #define pgd_page(pgd) NULL +#define pud_leaf pud_large static inline unsigned long pud_large(pud_t pud) { pte_t pte = __pte(pud_val(pud)); From patchwork Sun Dec 1 01:52:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268287 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8C5B3921 for ; Sun, 1 Dec 2019 01:52:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 59EB2206E1 for ; Sun, 1 Dec 2019 01:52:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="h6pLPI2D" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 59EB2206E1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2C4756B02C4; Sat, 30 Nov 2019 20:52:23 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 274D76B02C6; Sat, 30 Nov 2019 20:52:23 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 163FB6B02C7; Sat, 30 Nov 2019 20:52:23 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0168.hostedemail.com [216.40.44.168]) by kanga.kvack.org (Postfix) with ESMTP id F3A406B02C4 for ; Sat, 30 Nov 2019 20:52:22 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id A8EF3180AD817 for ; Sun, 1 Dec 2019 01:52:22 +0000 (UTC) X-FDA: 76214897724.01.bell67_14b4634802a5e X-Spam-Summary: 2,0,0,a94539e8bdbe1d13,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:41:355:379:800:960: 967:973: X-HE-Tag: bell67_14b4634802a5e X-Filterd-Recvd-Size: 5698 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:52:22 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 35521208E4; Sun, 1 Dec 2019 01:52:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165141; bh=IwJPH5iP/FJfkDyM3FXOWvRJIWlMvXN14TvJoSk13t4=; h=Date:From:To:Subject:From; b=h6pLPI2DbJIoqNcjEc6oKZrnDzCgE2SL/7a+w3K1d6c99yWKybCYwBeSJbeKUYeQ6 jtgi7sEAAZeui9RTTy0G8ykSMkS/p3EaD7j3s1sJJOmxOZxfCImSeWEovZgzH6fKsK QjwBLxa23OhMLBoqxE8uiubAAKA5Z5brzCeCkec8= Date: Sat, 30 Nov 2019 17:52:19 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 055/158] x86: mm: add p?d_leaf() definitions Message-ID: <20191201015219.d1U0SayGe%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: x86: mm: add p?d_leaf() definitions walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information is provided by the p?d_leaf() functions/macros. For x86 we already have p?d_large() functions, so simply add macros to provide the generic p?d_leaf() names for the generic code. Link: http://lkml.kernel.org/r/20191028135910.33253-11-steven.price@arm.com Signed-off-by: Steven Price Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Thomas Gleixner Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Cc: Zong Li Signed-off-by: Andrew Morton --- arch/x86/include/asm/pgtable.h | 5 +++++ 1 file changed, 5 insertions(+) --- a/arch/x86/include/asm/pgtable.h~x86-mm-add-pd_leaf-definitions +++ a/arch/x86/include/asm/pgtable.h @@ -239,6 +239,7 @@ static inline unsigned long pgd_pfn(pgd_ return (pgd_val(pgd) & PTE_PFN_MASK) >> PAGE_SHIFT; } +#define p4d_leaf p4d_large static inline int p4d_large(p4d_t p4d) { /* No 512 GiB pages yet */ @@ -247,6 +248,7 @@ static inline int p4d_large(p4d_t p4d) #define pte_page(pte) pfn_to_page(pte_pfn(pte)) +#define pmd_leaf pmd_large static inline int pmd_large(pmd_t pte) { return pmd_flags(pte) & _PAGE_PSE; @@ -874,6 +876,7 @@ static inline pmd_t *pmd_offset(pud_t *p return (pmd_t *)pud_page_vaddr(*pud) + pmd_index(address); } +#define pud_leaf pud_large static inline int pud_large(pud_t pud) { return (pud_val(pud) & (_PAGE_PSE | _PAGE_PRESENT)) == @@ -885,6 +888,7 @@ static inline int pud_bad(pud_t pud) return (pud_flags(pud) & ~(_KERNPG_TABLE | _PAGE_USER)) != 0; } #else +#define pud_leaf pud_large static inline int pud_large(pud_t pud) { return 0; @@ -1233,6 +1237,7 @@ static inline bool pgdp_maps_userspace(v return (((ptr & ~PAGE_MASK) / sizeof(pgd_t)) < PGD_KERNEL_START); } +#define pgd_leaf pgd_large static inline int pgd_large(pgd_t pgd) { return 0; } #ifdef CONFIG_PAGE_TABLE_ISOLATION From patchwork Sun Dec 1 01:52:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268289 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D1419921 for ; Sun, 1 Dec 2019 01:52:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 91D2E215E5 for ; Sun, 1 Dec 2019 01:52:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="QYdv3OzO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 91D2E215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 733286B02C6; Sat, 30 Nov 2019 20:52:28 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6BAF26B02C8; Sat, 30 Nov 2019 20:52:28 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5D23B6B02C9; Sat, 30 Nov 2019 20:52:28 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0097.hostedemail.com [216.40.44.97]) by kanga.kvack.org (Postfix) with ESMTP id 473AE6B02C6 for ; Sat, 30 Nov 2019 20:52:28 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id EFC3F2C7C for ; Sun, 1 Dec 2019 01:52:27 +0000 (UTC) X-FDA: 76214897976.04.force57_157958eb6435a X-Spam-Summary: 2,0,0,1d5b5f9b395e4e37,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:2:41:69:355:379:800 :960:967 X-HE-Tag: force57_157958eb6435a X-Filterd-Recvd-Size: 8108 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:52:27 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D0A13208C3; Sun, 1 Dec 2019 01:52:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165146; bh=iQD54ihx+qYnzm6NaJOcT4d7GLAOB8UFHXUh5C1QGlw=; h=Date:From:To:Subject:From; b=QYdv3OzOia8626nHiFdgQNBUHjoIrD/96HATDVChnya3a52lTOt1W1AWEEVKnq5PY 5dB+q3nOPQTS6AWmKodIgOKG7FxWy5L1r/Bg+uU/y46r3HAKHMqZ2DcumApxHxa46O EsdqEhvUR3r/3u6JWzi9d5T4l1+lXJeW/ZqIt8uI= Date: Sat, 30 Nov 2019 17:52:24 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 056/158] mm: pagewalk: add p4d_entry() and pgd_entry() Message-ID: <20191201015224.ZZwhTgzAI%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: mm: pagewalk: add p4d_entry() and pgd_entry() pgd_entry() and pud_entry() were removed by commit 0b1fbfe50006c410 ("mm/pagewalk: remove pgd_entry() and pud_entry()") because there were no users. We're about to add users so reintroduce them, along with p4d_entry() as we now have 5 levels of tables. Note that commit a00cc7d9dd93d66a ("mm, x86: add support for PUD-sized transparent hugepages") already re-added pud_entry() but with different semantics to the other callbacks. Since there have never been upstream users of this, revert the semantics back to match the other callbacks. This means pud_entry() is called for all entries, not just transparent huge pages. Link: http://lkml.kernel.org/r/20191028135910.33253-12-steven.price@arm.com Signed-off-by: Steven Price Tested-by: Zong Li Cc: Matthew Wilcox Cc: Dave Jiang Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Signed-off-by: Andrew Morton --- include/linux/pagewalk.h | 19 +++++++++++++------ mm/pagewalk.c | 27 ++++++++++++++++----------- 2 files changed, 29 insertions(+), 17 deletions(-) --- a/include/linux/pagewalk.h~mm-pagewalk-add-p4d_entry-and-pgd_entry +++ a/include/linux/pagewalk.h @@ -8,15 +8,15 @@ struct mm_walk; /** * mm_walk_ops - callbacks for walk_page_range - * @pud_entry: if set, called for each non-empty PUD (2nd-level) entry - * this handler should only handle pud_trans_huge() puds. - * the pmd_entry or pte_entry callbacks will be used for - * regular PUDs. - * @pmd_entry: if set, called for each non-empty PMD (3rd-level) entry + * @pgd_entry: if set, called for each non-empty PGD (top-level) entry + * @p4d_entry: if set, called for each non-empty P4D entry + * @pud_entry: if set, called for each non-empty PUD entry + * @pmd_entry: if set, called for each non-empty PMD entry * this handler is required to be able to handle * pmd_trans_huge() pmds. They may simply choose to * split_huge_page() instead of handling it explicitly. - * @pte_entry: if set, called for each non-empty PTE (4th-level) entry + * @pte_entry: if set, called for each non-empty PTE (lowest-level) + * entry * @pte_hole: if set, called for each hole at all levels * @hugetlb_entry: if set, called for each hugetlb entry * @test_walk: caller specific callback function to determine whether @@ -27,8 +27,15 @@ struct mm_walk; * @pre_vma: if set, called before starting walk on a non-null vma. * @post_vma: if set, called after a walk on a non-null vma, provided * that @pre_vma and the vma walk succeeded. + * + * p?d_entry callbacks are called even if those levels are folded on a + * particular architecture/configuration. */ struct mm_walk_ops { + int (*pgd_entry)(pgd_t *pgd, unsigned long addr, + unsigned long next, struct mm_walk *walk); + int (*p4d_entry)(p4d_t *p4d, unsigned long addr, + unsigned long next, struct mm_walk *walk); int (*pud_entry)(pud_t *pud, unsigned long addr, unsigned long next, struct mm_walk *walk); int (*pmd_entry)(pmd_t *pmd, unsigned long addr, --- a/mm/pagewalk.c~mm-pagewalk-add-p4d_entry-and-pgd_entry +++ a/mm/pagewalk.c @@ -94,15 +94,9 @@ static int walk_pud_range(p4d_t *p4d, un } if (ops->pud_entry) { - spinlock_t *ptl = pud_trans_huge_lock(pud, walk->vma); - - if (ptl) { - err = ops->pud_entry(pud, addr, next, walk); - spin_unlock(ptl); - if (err) - break; - continue; - } + err = ops->pud_entry(pud, addr, next, walk); + if (err) + break; } split_huge_pud(walk->vma, pud, addr); @@ -136,7 +130,12 @@ static int walk_p4d_range(pgd_t *pgd, un break; continue; } - if (ops->pmd_entry || ops->pte_entry) + if (ops->p4d_entry) { + err = ops->p4d_entry(p4d, addr, next, walk); + if (err) + break; + } + if (ops->pud_entry || ops->pmd_entry || ops->pte_entry) err = walk_pud_range(p4d, addr, next, walk); if (err) break; @@ -163,7 +162,13 @@ static int walk_pgd_range(unsigned long break; continue; } - if (ops->pmd_entry || ops->pte_entry) + if (ops->pgd_entry) { + err = ops->pgd_entry(pgd, addr, next, walk); + if (err) + break; + } + if (ops->p4d_entry || ops->pud_entry || ops->pmd_entry || + ops->pte_entry) err = walk_p4d_range(pgd, addr, next, walk); if (err) break; From patchwork Sun Dec 1 01:52:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268291 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E655E112B for ; Sun, 1 Dec 2019 01:52:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A8A592084D for ; Sun, 1 Dec 2019 01:52:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="mRYMy0Aj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A8A592084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7A81C6B02CA; Sat, 30 Nov 2019 20:52:33 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 77EBD6B02CB; Sat, 30 Nov 2019 20:52:33 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6950E6B02CC; Sat, 30 Nov 2019 20:52:33 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0133.hostedemail.com [216.40.44.133]) by kanga.kvack.org (Postfix) with ESMTP id 52AEF6B02CA for ; Sat, 30 Nov 2019 20:52:33 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id F0F8D2C7C for ; Sun, 1 Dec 2019 01:52:32 +0000 (UTC) X-FDA: 76214898186.15.flame06_16334689ba763 X-Spam-Summary: 2,0,0,2534a848b67b08cc,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:cai@lca.pw:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:sfr@canb.auug.org.au:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.c om,RULES X-HE-Tag: flame06_16334689ba763 X-Filterd-Recvd-Size: 8051 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:52:32 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 42AAB215E5; Sun, 1 Dec 2019 01:52:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165151; bh=uzDc+uufhtMBaymUpqZStdlHoygzBFXgxPNBC31CVZ8=; h=Date:From:To:Subject:From; b=mRYMy0Aj3ERI60UnXrdsMvO1Ohc6Q7Mi5iQ4Suc+Fp5z/E3xK9BGiOTyLt+JmZwrY qjqgEKdBg8ZPtxvVbxMEUF0yJ3Gta3RVWpaeVL5wYDigUUvzG4pOvlD5ltucG2sGBs pGj+bBdiEk6N8SgUSRpBxq81pgwDTU2pZu3AzrAw= Date: Sat, 30 Nov 2019 17:52:29 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, cai@lca.pw, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, sfr@canb.auug.org.au, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 057/158] mm: pagewalk: allow walking without vma Message-ID: <20191201015229.Athbgankp%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: mm: pagewalk: allow walking without vma Since 48684a65b4e3: "mm: pagewalk: fix misbehavior of walk_page_range for vma(VM_PFNMAP)", page_table_walk() will report any kernel area as a hole, because it lacks a vma. This means each arch has re-implemented page table walking when needed, for example in the per-arch ptdump walker. Remove the requirement to have a vma in the generic code and add a new function walk_page_range_novma() which ignores the VMAs and simply walks the page tables. [steven.price@arm.com: v15] Link: http://lkml.kernel.org/r/20191101140942.51554-13-steven.price@arm.com [steven.price@arm.com: fix boot crash] Link: http://lkml.kernel.org/r/20191028135910.33253-13-steven.price@arm.com Cc: Zong Li Cc: Naoya Horiguchi Cc: Shiraz Hashim Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Cc: Qian Cai Cc: Stephen Rothwell Signed-off-by: Andrew Morton --- include/linux/pagewalk.h | 5 ++++ mm/pagewalk.c | 44 ++++++++++++++++++++++++++++++------- 2 files changed, 41 insertions(+), 8 deletions(-) --- a/include/linux/pagewalk.h~mm-pagewalk-allow-walking-without-vma +++ a/include/linux/pagewalk.h @@ -59,6 +59,7 @@ struct mm_walk_ops { * @ops: operation to call during the walk * @mm: mm_struct representing the target process of page table walk * @vma: vma currently walked (NULL if walking outside vmas) + * @no_vma: walk ignoring vmas (vma will always be NULL) * @private: private data for callbacks' usage * * (see the comment on walk_page_range() for more details) @@ -67,12 +68,16 @@ struct mm_walk { const struct mm_walk_ops *ops; struct mm_struct *mm; struct vm_area_struct *vma; + bool no_vma; void *private; }; int walk_page_range(struct mm_struct *mm, unsigned long start, unsigned long end, const struct mm_walk_ops *ops, void *private); +int walk_page_range_novma(struct mm_struct *mm, unsigned long start, + unsigned long end, const struct mm_walk_ops *ops, + void *private); int walk_page_vma(struct vm_area_struct *vma, const struct mm_walk_ops *ops, void *private); int walk_page_mapping(struct address_space *mapping, pgoff_t first_index, --- a/mm/pagewalk.c~mm-pagewalk-allow-walking-without-vma +++ a/mm/pagewalk.c @@ -39,7 +39,7 @@ static int walk_pmd_range(pud_t *pud, un do { again: next = pmd_addr_end(addr, end); - if (pmd_none(*pmd) || !walk->vma) { + if (pmd_none(*pmd) || (!walk->vma && !walk->no_vma)) { if (ops->pte_hole) err = ops->pte_hole(addr, next, walk); if (err) @@ -62,9 +62,14 @@ again: if (!ops->pte_entry) continue; - split_huge_pmd(walk->vma, pmd, addr); - if (pmd_trans_unstable(pmd)) - goto again; + if (walk->vma) { + split_huge_pmd(walk->vma, pmd, addr); + if (pmd_trans_unstable(pmd)) + goto again; + } else if (pmd_leaf(*pmd) || !pmd_present(*pmd)) { + continue; + } + err = walk_pte_range(pmd, addr, next, walk); if (err) break; @@ -85,7 +90,7 @@ static int walk_pud_range(p4d_t *p4d, un do { again: next = pud_addr_end(addr, end); - if (pud_none(*pud) || !walk->vma) { + if (pud_none(*pud) || (!walk->vma && !walk->no_vma)) { if (ops->pte_hole) err = ops->pte_hole(addr, next, walk); if (err) @@ -99,9 +104,13 @@ static int walk_pud_range(p4d_t *p4d, un break; } - split_huge_pud(walk->vma, pud, addr); - if (pud_none(*pud)) - goto again; + if (walk->vma) { + split_huge_pud(walk->vma, pud, addr); + if (pud_none(*pud)) + goto again; + } else if (pud_leaf(*pud) || !pud_present(*pud)) { + continue; + } if (ops->pmd_entry || ops->pte_entry) err = walk_pmd_range(pud, addr, next, walk); @@ -374,6 +383,25 @@ int walk_page_range(struct mm_struct *mm return err; } +int walk_page_range_novma(struct mm_struct *mm, unsigned long start, + unsigned long end, const struct mm_walk_ops *ops, + void *private) +{ + struct mm_walk walk = { + .ops = ops, + .mm = mm, + .private = private, + .no_vma = true + }; + + if (start >= end || !walk.mm) + return -EINVAL; + + lockdep_assert_held(&walk.mm->mmap_sem); + + return __walk_page_range(start, end, &walk); +} + int walk_page_vma(struct vm_area_struct *vma, const struct mm_walk_ops *ops, void *private) { From patchwork Sun Dec 1 01:52:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268293 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 939E2112B for ; Sun, 1 Dec 2019 01:52:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5697D206E1 for ; Sun, 1 Dec 2019 01:52:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="SJt+dt8d" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5697D206E1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 166896B02CB; Sat, 30 Nov 2019 20:52:38 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 13CB46B02CD; Sat, 30 Nov 2019 20:52:38 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 053946B02CE; Sat, 30 Nov 2019 20:52:38 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0129.hostedemail.com [216.40.44.129]) by kanga.kvack.org (Postfix) with ESMTP id E493F6B02CB for ; Sat, 30 Nov 2019 20:52:37 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 9A5272A87 for ; Sun, 1 Dec 2019 01:52:37 +0000 (UTC) X-FDA: 76214898354.23.bat35_16e1893a65960 X-Spam-Summary: 2,0,0,d6c5ca502456421b,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:41:355:379:800:960: 967:973: X-HE-Tag: bat35_16e1893a65960 X-Filterd-Recvd-Size: 6969 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:52:37 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0FED82084D; Sun, 1 Dec 2019 01:52:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165156; bh=iAklNOP04yUqFkjJSb1BKCgNRlcd/ky1dTIicmIGqMg=; h=Date:From:To:Subject:From; b=SJt+dt8d5jvK3triE090YlSwfy3bHrqJV372jSy1klI/b8hsmpEhW/olzB8TQaTFN 5ucJDLQIe7uLJ/6ZW89I84OEcyAt/EbnurqP6+u3B1R0xNEi9lVVs5IX95XENsPZww Sc2QiJIFags5Zwd/9Csy2y5zXMCPyaxcyPyUz87s= Date: Sat, 30 Nov 2019 17:52:34 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 058/158] mm: pagewalk: add test_p?d callbacks Message-ID: <20191201015234.2h9y8PXQL%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: mm: pagewalk: add test_p?d callbacks It is useful to be able to skip parts of the page table tree even when walking without VMAs. Add test_p?d callbacks similar to test_walk but which are called just before a table at that level is walked. If the callback returns non-zero then the entire table is skipped. Link: http://lkml.kernel.org/r/20191028135910.33253-14-steven.price@arm.com Signed-off-by: Steven Price Tested-by: Zong Li Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Signed-off-by: Andrew Morton --- include/linux/pagewalk.h | 11 +++++++++++ mm/pagewalk.c | 24 ++++++++++++++++++++++++ 2 files changed, 35 insertions(+) --- a/include/linux/pagewalk.h~mm-pagewalk-add-test_pd-callbacks +++ a/include/linux/pagewalk.h @@ -24,6 +24,11 @@ struct mm_walk; * "do page table walk over the current vma", returning * a negative value means "abort current page table walk * right now" and returning 1 means "skip the current vma" + * @test_pmd: similar to test_walk(), but called for every pmd. + * @test_pud: similar to test_walk(), but called for every pud. + * @test_p4d: similar to test_walk(), but called for every p4d. + * Returning 0 means walk this part of the page tables, + * returning 1 means to skip this range. * @pre_vma: if set, called before starting walk on a non-null vma. * @post_vma: if set, called after a walk on a non-null vma, provided * that @pre_vma and the vma walk succeeded. @@ -47,6 +52,12 @@ struct mm_walk_ops { int (*hugetlb_entry)(pte_t *pte, unsigned long hmask, unsigned long addr, unsigned long next, struct mm_walk *walk); + int (*test_pmd)(unsigned long addr, unsigned long next, + pmd_t *pmd_start, struct mm_walk *walk); + int (*test_pud)(unsigned long addr, unsigned long next, + pud_t *pud_start, struct mm_walk *walk); + int (*test_p4d)(unsigned long addr, unsigned long next, + p4d_t *p4d_start, struct mm_walk *walk); int (*test_walk)(unsigned long addr, unsigned long next, struct mm_walk *walk); int (*pre_vma)(unsigned long start, unsigned long end, --- a/mm/pagewalk.c~mm-pagewalk-add-test_pd-callbacks +++ a/mm/pagewalk.c @@ -35,6 +35,14 @@ static int walk_pmd_range(pud_t *pud, un const struct mm_walk_ops *ops = walk->ops; int err = 0; + if (ops->test_pmd) { + err = ops->test_pmd(addr, end, pmd_offset(pud, 0UL), walk); + if (err < 0) + return err; + if (err > 0) + return 0; + } + pmd = pmd_offset(pud, addr); do { again: @@ -86,6 +94,14 @@ static int walk_pud_range(p4d_t *p4d, un const struct mm_walk_ops *ops = walk->ops; int err = 0; + if (ops->test_pud) { + err = ops->test_pud(addr, end, pud_offset(p4d, 0UL), walk); + if (err < 0) + return err; + if (err > 0) + return 0; + } + pud = pud_offset(p4d, addr); do { again: @@ -129,6 +145,14 @@ static int walk_p4d_range(pgd_t *pgd, un const struct mm_walk_ops *ops = walk->ops; int err = 0; + if (ops->test_p4d) { + err = ops->test_p4d(addr, end, p4d_offset(pgd, 0UL), walk); + if (err < 0) + return err; + if (err > 0) + return 0; + } + p4d = p4d_offset(pgd, addr); do { next = p4d_addr_end(addr, end); From patchwork Sun Dec 1 01:52:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268295 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3AD37921 for ; Sun, 1 Dec 2019 01:52:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EEBB4215E5 for ; Sun, 1 Dec 2019 01:52:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZlDCMDxr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EEBB4215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A17686B02CD; Sat, 30 Nov 2019 20:52:42 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9C9206B02CF; Sat, 30 Nov 2019 20:52:42 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DE6A6B02D0; Sat, 30 Nov 2019 20:52:42 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0180.hostedemail.com [216.40.44.180]) by kanga.kvack.org (Postfix) with ESMTP id 7889F6B02CD for ; Sat, 30 Nov 2019 20:52:42 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 426E4824999B for ; Sun, 1 Dec 2019 01:52:42 +0000 (UTC) X-FDA: 76214898564.23.tramp79_178d48d1ae612 X-Spam-Summary: 2,0,0,418ee520c23e8ad7,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:1:41:355:379:800:96 0:967:97 X-HE-Tag: tramp79_178d48d1ae612 X-Filterd-Recvd-Size: 12554 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:52:41 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D74F72084E; Sun, 1 Dec 2019 01:52:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165161; bh=iaO7T6ZUE1RAcfQYwTW6B+MKtjHz6J8jLGDLsi3euko=; h=Date:From:To:Subject:From; b=ZlDCMDxrLJfhClP/Sn3ME5F6f2FYwh44T9hBEuqd+wMF0+oULxswHPKt53XwKME9Y HT0VbQ0thE/pl9F6wMJEJUpfQlIa6a0YqgnCcIUPHwe26ICaaaB4csCH0eY+dg00jN sU54xXBwK1+02tgCgDKgNyqNQ5naqtODK5fT9dX8= Date: Sat, 30 Nov 2019 17:52:39 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 059/158] mm: pagewalk: add 'depth' parameter to pte_hole Message-ID: <20191201015239.AuqOwIAzc%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: mm: pagewalk: add 'depth' parameter to pte_hole The pte_hole() callback is called at multiple levels of the page tables. Code dumping the kernel page tables needs to know what at what depth the missing entry is. Add this is an extra parameter to pte_hole(). When the depth isn't know (e.g. processing a vma) then -1 is passed. The depth that is reported is the actual level where the entry is missing (ignoring any folding that is in place), i.e. any levels where PTRS_PER_P?D is set to 1 are ignored. Note that depth starts at 0 for a PGD so that PUD/PMD/PTE retain their natural numbers as levels 2/3/4. Link: http://lkml.kernel.org/r/20191028135910.33253-15-steven.price@arm.com Signed-off-by: Steven Price Tested-by: Zong Li Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Signed-off-by: Andrew Morton --- fs/proc/task_mmu.c | 4 ++-- include/linux/pagewalk.h | 7 +++++-- mm/hmm.c | 8 ++++---- mm/migrate.c | 5 +++-- mm/mincore.c | 1 + mm/pagewalk.c | 31 +++++++++++++++++++++++++------ 6 files changed, 40 insertions(+), 16 deletions(-) --- a/fs/proc/task_mmu.c~mm-pagewalk-add-depth-parameter-to-pte_hole +++ a/fs/proc/task_mmu.c @@ -505,7 +505,7 @@ static void smaps_account(struct mem_siz #ifdef CONFIG_SHMEM static int smaps_pte_hole(unsigned long addr, unsigned long end, - struct mm_walk *walk) + __always_unused int depth, struct mm_walk *walk) { struct mem_size_stats *mss = walk->private; @@ -1282,7 +1282,7 @@ static int add_to_pagemap(unsigned long } static int pagemap_pte_hole(unsigned long start, unsigned long end, - struct mm_walk *walk) + __always_unused int depth, struct mm_walk *walk) { struct pagemapread *pm = walk->private; unsigned long addr = start; --- a/include/linux/pagewalk.h~mm-pagewalk-add-depth-parameter-to-pte_hole +++ a/include/linux/pagewalk.h @@ -17,7 +17,10 @@ struct mm_walk; * split_huge_page() instead of handling it explicitly. * @pte_entry: if set, called for each non-empty PTE (lowest-level) * entry - * @pte_hole: if set, called for each hole at all levels + * @pte_hole: if set, called for each hole at all levels, + * depth is -1 if not known, 0:PGD, 1:P4D, 2:PUD, 3:PMD + * 4:PTE. Any folded depths (where PTRS_PER_P?D is equal + * to 1) are skipped. * @hugetlb_entry: if set, called for each hugetlb entry * @test_walk: caller specific callback function to determine whether * we walk over the current vma or not. Returning 0 means @@ -48,7 +51,7 @@ struct mm_walk_ops { int (*pte_entry)(pte_t *pte, unsigned long addr, unsigned long next, struct mm_walk *walk); int (*pte_hole)(unsigned long addr, unsigned long next, - struct mm_walk *walk); + int depth, struct mm_walk *walk); int (*hugetlb_entry)(pte_t *pte, unsigned long hmask, unsigned long addr, unsigned long next, struct mm_walk *walk); --- a/mm/hmm.c~mm-pagewalk-add-depth-parameter-to-pte_hole +++ a/mm/hmm.c @@ -186,7 +186,7 @@ static void hmm_range_need_fault(const s } static int hmm_vma_walk_hole(unsigned long addr, unsigned long end, - struct mm_walk *walk) + __always_unused int depth, struct mm_walk *walk) { struct hmm_vma_walk *hmm_vma_walk = walk->private; struct hmm_range *range = hmm_vma_walk->range; @@ -380,7 +380,7 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp, again: pmd = READ_ONCE(*pmdp); if (pmd_none(pmd)) - return hmm_vma_walk_hole(start, end, walk); + return hmm_vma_walk_hole(start, end, -1, walk); if (thp_migration_supported() && is_pmd_migration_entry(pmd)) { bool fault, write_fault; @@ -482,7 +482,7 @@ static int hmm_vma_walk_pud(pud_t *pudp, again: pud = READ_ONCE(*pudp); if (pud_none(pud)) - return hmm_vma_walk_hole(start, end, walk); + return hmm_vma_walk_hole(start, end, -1, walk); if (pud_huge(pud) && pud_devmap(pud)) { unsigned long i, npages, pfn; @@ -490,7 +490,7 @@ again: bool fault, write_fault; if (!pud_present(pud)) - return hmm_vma_walk_hole(start, end, walk); + return hmm_vma_walk_hole(start, end, -1, walk); i = (addr - range->start) >> PAGE_SHIFT; npages = (end - addr) >> PAGE_SHIFT; --- a/mm/migrate.c~mm-pagewalk-add-depth-parameter-to-pte_hole +++ a/mm/migrate.c @@ -2123,6 +2123,7 @@ out_unlock: #ifdef CONFIG_DEVICE_PRIVATE static int migrate_vma_collect_hole(unsigned long start, unsigned long end, + __always_unused int depth, struct mm_walk *walk) { struct migrate_vma *migrate = walk->private; @@ -2167,7 +2168,7 @@ static int migrate_vma_collect_pmd(pmd_t again: if (pmd_none(*pmdp)) - return migrate_vma_collect_hole(start, end, walk); + return migrate_vma_collect_hole(start, end, -1, walk); if (pmd_trans_huge(*pmdp)) { struct page *page; @@ -2200,7 +2201,7 @@ again: return migrate_vma_collect_skip(start, end, walk); if (pmd_none(*pmdp)) - return migrate_vma_collect_hole(start, end, + return migrate_vma_collect_hole(start, end, -1, walk); } } --- a/mm/mincore.c~mm-pagewalk-add-depth-parameter-to-pte_hole +++ a/mm/mincore.c @@ -112,6 +112,7 @@ static int __mincore_unmapped_range(unsi } static int mincore_unmapped_range(unsigned long addr, unsigned long end, + __always_unused int depth, struct mm_walk *walk) { walk->private += __mincore_unmapped_range(addr, end, --- a/mm/pagewalk.c~mm-pagewalk-add-depth-parameter-to-pte_hole +++ a/mm/pagewalk.c @@ -4,6 +4,22 @@ #include #include +/* + * We want to know the real level where a entry is located ignoring any + * folding of levels which may be happening. For example if p4d is folded then + * a missing entry found at level 1 (p4d) is actually at level 0 (pgd). + */ +static int real_depth(int depth) +{ + if (depth == 3 && PTRS_PER_PMD == 1) + depth = 2; + if (depth == 2 && PTRS_PER_PUD == 1) + depth = 1; + if (depth == 1 && PTRS_PER_P4D == 1) + depth = 0; + return depth; +} + static int walk_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, struct mm_walk *walk) { @@ -34,6 +50,7 @@ static int walk_pmd_range(pud_t *pud, un unsigned long next; const struct mm_walk_ops *ops = walk->ops; int err = 0; + int depth = real_depth(3); if (ops->test_pmd) { err = ops->test_pmd(addr, end, pmd_offset(pud, 0UL), walk); @@ -49,7 +66,7 @@ again: next = pmd_addr_end(addr, end); if (pmd_none(*pmd) || (!walk->vma && !walk->no_vma)) { if (ops->pte_hole) - err = ops->pte_hole(addr, next, walk); + err = ops->pte_hole(addr, next, depth, walk); if (err) break; continue; @@ -93,6 +110,7 @@ static int walk_pud_range(p4d_t *p4d, un unsigned long next; const struct mm_walk_ops *ops = walk->ops; int err = 0; + int depth = real_depth(2); if (ops->test_pud) { err = ops->test_pud(addr, end, pud_offset(p4d, 0UL), walk); @@ -108,7 +126,7 @@ static int walk_pud_range(p4d_t *p4d, un next = pud_addr_end(addr, end); if (pud_none(*pud) || (!walk->vma && !walk->no_vma)) { if (ops->pte_hole) - err = ops->pte_hole(addr, next, walk); + err = ops->pte_hole(addr, next, depth, walk); if (err) break; continue; @@ -144,6 +162,7 @@ static int walk_p4d_range(pgd_t *pgd, un unsigned long next; const struct mm_walk_ops *ops = walk->ops; int err = 0; + int depth = real_depth(1); if (ops->test_p4d) { err = ops->test_p4d(addr, end, p4d_offset(pgd, 0UL), walk); @@ -158,7 +177,7 @@ static int walk_p4d_range(pgd_t *pgd, un next = p4d_addr_end(addr, end); if (p4d_none_or_clear_bad(p4d)) { if (ops->pte_hole) - err = ops->pte_hole(addr, next, walk); + err = ops->pte_hole(addr, next, depth, walk); if (err) break; continue; @@ -190,7 +209,7 @@ static int walk_pgd_range(unsigned long next = pgd_addr_end(addr, end); if (pgd_none_or_clear_bad(pgd)) { if (ops->pte_hole) - err = ops->pte_hole(addr, next, walk); + err = ops->pte_hole(addr, next, 0, walk); if (err) break; continue; @@ -237,7 +256,7 @@ static int walk_hugetlb_range(unsigned l if (pte) err = ops->hugetlb_entry(pte, hmask, addr, next, walk); else if (ops->pte_hole) - err = ops->pte_hole(addr, next, walk); + err = ops->pte_hole(addr, next, -1, walk); if (err) break; @@ -281,7 +300,7 @@ static int walk_page_test(unsigned long if (vma->vm_flags & VM_PFNMAP) { int err = 1; if (ops->pte_hole) - err = ops->pte_hole(start, end, walk); + err = ops->pte_hole(start, end, -1, walk); return err ? err : 1; } return 0; From patchwork Sun Dec 1 01:52:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268297 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3F34D112B for ; Sun, 1 Dec 2019 01:52:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E49FF2084D for ; Sun, 1 Dec 2019 01:52:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="TBZM73bQ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E49FF2084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B78456B02CF; Sat, 30 Nov 2019 20:52:47 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B29066B02D1; Sat, 30 Nov 2019 20:52:47 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3DCE6B02D2; Sat, 30 Nov 2019 20:52:47 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0042.hostedemail.com [216.40.44.42]) by kanga.kvack.org (Postfix) with ESMTP id 8E73E6B02CF for ; Sat, 30 Nov 2019 20:52:47 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 441E92A87 for ; Sun, 1 Dec 2019 01:52:47 +0000 (UTC) X-FDA: 76214898774.28.mass86_18429e7fe4334 X-Spam-Summary: 2,0,0,30336c45ea45f48f,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:1:2:41:69:355:379:8 00:960:9 X-HE-Tag: mass86_18429e7fe4334 X-Filterd-Recvd-Size: 11385 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:52:46 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 868EC208C3; Sun, 1 Dec 2019 01:52:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165166; bh=KwY7RLbpjoDts6Z78a7wTnYn/IK8tbebTLrMnBZy+mQ=; h=Date:From:To:Subject:From; b=TBZM73bQq527nqmtkKc/4ETBRHvwf0um4Q3kyGghfM1Sl5jxw9Cd/qkJlAT5j1mip bAltEglolkLZ8cGeF7ad+cD8whv+fFLZZgc8dX9e0AEhZgCAFXuzCbt1qD/fw3sfQi 86otJ4ims6/TbkuWxWRgTMfjg6wtwe6tt11PZc/8= Date: Sat, 30 Nov 2019 17:52:44 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 060/158] x86: mm: point to struct seq_file from struct pg_state Message-ID: <20191201015244.ke25QFxcP%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: x86: mm: point to struct seq_file from struct pg_state mm/dump_pagetables.c passes both struct seq_file and struct pg_state down the chain of walk_*_level() functions to be passed to note_page(). Instead place the struct seq_file in struct pg_state and access it from struct pg_state (which is private to this file) in note_page(). Link: http://lkml.kernel.org/r/20191028135910.33253-16-steven.price@arm.com Signed-off-by: Steven Price Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Cc: Zong Li Signed-off-by: Andrew Morton --- arch/x86/mm/dump_pagetables.c | 69 ++++++++++++++++---------------- 1 file changed, 35 insertions(+), 34 deletions(-) --- a/arch/x86/mm/dump_pagetables.c~x86-mm-point-to-struct-seq_file-from-struct-pg_state +++ a/arch/x86/mm/dump_pagetables.c @@ -36,6 +36,7 @@ struct pg_state { bool to_dmesg; bool check_wx; unsigned long wx_pages; + struct seq_file *seq; }; struct addr_marker { @@ -265,11 +266,12 @@ static void note_wx(struct pg_state *st) * of PTE entries; the next one is different so we need to * print what we collected so far. */ -static void note_page(struct seq_file *m, struct pg_state *st, - pgprot_t new_prot, pgprotval_t new_eff, int level) +static void note_page(struct pg_state *st, pgprot_t new_prot, + pgprotval_t new_eff, int level) { pgprotval_t prot, cur, eff; static const char units[] = "BKMGTPE"; + struct seq_file *m = st->seq; /* * If we have a "break" in the series, we need to flush the state that @@ -354,8 +356,8 @@ static inline pgprotval_t effective_prot ((prot1 | prot2) & _PAGE_NX); } -static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, - pgprotval_t eff_in, unsigned long P) +static void walk_pte_level(struct pg_state *st, pmd_t addr, pgprotval_t eff_in, + unsigned long P) { int i; pte_t *pte; @@ -366,7 +368,7 @@ static void walk_pte_level(struct seq_fi pte = pte_offset_map(&addr, st->current_address); prot = pte_flags(*pte); eff = effective_prot(eff_in, prot); - note_page(m, st, __pgprot(prot), eff, 5); + note_page(st, __pgprot(prot), eff, 5); pte_unmap(pte); } } @@ -379,22 +381,20 @@ static void walk_pte_level(struct seq_fi * us dozens of seconds (minutes for 5-level config) while checking for * W+X mapping or reading kernel_page_tables debugfs file. */ -static inline bool kasan_page_table(struct seq_file *m, struct pg_state *st, - void *pt) +static inline bool kasan_page_table(struct pg_state *st, void *pt) { if (__pa(pt) == __pa(kasan_early_shadow_pmd) || (pgtable_l5_enabled() && __pa(pt) == __pa(kasan_early_shadow_p4d)) || __pa(pt) == __pa(kasan_early_shadow_pud)) { pgprotval_t prot = pte_flags(kasan_early_shadow_pte[0]); - note_page(m, st, __pgprot(prot), 0, 5); + note_page(st, __pgprot(prot), 0, 5); return true; } return false; } #else -static inline bool kasan_page_table(struct seq_file *m, struct pg_state *st, - void *pt) +static inline bool kasan_page_table(struct pg_state *st, void *pt) { return false; } @@ -402,7 +402,7 @@ static inline bool kasan_page_table(stru #if PTRS_PER_PMD > 1 -static void walk_pmd_level(struct seq_file *m, struct pg_state *st, pud_t addr, +static void walk_pmd_level(struct pg_state *st, pud_t addr, pgprotval_t eff_in, unsigned long P) { int i; @@ -416,27 +416,27 @@ static void walk_pmd_level(struct seq_fi prot = pmd_flags(*start); eff = effective_prot(eff_in, prot); if (pmd_large(*start) || !pmd_present(*start)) { - note_page(m, st, __pgprot(prot), eff, 4); - } else if (!kasan_page_table(m, st, pmd_start)) { - walk_pte_level(m, st, *start, eff, + note_page(st, __pgprot(prot), eff, 4); + } else if (!kasan_page_table(st, pmd_start)) { + walk_pte_level(st, *start, eff, P + i * PMD_LEVEL_MULT); } } else - note_page(m, st, __pgprot(0), 0, 4); + note_page(st, __pgprot(0), 0, 4); start++; } } #else -#define walk_pmd_level(m,s,a,e,p) walk_pte_level(m,s,__pmd(pud_val(a)),e,p) +#define walk_pmd_level(s,a,e,p) walk_pte_level(s,__pmd(pud_val(a)),e,p) #define pud_large(a) pmd_large(__pmd(pud_val(a))) #define pud_none(a) pmd_none(__pmd(pud_val(a))) #endif #if PTRS_PER_PUD > 1 -static void walk_pud_level(struct seq_file *m, struct pg_state *st, p4d_t addr, - pgprotval_t eff_in, unsigned long P) +static void walk_pud_level(struct pg_state *st, p4d_t addr, pgprotval_t eff_in, + unsigned long P) { int i; pud_t *start, *pud_start; @@ -450,33 +450,33 @@ static void walk_pud_level(struct seq_fi prot = pud_flags(*start); eff = effective_prot(eff_in, prot); if (pud_large(*start) || !pud_present(*start)) { - note_page(m, st, __pgprot(prot), eff, 3); - } else if (!kasan_page_table(m, st, pud_start)) { - walk_pmd_level(m, st, *start, eff, + note_page(st, __pgprot(prot), eff, 3); + } else if (!kasan_page_table(st, pud_start)) { + walk_pmd_level(st, *start, eff, P + i * PUD_LEVEL_MULT); } } else - note_page(m, st, __pgprot(0), 0, 3); + note_page(st, __pgprot(0), 0, 3); start++; } } #else -#define walk_pud_level(m,s,a,e,p) walk_pmd_level(m,s,__pud(p4d_val(a)),e,p) +#define walk_pud_level(s,a,e,p) walk_pmd_level(s,__pud(p4d_val(a)),e,p) #define p4d_large(a) pud_large(__pud(p4d_val(a))) #define p4d_none(a) pud_none(__pud(p4d_val(a))) #endif -static void walk_p4d_level(struct seq_file *m, struct pg_state *st, pgd_t addr, - pgprotval_t eff_in, unsigned long P) +static void walk_p4d_level(struct pg_state *st, pgd_t addr, pgprotval_t eff_in, + unsigned long P) { int i; p4d_t *start, *p4d_start; pgprotval_t prot, eff; if (PTRS_PER_P4D == 1) - return walk_pud_level(m, st, __p4d(pgd_val(addr)), eff_in, P); + return walk_pud_level(st, __p4d(pgd_val(addr)), eff_in, P); p4d_start = start = (p4d_t *)pgd_page_vaddr(addr); @@ -486,13 +486,13 @@ static void walk_p4d_level(struct seq_fi prot = p4d_flags(*start); eff = effective_prot(eff_in, prot); if (p4d_large(*start) || !p4d_present(*start)) { - note_page(m, st, __pgprot(prot), eff, 2); - } else if (!kasan_page_table(m, st, p4d_start)) { - walk_pud_level(m, st, *start, eff, + note_page(st, __pgprot(prot), eff, 2); + } else if (!kasan_page_table(st, p4d_start)) { + walk_pud_level(st, *start, eff, P + i * P4D_LEVEL_MULT); } } else - note_page(m, st, __pgprot(0), 0, 2); + note_page(st, __pgprot(0), 0, 2); start++; } @@ -529,6 +529,7 @@ static void ptdump_walk_pgd_level_core(s } st.check_wx = checkwx; + st.seq = m; if (checkwx) st.wx_pages = 0; @@ -542,13 +543,13 @@ static void ptdump_walk_pgd_level_core(s eff = prot; #endif if (pgd_large(*start) || !pgd_present(*start)) { - note_page(m, &st, __pgprot(prot), eff, 1); + note_page(&st, __pgprot(prot), eff, 1); } else { - walk_p4d_level(m, &st, *start, eff, + walk_p4d_level(&st, *start, eff, i * PGD_LEVEL_MULT); } } else - note_page(m, &st, __pgprot(0), 0, 1); + note_page(&st, __pgprot(0), 0, 1); cond_resched(); start++; @@ -556,7 +557,7 @@ static void ptdump_walk_pgd_level_core(s /* Flush out the last page */ st.current_address = normalize_addr(PTRS_PER_PGD*PGD_LEVEL_MULT); - note_page(m, &st, __pgprot(0), 0, 0); + note_page(&st, __pgprot(0), 0, 0); if (!checkwx) return; if (st.wx_pages) From patchwork Sun Dec 1 01:52:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268299 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2469B112B for ; Sun, 1 Dec 2019 01:52:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DBE5C215A5 for ; Sun, 1 Dec 2019 01:52:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="ALn1Yne2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DBE5C215A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CFEB96B02D1; Sat, 30 Nov 2019 20:52:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CD6086B02D3; Sat, 30 Nov 2019 20:52:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C12B96B02D4; Sat, 30 Nov 2019 20:52:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0099.hostedemail.com [216.40.44.99]) by kanga.kvack.org (Postfix) with ESMTP id AC0426B02D1 for ; Sat, 30 Nov 2019 20:52:53 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 604362A87 for ; Sun, 1 Dec 2019 01:52:53 +0000 (UTC) X-FDA: 76214899026.30.glove55_192a3018ac63c X-Spam-Summary: 2,0,0,9b9a8bea31301887,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:41:355:379:800:960: 967:973: X-HE-Tag: glove55_192a3018ac63c X-Filterd-Recvd-Size: 6530 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:52:52 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D3E772084D; Sun, 1 Dec 2019 01:52:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165172; bh=0c10hH3J/1w+xf1lK+eiBuC7ekIR2yaaecK/jLL+fHY=; h=Date:From:To:Subject:From; b=ALn1Yne2i2pyrdw8vKajxOQ/EMSvoqq5EW7D/tjhWAZaJDXhAzklG9GrCrVb4ijDl sLh6HnInwzOjdabpYYpU7G5rFzU5KT1cS1Ytw46JTi+0vjzhjNexLhjQUwSuzxaT9i MVHCFWa2nmp1KEgo3aaxzLhNlW79gcrO4EOZUOgU= Date: Sat, 30 Nov 2019 17:52:49 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 061/158] x86: mm+efi: convert ptdump_walk_pgd_level() to take a mm_struct Message-ID: <20191201015249.gZR05Wk5A%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: x86: mm+efi: convert ptdump_walk_pgd_level() to take a mm_struct To enable x86 to use the generic walk_page_range() function, the callers of ptdump_walk_pgd_level() need to pass an mm_struct rather than the raw pgd_t pointer. Luckily since commit 7e904a91bf60 ("efi: Use efi_mm in x86 as well as ARM") we now have an mm_struct for EFI on x86. Link: http://lkml.kernel.org/r/20191028135910.33253-17-steven.price@arm.com Signed-off-by: Steven Price Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Thomas Gleixner Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Cc: Zong Li Signed-off-by: Andrew Morton --- arch/x86/include/asm/pgtable.h | 2 +- arch/x86/mm/dump_pagetables.c | 4 ++-- arch/x86/platform/efi/efi_32.c | 2 +- arch/x86/platform/efi/efi_64.c | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) --- a/arch/x86/include/asm/pgtable.h~x86-mmefi-convert-ptdump_walk_pgd_level-to-take-a-mm_struct +++ a/arch/x86/include/asm/pgtable.h @@ -29,7 +29,7 @@ extern pgd_t early_top_pgt[PTRS_PER_PGD]; int __init __early_make_pgtable(unsigned long address, pmdval_t pmd); -void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd); +void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm); void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user); void ptdump_walk_pgd_level_checkwx(void); void ptdump_walk_user_pgd_level_checkwx(void); --- a/arch/x86/mm/dump_pagetables.c~x86-mmefi-convert-ptdump_walk_pgd_level-to-take-a-mm_struct +++ a/arch/x86/mm/dump_pagetables.c @@ -567,9 +567,9 @@ static void ptdump_walk_pgd_level_core(s pr_info("x86/mm: Checked W+X mappings: passed, no W+X pages found.\n"); } -void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd) +void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm) { - ptdump_walk_pgd_level_core(m, pgd, false, true); + ptdump_walk_pgd_level_core(m, mm->pgd, false, true); } void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user) --- a/arch/x86/platform/efi/efi_32.c~x86-mmefi-convert-ptdump_walk_pgd_level-to-take-a-mm_struct +++ a/arch/x86/platform/efi/efi_32.c @@ -49,7 +49,7 @@ void efi_sync_low_kernel_mappings(void) void __init efi_dump_pagetable(void) { #ifdef CONFIG_EFI_PGT_DUMP - ptdump_walk_pgd_level(NULL, swapper_pg_dir); + ptdump_walk_pgd_level(NULL, &init_mm); #endif } --- a/arch/x86/platform/efi/efi_64.c~x86-mmefi-convert-ptdump_walk_pgd_level-to-take-a-mm_struct +++ a/arch/x86/platform/efi/efi_64.c @@ -614,9 +614,9 @@ void __init efi_dump_pagetable(void) { #ifdef CONFIG_EFI_PGT_DUMP if (efi_enabled(EFI_OLD_MEMMAP)) - ptdump_walk_pgd_level(NULL, swapper_pg_dir); + ptdump_walk_pgd_level(NULL, &init_mm); else - ptdump_walk_pgd_level(NULL, efi_mm.pgd); + ptdump_walk_pgd_level(NULL, &efi_mm); #endif } From patchwork Sun Dec 1 01:52:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268301 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 12368921 for ; Sun, 1 Dec 2019 01:53:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BB822217AB for ; Sun, 1 Dec 2019 01:52:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="YYGR3bEj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BB822217AB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 86B136B02D3; Sat, 30 Nov 2019 20:52:58 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 81A896B02D5; Sat, 30 Nov 2019 20:52:58 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 731026B02D6; Sat, 30 Nov 2019 20:52:58 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0182.hostedemail.com [216.40.44.182]) by kanga.kvack.org (Postfix) with ESMTP id 5DF6D6B02D3 for ; Sat, 30 Nov 2019 20:52:58 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 1D0522C7C for ; Sun, 1 Dec 2019 01:52:58 +0000 (UTC) X-FDA: 76214899236.18.tub03_19dc0e165ac03 X-Spam-Summary: 2,0,0,7d7114397a28908c,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:2:41:355:379:800:96 0:967:97 X-HE-Tag: tub03_19dc0e165ac03 X-Filterd-Recvd-Size: 7618 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:52:57 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 8B52E215A5; Sun, 1 Dec 2019 01:52:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165177; bh=ZQxiZMilUZKziMoOv7yn4+5PskyA+540RPvT8IvuoyU=; h=Date:From:To:Subject:From; b=YYGR3bEjlruQ6FethUZPexO+kKXTa89Xi1IXqVzadNSi7pb698mdskGpX9XkZmkgA 6N+rUK395Ir+HovmppALWc65J06PzCK2jo9RjpDaOu9vSxZ0VimReASxVcJ+L/ABVk HNHsk3r2ueOEC8Q11DaMinrkT6fDwQkV/E0QGXgo= Date: Sat, 30 Nov 2019 17:52:55 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 062/158] x86: mm: convert ptdump_walk_pgd_level_debugfs() to take an mm_struct Message-ID: <20191201015255.njyURGtvr%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: x86: mm: convert ptdump_walk_pgd_level_debugfs() to take an mm_struct To enable x86 to use the generic walk_page_range() function, the callers of ptdump_walk_pgd_level_debugfs() need to pass in the mm_struct. This means that ptdump_walk_pgd_level_core() is now always passed a valid pgd, so drop the support for pgd==NULL. Link: http://lkml.kernel.org/r/20191028135910.33253-18-steven.price@arm.com Signed-off-by: Steven Price Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Thomas Gleixner Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Cc: Zong Li Signed-off-by: Andrew Morton --- arch/x86/include/asm/pgtable.h | 3 ++- arch/x86/mm/debug_pagetables.c | 8 ++++---- arch/x86/mm/dump_pagetables.c | 14 ++++++-------- 3 files changed, 12 insertions(+), 13 deletions(-) --- a/arch/x86/include/asm/pgtable.h~x86-mm-convert-ptdump_walk_pgd_level_debugfs-to-take-an-mm_struct +++ a/arch/x86/include/asm/pgtable.h @@ -30,7 +30,8 @@ extern pgd_t early_top_pgt[PTRS_PER_PGD] int __init __early_make_pgtable(unsigned long address, pmdval_t pmd); void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm); -void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user); +void ptdump_walk_pgd_level_debugfs(struct seq_file *m, struct mm_struct *mm, + bool user); void ptdump_walk_pgd_level_checkwx(void); void ptdump_walk_user_pgd_level_checkwx(void); --- a/arch/x86/mm/debug_pagetables.c~x86-mm-convert-ptdump_walk_pgd_level_debugfs-to-take-an-mm_struct +++ a/arch/x86/mm/debug_pagetables.c @@ -7,7 +7,7 @@ static int ptdump_show(struct seq_file *m, void *v) { - ptdump_walk_pgd_level_debugfs(m, NULL, false); + ptdump_walk_pgd_level_debugfs(m, &init_mm, false); return 0; } @@ -17,7 +17,7 @@ static int ptdump_curknl_show(struct seq { if (current->mm->pgd) { down_read(¤t->mm->mmap_sem); - ptdump_walk_pgd_level_debugfs(m, current->mm->pgd, false); + ptdump_walk_pgd_level_debugfs(m, current->mm, false); up_read(¤t->mm->mmap_sem); } return 0; @@ -30,7 +30,7 @@ static int ptdump_curusr_show(struct seq { if (current->mm->pgd) { down_read(¤t->mm->mmap_sem); - ptdump_walk_pgd_level_debugfs(m, current->mm->pgd, true); + ptdump_walk_pgd_level_debugfs(m, current->mm, true); up_read(¤t->mm->mmap_sem); } return 0; @@ -43,7 +43,7 @@ DEFINE_SHOW_ATTRIBUTE(ptdump_curusr); static int ptdump_efi_show(struct seq_file *m, void *v) { if (efi_mm.pgd) - ptdump_walk_pgd_level_debugfs(m, efi_mm.pgd, false); + ptdump_walk_pgd_level_debugfs(m, &efi_mm, false); return 0; } --- a/arch/x86/mm/dump_pagetables.c~x86-mm-convert-ptdump_walk_pgd_level_debugfs-to-take-an-mm_struct +++ a/arch/x86/mm/dump_pagetables.c @@ -518,16 +518,12 @@ static inline bool is_hypervisor_range(i static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, bool checkwx, bool dmesg) { - pgd_t *start = INIT_PGD; + pgd_t *start = pgd; pgprotval_t prot, eff; int i; struct pg_state st = {}; - if (pgd) { - start = pgd; - st.to_dmesg = dmesg; - } - + st.to_dmesg = dmesg; st.check_wx = checkwx; st.seq = m; if (checkwx) @@ -572,8 +568,10 @@ void ptdump_walk_pgd_level(struct seq_fi ptdump_walk_pgd_level_core(m, mm->pgd, false, true); } -void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user) +void ptdump_walk_pgd_level_debugfs(struct seq_file *m, struct mm_struct *mm, + bool user) { + pgd_t *pgd = mm->pgd; #ifdef CONFIG_PAGE_TABLE_ISOLATION if (user && boot_cpu_has(X86_FEATURE_PTI)) pgd = kernel_to_user_pgdp(pgd); @@ -599,7 +597,7 @@ void ptdump_walk_user_pgd_level_checkwx( void ptdump_walk_pgd_level_checkwx(void) { - ptdump_walk_pgd_level_core(NULL, NULL, true, false); + ptdump_walk_pgd_level_core(NULL, INIT_PGD, true, false); } static int __init pt_dump_init(void) From patchwork Sun Dec 1 01:52:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268303 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ECD3C921 for ; Sun, 1 Dec 2019 01:53:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B0E1F2084D for ; Sun, 1 Dec 2019 01:53:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="FuZrS+kI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B0E1F2084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9560E6B02D5; Sat, 30 Nov 2019 20:53:03 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9052D6B02D7; Sat, 30 Nov 2019 20:53:03 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81A9A6B02D8; Sat, 30 Nov 2019 20:53:03 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0063.hostedemail.com [216.40.44.63]) by kanga.kvack.org (Postfix) with ESMTP id 6B8516B02D5 for ; Sat, 30 Nov 2019 20:53:03 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 1FEE2181AEF10 for ; Sun, 1 Dec 2019 01:53:03 +0000 (UTC) X-FDA: 76214899446.16.turn17_1a96218e85609 X-Spam-Summary: 2,0,0,00bdfb068a4e8b26,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:2:41:69:355:379:800 :960:967 X-HE-Tag: turn17_1a96218e85609 X-Filterd-Recvd-Size: 7272 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:53:02 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 6EFFC21774; Sun, 1 Dec 2019 01:53:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165182; bh=v6oBQKSpVT5LKjjFcyiBnGRfKsfDOsv4Y5Ro8ykZGc8=; h=Date:From:To:Subject:From; b=FuZrS+kIvmT6IaLipZuG5cQh40az6/WhuRka/paW3kH1127eRj3ODO0aPI2cQVVGs sFM4ERmyC2yLoQ6EIYkAtw3596M/KHyAc7ziDhG4fL38nuVQVE445yReYaD+oRGUUw t466tXLWq3dTkm5uZSFpnnBvDsrI/uWXP1T0p5O8= Date: Sat, 30 Nov 2019 17:52:59 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 063/158] x86: mm: convert ptdump_walk_pgd_level_core() to take an mm_struct Message-ID: <20191201015259.-XhuujewC%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: x86: mm: convert ptdump_walk_pgd_level_core() to take an mm_struct An mm_struct is needed to enable x86 to use of the generic walk_page_range() function. In the case of walking the user page tables (when CONFIG_PAGE_TABLE_ISOLATION is enabled), it is necessary to create a fake_mm structure because there isn't an mm_struct with a pointer to the pgd of the user page tables. This fake_mm structure is initialised with the minimum necessary for the generic page walk code. Link: http://lkml.kernel.org/r/20191028135910.33253-19-steven.price@arm.com Signed-off-by: Steven Price Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Thomas Gleixner Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Cc: Zong Li Signed-off-by: Andrew Morton --- arch/x86/mm/dump_pagetables.c | 36 ++++++++++++++++++-------------- 1 file changed, 21 insertions(+), 15 deletions(-) --- a/arch/x86/mm/dump_pagetables.c~x86-mm-convert-ptdump_walk_pgd_level_core-to-take-an-mm_struct +++ a/arch/x86/mm/dump_pagetables.c @@ -107,8 +107,6 @@ static struct addr_marker address_marker [END_OF_SPACE_NR] = { -1, NULL } }; -#define INIT_PGD ((pgd_t *) &init_top_pgt) - #else /* CONFIG_X86_64 */ enum address_markers_idx { @@ -143,8 +141,6 @@ static struct addr_marker address_marker [END_OF_SPACE_NR] = { -1, NULL } }; -#define INIT_PGD (swapper_pg_dir) - #endif /* !CONFIG_X86_64 */ /* Multipliers for offsets within the PTEs */ @@ -515,10 +511,10 @@ static inline bool is_hypervisor_range(i #endif } -static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, +static void ptdump_walk_pgd_level_core(struct seq_file *m, struct mm_struct *mm, bool checkwx, bool dmesg) { - pgd_t *start = pgd; + pgd_t *start = mm->pgd; pgprotval_t prot, eff; int i; struct pg_state st = {}; @@ -565,39 +561,49 @@ static void ptdump_walk_pgd_level_core(s void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm) { - ptdump_walk_pgd_level_core(m, mm->pgd, false, true); + ptdump_walk_pgd_level_core(m, mm, false, true); +} + +#ifdef CONFIG_PAGE_TABLE_ISOLATION +static void ptdump_walk_pgd_level_user_core(struct seq_file *m, + struct mm_struct *mm, + bool checkwx, bool dmesg) +{ + struct mm_struct fake_mm = { + .pgd = kernel_to_user_pgdp(mm->pgd) + }; + init_rwsem(&fake_mm.mmap_sem); + ptdump_walk_pgd_level_core(m, &fake_mm, checkwx, dmesg); } +#endif void ptdump_walk_pgd_level_debugfs(struct seq_file *m, struct mm_struct *mm, bool user) { - pgd_t *pgd = mm->pgd; #ifdef CONFIG_PAGE_TABLE_ISOLATION if (user && boot_cpu_has(X86_FEATURE_PTI)) - pgd = kernel_to_user_pgdp(pgd); + ptdump_walk_pgd_level_user_core(m, mm, false, false); + else #endif - ptdump_walk_pgd_level_core(m, pgd, false, false); + ptdump_walk_pgd_level_core(m, mm, false, false); } EXPORT_SYMBOL_GPL(ptdump_walk_pgd_level_debugfs); void ptdump_walk_user_pgd_level_checkwx(void) { #ifdef CONFIG_PAGE_TABLE_ISOLATION - pgd_t *pgd = INIT_PGD; - if (!(__supported_pte_mask & _PAGE_NX) || !boot_cpu_has(X86_FEATURE_PTI)) return; pr_info("x86/mm: Checking user space page tables\n"); - pgd = kernel_to_user_pgdp(pgd); - ptdump_walk_pgd_level_core(NULL, pgd, true, false); + ptdump_walk_pgd_level_user_core(NULL, &init_mm, true, false); #endif } void ptdump_walk_pgd_level_checkwx(void) { - ptdump_walk_pgd_level_core(NULL, INIT_PGD, true, false); + ptdump_walk_pgd_level_core(NULL, &init_mm, true, false); } static int __init pt_dump_init(void) From patchwork Sun Dec 1 01:53:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268305 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CBD8F112B for ; Sun, 1 Dec 2019 01:53:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7CC432084D for ; Sun, 1 Dec 2019 01:53:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZgZMfAwl" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7CC432084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3AF426B02D7; Sat, 30 Nov 2019 20:53:08 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 386BA6B02D9; Sat, 30 Nov 2019 20:53:08 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 29C9A6B02DA; Sat, 30 Nov 2019 20:53:08 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0172.hostedemail.com [216.40.44.172]) by kanga.kvack.org (Postfix) with ESMTP id 14CEB6B02D7 for ; Sat, 30 Nov 2019 20:53:08 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id BC2C5180AD817 for ; Sun, 1 Dec 2019 01:53:07 +0000 (UTC) X-FDA: 76214899614.23.blow01_1b45e0248ca0f X-Spam-Summary: 2,0,0,578d2aef15837e5b,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:cai@lca.pw:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:1:2:41:3 55:379:8 X-HE-Tag: blow01_1b45e0248ca0f X-Filterd-Recvd-Size: 10351 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:53:07 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 492BA208E4; Sun, 1 Dec 2019 01:53:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165186; bh=lynPf64AeTJIXR9UwxUMZBMxdp2QloNXbWDyXUBO3i8=; h=Date:From:To:Subject:From; b=ZgZMfAwlGRYsrhH/PE6Xt0c+VVWACM9VmjXJQemDIWM+3NBsJ2NJ83l15+B7RK3eR 5mtWuu4+MtE2StDeYJdjLzEOIml7cCoWDczM5FqWlrxNZmbMVLHANRplg0vQW7Kild KZK4TZoBkF8Z83sdVz3dcQ8pDMJ+Jnln0Vz9flHE= Date: Sat, 30 Nov 2019 17:53:04 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, cai@lca.pw, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 064/158] mm: add generic ptdump Message-ID: <20191201015304.cRPsmKUTM%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: mm: add generic ptdump Add a generic version of page table dumping that architectures can opt-in to [steven.price@arm.com: v15] Link: http://lkml.kernel.org/r/20191101140942.51554-20-steven.price@arm.com [cai@lca.pw: fix a -Wold-style-declaration warning] Link: http://lkml.kernel.org/r/1572895385-29194-1-git-send-email-cai@lca.pw Link: http://lkml.kernel.org/r/20191028135910.33253-20-steven.price@arm.com Signed-off-by: Steven Price Signed-off-by: Qian Cai Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Cc: Zong Li Signed-off-by: Andrew Morton --- include/linux/ptdump.h | 21 +++++ mm/Kconfig.debug | 21 +++++ mm/Makefile | 1 mm/ptdump.c | 151 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 194 insertions(+) --- /dev/null +++ a/include/linux/ptdump.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _LINUX_PTDUMP_H +#define _LINUX_PTDUMP_H + +#include + +struct ptdump_range { + unsigned long start; + unsigned long end; +}; + +struct ptdump_state { + void (*note_page)(struct ptdump_state *st, unsigned long addr, + int level, unsigned long val); + const struct ptdump_range *range; +}; + +void ptdump_walk_pgd(struct ptdump_state *st, struct mm_struct *mm); + +#endif /* _LINUX_PTDUMP_H */ --- a/mm/Kconfig.debug~mm-add-generic-ptdump +++ a/mm/Kconfig.debug @@ -117,3 +117,24 @@ config DEBUG_RODATA_TEST depends on STRICT_KERNEL_RWX ---help--- This option enables a testcase for the setting rodata read-only. + +config GENERIC_PTDUMP + bool + +config PTDUMP_CORE + bool + +config PTDUMP_DEBUGFS + bool "Export kernel pagetable layout to userspace via debugfs" + depends on DEBUG_KERNEL + depends on DEBUG_FS + depends on GENERIC_PTDUMP + select PTDUMP_CORE + help + Say Y here if you want to show the kernel pagetable layout in a + debugfs file. This information is only useful for kernel developers + who are working in architecture specific areas of the kernel. + It is probably not a good idea to enable this feature in a production + kernel. + + If in doubt, say N. --- a/mm/Makefile~mm-add-generic-ptdump +++ a/mm/Makefile @@ -98,6 +98,7 @@ obj-$(CONFIG_CMA) += cma.o obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o obj-$(CONFIG_PAGE_EXTENSION) += page_ext.o obj-$(CONFIG_CMA_DEBUGFS) += cma_debug.o +obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_USERFAULTFD) += userfaultfd.o obj-$(CONFIG_IDLE_PAGE_TRACKING) += page_idle.o obj-$(CONFIG_FRAME_VECTOR) += frame_vector.o --- /dev/null +++ a/mm/ptdump.c @@ -0,0 +1,151 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include + +static int ptdump_pgd_entry(pgd_t *pgd, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + struct ptdump_state *st = walk->private; + pgd_t val = READ_ONCE(*pgd); + + if (pgd_leaf(val)) + st->note_page(st, addr, 1, pgd_val(val)); + + return 0; +} + +static int ptdump_p4d_entry(p4d_t *p4d, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + struct ptdump_state *st = walk->private; + p4d_t val = READ_ONCE(*p4d); + + if (p4d_leaf(val)) + st->note_page(st, addr, 2, p4d_val(val)); + + return 0; +} + +static int ptdump_pud_entry(pud_t *pud, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + struct ptdump_state *st = walk->private; + pud_t val = READ_ONCE(*pud); + + if (pud_leaf(val)) + st->note_page(st, addr, 3, pud_val(val)); + + return 0; +} + +static int ptdump_pmd_entry(pmd_t *pmd, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + struct ptdump_state *st = walk->private; + pmd_t val = READ_ONCE(*pmd); + + if (pmd_leaf(val)) + st->note_page(st, addr, 4, pmd_val(val)); + + return 0; +} + +static int ptdump_pte_entry(pte_t *pte, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + struct ptdump_state *st = walk->private; + + st->note_page(st, addr, 5, pte_val(READ_ONCE(*pte))); + + return 0; +} + +#ifdef CONFIG_KASAN +/* + * This is an optimization for KASAN=y case. Since all kasan page tables + * eventually point to the kasan_early_shadow_page we could call note_page() + * right away without walking through lower level page tables. This saves + * us dozens of seconds (minutes for 5-level config) while checking for + * W+X mapping or reading kernel_page_tables debugfs file. + */ +static inline int note_kasan_page_table(struct mm_walk *walk, + unsigned long addr) +{ + struct ptdump_state *st = walk->private; + + st->note_page(st, addr, 5, pte_val(kasan_early_shadow_pte[0])); + return 1; +} + +static int ptdump_test_p4d(unsigned long addr, unsigned long next, + p4d_t *p4d, struct mm_walk *walk) +{ +#if CONFIG_PGTABLE_LEVELS > 4 + if (p4d == lm_alias(kasan_early_shadow_p4d)) + return note_kasan_page_table(walk, addr); +#endif + return 0; +} + +static int ptdump_test_pud(unsigned long addr, unsigned long next, + pud_t *pud, struct mm_walk *walk) +{ +#if CONFIG_PGTABLE_LEVELS > 3 + if (pud == lm_alias(kasan_early_shadow_pud)) + return note_kasan_page_table(walk, addr); +#endif + return 0; +} + +static int ptdump_test_pmd(unsigned long addr, unsigned long next, + pmd_t *pmd, struct mm_walk *walk) +{ +#if CONFIG_PGTABLE_LEVELS > 2 + if (pmd == lm_alias(kasan_early_shadow_pmd)) + return note_kasan_page_table(walk, addr); +#endif + return 0; +} +#endif /* CONFIG_KASAN */ + +static int ptdump_hole(unsigned long addr, unsigned long next, + int depth, struct mm_walk *walk) +{ + struct ptdump_state *st = walk->private; + + st->note_page(st, addr, depth + 1, 0); + + return 0; +} + +static const struct mm_walk_ops ptdump_ops = { + .pgd_entry = ptdump_pgd_entry, + .p4d_entry = ptdump_p4d_entry, + .pud_entry = ptdump_pud_entry, + .pmd_entry = ptdump_pmd_entry, + .pte_entry = ptdump_pte_entry, +#ifdef CONFIG_KASAN + .test_p4d = ptdump_test_p4d, + .test_pud = ptdump_test_pud, + .test_pmd = ptdump_test_pmd, +#endif + .pte_hole = ptdump_hole, +}; + +void ptdump_walk_pgd(struct ptdump_state *st, struct mm_struct *mm) +{ + const struct ptdump_range *range = st->range; + + down_read(&mm->mmap_sem); + while (range->start != range->end) { + walk_page_range_novma(mm, range->start, range->end, + &ptdump_ops, st); + range++; + } + up_read(&mm->mmap_sem); + + /* Flush out the last page */ + st->note_page(st, 0, 0, 0); +} From patchwork Sun Dec 1 01:53:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268307 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C0700112B for ; Sun, 1 Dec 2019 01:53:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 731DF2231B for ; Sun, 1 Dec 2019 01:53:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="nCgfgUt1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 731DF2231B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4242C6B02D9; Sat, 30 Nov 2019 20:53:13 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3D4F36B02DB; Sat, 30 Nov 2019 20:53:13 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 29C8B6B02DC; Sat, 30 Nov 2019 20:53:13 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0221.hostedemail.com [216.40.44.221]) by kanga.kvack.org (Postfix) with ESMTP id 113956B02D9 for ; Sat, 30 Nov 2019 20:53:13 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id BB7BE180AD817 for ; Sun, 1 Dec 2019 01:53:12 +0000 (UTC) X-FDA: 76214899824.19.wax04_1bf7f530a2339 X-Spam-Summary: 2,0,0,740158fc529c6a12,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:4:41:69:355:379:800 :960:967 X-HE-Tag: wax04_1bf7f530a2339 X-Filterd-Recvd-Size: 18848 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:53:12 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id F34222084D; Sun, 1 Dec 2019 01:53:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165191; bh=5Mowcje64rf/8OitS8EhlaM/Fl3G9h31Nltsjm5LXIw=; h=Date:From:To:Subject:From; b=nCgfgUt1mwO2XULyoxiYtiW0qmWCv2ivcqGvCDduELq6bsiyoe2pt6Qqbr5zcld9B Mcrbz4K4rTftnSuDLkBumPt4HgnKaeGJufoumO20Kr6S/uXyqcNQHvSJe7vdWvWDpN yjsz9xGzwRni9pn5EpO2bFjcR0W8rn5ehiUiikUI= Date: Sat, 30 Nov 2019 17:53:09 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 065/158] x86: mm: convert dump_pagetables to use walk_page_range Message-ID: <20191201015309.OuQqjf-Sk%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: x86: mm: convert dump_pagetables to use walk_page_range Make use of the new functionality in walk_page_range to remove the arch page walking code and use the generic code to walk the page tables. The effective permissions are passed down the chain using new fields in struct pg_state. The KASAN optimisation is implemented by including test_p?d callbacks which can decide to skip an entire tree of entries Link: http://lkml.kernel.org/r/20191028135910.33253-21-steven.price@arm.com Signed-off-by: Steven Price Cc: Andrey Ryabinin Cc: Alexander Potapenko Cc: Dmitry Vyukov Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: Thomas Gleixner Cc: Albert Ou Cc: Alexandre Ghiti Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Heiko Carstens Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Cc: Zong Li Signed-off-by: Andrew Morton --- arch/x86/Kconfig | 1 arch/x86/Kconfig.debug | 20 -- arch/x86/mm/Makefile | 4 arch/x86/mm/dump_pagetables.c | 289 ++++++-------------------------- 4 files changed, 66 insertions(+), 248 deletions(-) --- a/arch/x86/Kconfig~x86-mm-convert-dump_pagetables-to-use-walk_page_range +++ a/arch/x86/Kconfig @@ -119,6 +119,7 @@ config X86 select GENERIC_IRQ_RESERVATION_MODE select GENERIC_IRQ_SHOW select GENERIC_PENDING_IRQ if SMP + select GENERIC_PTDUMP select GENERIC_SMP_IDLE_THREAD select GENERIC_STRNCPY_FROM_USER select GENERIC_STRNLEN_USER --- a/arch/x86/Kconfig.debug~x86-mm-convert-dump_pagetables-to-use-walk_page_range +++ a/arch/x86/Kconfig.debug @@ -62,26 +62,10 @@ config EARLY_PRINTK_USB_XDBC config MCSAFE_TEST def_bool n -config X86_PTDUMP_CORE - def_bool n - -config X86_PTDUMP - tristate "Export kernel pagetable layout to userspace via debugfs" - depends on DEBUG_KERNEL - select DEBUG_FS - select X86_PTDUMP_CORE - ---help--- - Say Y here if you want to show the kernel pagetable layout in a - debugfs file. This information is only useful for kernel developers - who are working in architecture specific areas of the kernel. - It is probably not a good idea to enable this feature in a production - kernel. - If in doubt, say "N" - config EFI_PGT_DUMP bool "Dump the EFI pagetable" depends on EFI - select X86_PTDUMP_CORE + select PTDUMP_CORE ---help--- Enable this if you want to dump the EFI page table before enabling virtual mode. This can be used to debug miscellaneous @@ -90,7 +74,7 @@ config EFI_PGT_DUMP config DEBUG_WX bool "Warn on W+X mappings at boot" - select X86_PTDUMP_CORE + select PTDUMP_CORE ---help--- Generate a warning if any W+X mappings are found at boot. --- a/arch/x86/mm/dump_pagetables.c~x86-mm-convert-dump_pagetables-to-use-walk_page_range +++ a/arch/x86/mm/dump_pagetables.c @@ -16,6 +16,7 @@ #include #include #include +#include #include #include @@ -26,11 +27,12 @@ * when a "break" in the continuity is found. */ struct pg_state { + struct ptdump_state ptdump; int level; - pgprot_t current_prot; + pgprotval_t current_prot; pgprotval_t effective_prot; + pgprotval_t prot_levels[5]; unsigned long start_address; - unsigned long current_address; const struct addr_marker *marker; unsigned long lines; bool to_dmesg; @@ -171,9 +173,8 @@ static struct addr_marker address_marker /* * Print a readable form of a pgprot_t to the seq_file */ -static void printk_prot(struct seq_file *m, pgprot_t prot, int level, bool dmsg) +static void printk_prot(struct seq_file *m, pgprotval_t pr, int level, bool dmsg) { - pgprotval_t pr = pgprot_val(prot); static const char * const level_name[] = { "cr3", "pgd", "p4d", "pud", "pmd", "pte" }; @@ -220,24 +221,11 @@ static void printk_prot(struct seq_file pt_dump_cont_printf(m, dmsg, "%s\n", level_name[level]); } -/* - * On 64 bits, sign-extend the 48 bit address to 64 bit - */ -static unsigned long normalize_addr(unsigned long u) -{ - int shift; - if (!IS_ENABLED(CONFIG_X86_64)) - return u; - - shift = 64 - (__VIRTUAL_MASK_SHIFT + 1); - return (signed long)(u << shift) >> shift; -} - -static void note_wx(struct pg_state *st) +static void note_wx(struct pg_state *st, unsigned long addr) { unsigned long npages; - npages = (st->current_address - st->start_address) / PAGE_SIZE; + npages = (addr - st->start_address) / PAGE_SIZE; #ifdef CONFIG_PCI_BIOS /* @@ -245,7 +233,7 @@ static void note_wx(struct pg_state *st) * Inform about it, but avoid the warning. */ if (pcibios_enabled && st->start_address >= PAGE_OFFSET + BIOS_BEGIN && - st->current_address <= PAGE_OFFSET + BIOS_END) { + addr <= PAGE_OFFSET + BIOS_END) { pr_warn_once("x86/mm: PCI BIOS W+X mapping %lu pages\n", npages); return; } @@ -257,25 +245,44 @@ static void note_wx(struct pg_state *st) (void *)st->start_address); } +static inline pgprotval_t effective_prot(pgprotval_t prot1, pgprotval_t prot2) +{ + return (prot1 & prot2 & (_PAGE_USER | _PAGE_RW)) | + ((prot1 | prot2) & _PAGE_NX); +} + /* * This function gets called on a break in a continuous series * of PTE entries; the next one is different so we need to * print what we collected so far. */ -static void note_page(struct pg_state *st, pgprot_t new_prot, - pgprotval_t new_eff, int level) +static void note_page(struct ptdump_state *pt_st, unsigned long addr, int level, + unsigned long val) { - pgprotval_t prot, cur, eff; + struct pg_state *st = container_of(pt_st, struct pg_state, ptdump); + pgprotval_t new_prot, new_eff; + pgprotval_t cur, eff; static const char units[] = "BKMGTPE"; struct seq_file *m = st->seq; + new_prot = val & PTE_FLAGS_MASK; + + if (level > 1) { + new_eff = effective_prot(st->prot_levels[level - 2], + new_prot); + } else { + new_eff = new_prot; + } + + if (level > 0) + st->prot_levels[level - 1] = new_eff; + /* * If we have a "break" in the series, we need to flush the state that * we have now. "break" is either changing perms, levels or * address space marker. */ - prot = pgprot_val(new_prot); - cur = pgprot_val(st->current_prot); + cur = st->current_prot; eff = st->effective_prot; if (!st->level) { @@ -287,14 +294,14 @@ static void note_page(struct pg_state *s st->lines = 0; pt_dump_seq_printf(m, st->to_dmesg, "---[ %s ]---\n", st->marker->name); - } else if (prot != cur || new_eff != eff || level != st->level || - st->current_address >= st->marker[1].start_address) { + } else if (new_prot != cur || new_eff != eff || level != st->level || + addr >= st->marker[1].start_address) { const char *unit = units; unsigned long delta; int width = sizeof(unsigned long) * 2; if (st->check_wx && (eff & _PAGE_RW) && !(eff & _PAGE_NX)) - note_wx(st); + note_wx(st, addr); /* * Now print the actual finished series @@ -304,9 +311,9 @@ static void note_page(struct pg_state *s pt_dump_seq_printf(m, st->to_dmesg, "0x%0*lx-0x%0*lx ", width, st->start_address, - width, st->current_address); + width, addr); - delta = st->current_address - st->start_address; + delta = addr - st->start_address; while (!(delta & 1023) && unit[1]) { delta >>= 10; unit++; @@ -323,7 +330,7 @@ static void note_page(struct pg_state *s * such as the start of vmalloc space etc. * This helps in the interpretation. */ - if (st->current_address >= st->marker[1].start_address) { + if (addr >= st->marker[1].start_address) { if (st->marker->max_lines && st->lines > st->marker->max_lines) { unsigned long nskip = @@ -339,217 +346,43 @@ static void note_page(struct pg_state *s st->marker->name); } - st->start_address = st->current_address; + st->start_address = addr; st->current_prot = new_prot; st->effective_prot = new_eff; st->level = level; } } -static inline pgprotval_t effective_prot(pgprotval_t prot1, pgprotval_t prot2) -{ - return (prot1 & prot2 & (_PAGE_USER | _PAGE_RW)) | - ((prot1 | prot2) & _PAGE_NX); -} - -static void walk_pte_level(struct pg_state *st, pmd_t addr, pgprotval_t eff_in, - unsigned long P) -{ - int i; - pte_t *pte; - pgprotval_t prot, eff; - - for (i = 0; i < PTRS_PER_PTE; i++) { - st->current_address = normalize_addr(P + i * PTE_LEVEL_MULT); - pte = pte_offset_map(&addr, st->current_address); - prot = pte_flags(*pte); - eff = effective_prot(eff_in, prot); - note_page(st, __pgprot(prot), eff, 5); - pte_unmap(pte); - } -} -#ifdef CONFIG_KASAN - -/* - * This is an optimization for KASAN=y case. Since all kasan page tables - * eventually point to the kasan_early_shadow_page we could call note_page() - * right away without walking through lower level page tables. This saves - * us dozens of seconds (minutes for 5-level config) while checking for - * W+X mapping or reading kernel_page_tables debugfs file. - */ -static inline bool kasan_page_table(struct pg_state *st, void *pt) -{ - if (__pa(pt) == __pa(kasan_early_shadow_pmd) || - (pgtable_l5_enabled() && - __pa(pt) == __pa(kasan_early_shadow_p4d)) || - __pa(pt) == __pa(kasan_early_shadow_pud)) { - pgprotval_t prot = pte_flags(kasan_early_shadow_pte[0]); - note_page(st, __pgprot(prot), 0, 5); - return true; - } - return false; -} -#else -static inline bool kasan_page_table(struct pg_state *st, void *pt) -{ - return false; -} -#endif - -#if PTRS_PER_PMD > 1 - -static void walk_pmd_level(struct pg_state *st, pud_t addr, - pgprotval_t eff_in, unsigned long P) -{ - int i; - pmd_t *start, *pmd_start; - pgprotval_t prot, eff; - - pmd_start = start = (pmd_t *)pud_page_vaddr(addr); - for (i = 0; i < PTRS_PER_PMD; i++) { - st->current_address = normalize_addr(P + i * PMD_LEVEL_MULT); - if (!pmd_none(*start)) { - prot = pmd_flags(*start); - eff = effective_prot(eff_in, prot); - if (pmd_large(*start) || !pmd_present(*start)) { - note_page(st, __pgprot(prot), eff, 4); - } else if (!kasan_page_table(st, pmd_start)) { - walk_pte_level(st, *start, eff, - P + i * PMD_LEVEL_MULT); - } - } else - note_page(st, __pgprot(0), 0, 4); - start++; - } -} - -#else -#define walk_pmd_level(s,a,e,p) walk_pte_level(s,__pmd(pud_val(a)),e,p) -#define pud_large(a) pmd_large(__pmd(pud_val(a))) -#define pud_none(a) pmd_none(__pmd(pud_val(a))) -#endif - -#if PTRS_PER_PUD > 1 - -static void walk_pud_level(struct pg_state *st, p4d_t addr, pgprotval_t eff_in, - unsigned long P) -{ - int i; - pud_t *start, *pud_start; - pgprotval_t prot, eff; - - pud_start = start = (pud_t *)p4d_page_vaddr(addr); - - for (i = 0; i < PTRS_PER_PUD; i++) { - st->current_address = normalize_addr(P + i * PUD_LEVEL_MULT); - if (!pud_none(*start)) { - prot = pud_flags(*start); - eff = effective_prot(eff_in, prot); - if (pud_large(*start) || !pud_present(*start)) { - note_page(st, __pgprot(prot), eff, 3); - } else if (!kasan_page_table(st, pud_start)) { - walk_pmd_level(st, *start, eff, - P + i * PUD_LEVEL_MULT); - } - } else - note_page(st, __pgprot(0), 0, 3); - - start++; - } -} - -#else -#define walk_pud_level(s,a,e,p) walk_pmd_level(s,__pud(p4d_val(a)),e,p) -#define p4d_large(a) pud_large(__pud(p4d_val(a))) -#define p4d_none(a) pud_none(__pud(p4d_val(a))) -#endif - -static void walk_p4d_level(struct pg_state *st, pgd_t addr, pgprotval_t eff_in, - unsigned long P) +static void ptdump_walk_pgd_level_core(struct seq_file *m, struct mm_struct *mm, + bool checkwx, bool dmesg) { - int i; - p4d_t *start, *p4d_start; - pgprotval_t prot, eff; - - if (PTRS_PER_P4D == 1) - return walk_pud_level(st, __p4d(pgd_val(addr)), eff_in, P); - - p4d_start = start = (p4d_t *)pgd_page_vaddr(addr); - - for (i = 0; i < PTRS_PER_P4D; i++) { - st->current_address = normalize_addr(P + i * P4D_LEVEL_MULT); - if (!p4d_none(*start)) { - prot = p4d_flags(*start); - eff = effective_prot(eff_in, prot); - if (p4d_large(*start) || !p4d_present(*start)) { - note_page(st, __pgprot(prot), eff, 2); - } else if (!kasan_page_table(st, p4d_start)) { - walk_pud_level(st, *start, eff, - P + i * P4D_LEVEL_MULT); - } - } else - note_page(st, __pgprot(0), 0, 2); - - start++; - } -} + const struct ptdump_range ptdump_ranges[] = { +#ifdef CONFIG_X86_64 -#define pgd_large(a) (pgtable_l5_enabled() ? pgd_large(a) : p4d_large(__p4d(pgd_val(a)))) -#define pgd_none(a) (pgtable_l5_enabled() ? pgd_none(a) : p4d_none(__p4d(pgd_val(a)))) +#define normalize_addr_shift (64 - (__VIRTUAL_MASK_SHIFT + 1)) +#define normalize_addr(u) ((signed long)((u) << normalize_addr_shift) >> \ + normalize_addr_shift) -static inline bool is_hypervisor_range(int idx) -{ -#ifdef CONFIG_X86_64 - /* - * A hole in the beginning of kernel address space reserved - * for a hypervisor. - */ - return (idx >= pgd_index(GUARD_HOLE_BASE_ADDR)) && - (idx < pgd_index(GUARD_HOLE_END_ADDR)); + {0, PTRS_PER_PGD * PGD_LEVEL_MULT / 2}, + {normalize_addr(PTRS_PER_PGD * PGD_LEVEL_MULT / 2), ~0UL}, #else - return false; + {0, ~0UL}, #endif -} + {0, 0} +}; -static void ptdump_walk_pgd_level_core(struct seq_file *m, struct mm_struct *mm, - bool checkwx, bool dmesg) -{ - pgd_t *start = mm->pgd; - pgprotval_t prot, eff; - int i; - struct pg_state st = {}; - - st.to_dmesg = dmesg; - st.check_wx = checkwx; - st.seq = m; - if (checkwx) - st.wx_pages = 0; - - for (i = 0; i < PTRS_PER_PGD; i++) { - st.current_address = normalize_addr(i * PGD_LEVEL_MULT); - if (!pgd_none(*start) && !is_hypervisor_range(i)) { - prot = pgd_flags(*start); -#ifdef CONFIG_X86_PAE - eff = _PAGE_USER | _PAGE_RW; -#else - eff = prot; -#endif - if (pgd_large(*start) || !pgd_present(*start)) { - note_page(&st, __pgprot(prot), eff, 1); - } else { - walk_p4d_level(&st, *start, eff, - i * PGD_LEVEL_MULT); - } - } else - note_page(&st, __pgprot(0), 0, 1); + struct pg_state st = { + .ptdump = { + .note_page = note_page, + .range = ptdump_ranges + }, + .to_dmesg = dmesg, + .check_wx = checkwx, + .seq = m + }; - cond_resched(); - start++; - } + ptdump_walk_pgd(&st.ptdump, mm); - /* Flush out the last page */ - st.current_address = normalize_addr(PTRS_PER_PGD*PGD_LEVEL_MULT); - note_page(&st, __pgprot(0), 0, 0); if (!checkwx) return; if (st.wx_pages) --- a/arch/x86/mm/Makefile~x86-mm-convert-dump_pagetables-to-use-walk_page_range +++ a/arch/x86/mm/Makefile @@ -28,8 +28,8 @@ obj-$(CONFIG_X86_PAT) += pat_interval.o obj-$(CONFIG_X86_32) += pgtable_32.o iomap_32.o obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o -obj-$(CONFIG_X86_PTDUMP_CORE) += dump_pagetables.o -obj-$(CONFIG_X86_PTDUMP) += debug_pagetables.o +obj-$(CONFIG_PTDUMP_CORE) += dump_pagetables.o +obj-$(CONFIG_PTDUMP_DEBUGFS) += debug_pagetables.o obj-$(CONFIG_HIGHMEM) += highmem_32.o From patchwork Sun Dec 1 01:53:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268309 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 03311921 for ; Sun, 1 Dec 2019 01:53:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A905E206E1 for ; Sun, 1 Dec 2019 01:53:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="XpMnAZPP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A905E206E1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6BBC76B02DB; Sat, 30 Nov 2019 20:53:18 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 66CE66B02DD; Sat, 30 Nov 2019 20:53:18 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A9CA6B02DE; Sat, 30 Nov 2019 20:53:18 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0221.hostedemail.com [216.40.44.221]) by kanga.kvack.org (Postfix) with ESMTP id 4123C6B02DB for ; Sat, 30 Nov 2019 20:53:18 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id EF998824999B for ; Sun, 1 Dec 2019 01:53:17 +0000 (UTC) X-FDA: 76214900034.24.rake34_1cbbb67061c4f X-Spam-Summary: 2,0,0,e365d77277ae5b30,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:1:41:69:355:379:800 :960:967 X-HE-Tag: rake34_1cbbb67061c4f X-Filterd-Recvd-Size: 13594 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf17.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:53:17 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C9BC72231B; Sun, 1 Dec 2019 01:53:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165196; bh=ZcmYncQeqvtF3FahznxAY8iyoyObPc/lvxLFbHkBv5w=; h=Date:From:To:Subject:From; b=XpMnAZPP1kgrcrthjlZnuxaB/wHFDuM7Fzd2LEwqFr/zjhFxmpCKz/K0pI7FTKLBA REtVW2JQl5GlHH3Gb9X5FkX78SShRugmJY4PtzIDct0nm1PUHaOobVqtffIdSy2WLF k7sj2LaotwPXNHJVCN+1zi2dizS6HaHbJW9bl2KM= Date: Sat, 30 Nov 2019 17:53:14 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 066/158] arm64: mm: convert mm/dump.c to use walk_page_range() Message-ID: <20191201015314.OVUDrvrBZ%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: arm64: mm: convert mm/dump.c to use walk_page_range() Now walk_page_range() can walk kernel page tables, we can switch the arm64 ptdump code over to using it, simplifying the code. Link: http://lkml.kernel.org/r/20191028135910.33253-22-steven.price@arm.com Signed-off-by: Steven Price Reviewed-by: Catalin Marinas Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Will Deacon Cc: Zong Li Signed-off-by: Andrew Morton --- arch/arm64/Kconfig | 1 arch/arm64/Kconfig.debug | 19 ---- arch/arm64/include/asm/ptdump.h | 8 - arch/arm64/mm/Makefile | 4 arch/arm64/mm/dump.c | 117 ++++++++------------------- arch/arm64/mm/mmu.c | 4 arch/arm64/mm/ptdump_debugfs.c | 2 drivers/firmware/efi/arm-runtime.c | 2 8 files changed, 50 insertions(+), 107 deletions(-) --- a/arch/arm64/include/asm/ptdump.h~arm64-mm-convert-mm-dumpc-to-use-walk_page_range +++ a/arch/arm64/include/asm/ptdump.h @@ -5,7 +5,7 @@ #ifndef __ASM_PTDUMP_H #define __ASM_PTDUMP_H -#ifdef CONFIG_ARM64_PTDUMP_CORE +#ifdef CONFIG_PTDUMP_CORE #include #include @@ -21,15 +21,15 @@ struct ptdump_info { unsigned long base_addr; }; -void ptdump_walk_pgd(struct seq_file *s, struct ptdump_info *info); -#ifdef CONFIG_ARM64_PTDUMP_DEBUGFS +void ptdump_walk(struct seq_file *s, struct ptdump_info *info); +#ifdef CONFIG_PTDUMP_DEBUGFS void ptdump_debugfs_register(struct ptdump_info *info, const char *name); #else static inline void ptdump_debugfs_register(struct ptdump_info *info, const char *name) { } #endif void ptdump_check_wx(void); -#endif /* CONFIG_ARM64_PTDUMP_CORE */ +#endif /* CONFIG_PTDUMP_CORE */ #ifdef CONFIG_DEBUG_WX #define debug_checkwx() ptdump_check_wx() --- a/arch/arm64/Kconfig~arm64-mm-convert-mm-dumpc-to-use-walk_page_range +++ a/arch/arm64/Kconfig @@ -103,6 +103,7 @@ config ARM64 select GENERIC_IRQ_SHOW select GENERIC_IRQ_SHOW_LEVEL select GENERIC_PCI_IOMAP + select GENERIC_PTDUMP select GENERIC_SCHED_CLOCK select GENERIC_SMP_IDLE_THREAD select GENERIC_STRNCPY_FROM_USER --- a/arch/arm64/Kconfig.debug~arm64-mm-convert-mm-dumpc-to-use-walk_page_range +++ a/arch/arm64/Kconfig.debug @@ -1,22 +1,5 @@ # SPDX-License-Identifier: GPL-2.0-only -config ARM64_PTDUMP_CORE - def_bool n - -config ARM64_PTDUMP_DEBUGFS - bool "Export kernel pagetable layout to userspace via debugfs" - depends on DEBUG_KERNEL - select ARM64_PTDUMP_CORE - select DEBUG_FS - help - Say Y here if you want to show the kernel pagetable layout in a - debugfs file. This information is only useful for kernel developers - who are working in architecture specific areas of the kernel. - It is probably not a good idea to enable this feature in a production - kernel. - - If in doubt, say N. - config PID_IN_CONTEXTIDR bool "Write the current PID to the CONTEXTIDR register" help @@ -42,7 +25,7 @@ config ARM64_RANDOMIZE_TEXT_OFFSET config DEBUG_WX bool "Warn on W+X mappings at boot" - select ARM64_PTDUMP_CORE + select PTDUMP_CORE ---help--- Generate a warning if any W+X mappings are found at boot. --- a/arch/arm64/mm/dump.c~arm64-mm-convert-mm-dumpc-to-use-walk_page_range +++ a/arch/arm64/mm/dump.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include @@ -75,10 +76,11 @@ static struct addr_marker address_marker * dumps out a description of the range. */ struct pg_state { + struct ptdump_state ptdump; struct seq_file *seq; const struct addr_marker *marker; unsigned long start_address; - unsigned level; + int level; u64 current_prot; bool check_wx; unsigned long wx_pages; @@ -178,6 +180,10 @@ static struct pg_level pg_level[] = { .name = "PGD", .bits = pte_bits, .num = ARRAY_SIZE(pte_bits), + }, { /* p4d */ + .name = "P4D", + .bits = pte_bits, + .num = ARRAY_SIZE(pte_bits), }, { /* pud */ .name = (CONFIG_PGTABLE_LEVELS > 3) ? "PUD" : "PGD", .bits = pte_bits, @@ -240,11 +246,15 @@ static void note_prot_wx(struct pg_state st->wx_pages += (addr - st->start_address) / PAGE_SIZE; } -static void note_page(struct pg_state *st, unsigned long addr, unsigned level, - u64 val) +static void note_page(struct ptdump_state *pt_st, unsigned long addr, int level, + unsigned long val) { + struct pg_state *st = container_of(pt_st, struct pg_state, ptdump); static const char units[] = "KMGTPE"; - u64 prot = val & pg_level[level].mask; + u64 prot = 0; + + if (level >= 0) + prot = val & pg_level[level].mask; if (!st->level) { st->level = level; @@ -292,85 +302,27 @@ static void note_page(struct pg_state *s } -static void walk_pte(struct pg_state *st, pmd_t *pmdp, unsigned long start, - unsigned long end) -{ - unsigned long addr = start; - pte_t *ptep = pte_offset_kernel(pmdp, start); - - do { - note_page(st, addr, 4, READ_ONCE(pte_val(*ptep))); - } while (ptep++, addr += PAGE_SIZE, addr != end); -} - -static void walk_pmd(struct pg_state *st, pud_t *pudp, unsigned long start, - unsigned long end) +void ptdump_walk(struct seq_file *s, struct ptdump_info *info) { - unsigned long next, addr = start; - pmd_t *pmdp = pmd_offset(pudp, start); - - do { - pmd_t pmd = READ_ONCE(*pmdp); - next = pmd_addr_end(addr, end); - - if (pmd_none(pmd) || pmd_sect(pmd)) { - note_page(st, addr, 3, pmd_val(pmd)); - } else { - BUG_ON(pmd_bad(pmd)); - walk_pte(st, pmdp, addr, next); - } - } while (pmdp++, addr = next, addr != end); -} - -static void walk_pud(struct pg_state *st, pgd_t *pgdp, unsigned long start, - unsigned long end) -{ - unsigned long next, addr = start; - pud_t *pudp = pud_offset(pgdp, start); - - do { - pud_t pud = READ_ONCE(*pudp); - next = pud_addr_end(addr, end); - - if (pud_none(pud) || pud_sect(pud)) { - note_page(st, addr, 2, pud_val(pud)); - } else { - BUG_ON(pud_bad(pud)); - walk_pmd(st, pudp, addr, next); - } - } while (pudp++, addr = next, addr != end); -} + unsigned long end = ~0UL; + struct pg_state st; -static void walk_pgd(struct pg_state *st, struct mm_struct *mm, - unsigned long start) -{ - unsigned long end = (start < TASK_SIZE_64) ? TASK_SIZE_64 : 0; - unsigned long next, addr = start; - pgd_t *pgdp = pgd_offset(mm, start); - - do { - pgd_t pgd = READ_ONCE(*pgdp); - next = pgd_addr_end(addr, end); - - if (pgd_none(pgd)) { - note_page(st, addr, 1, pgd_val(pgd)); - } else { - BUG_ON(pgd_bad(pgd)); - walk_pud(st, pgdp, addr, next); - } - } while (pgdp++, addr = next, addr != end); -} + if (info->base_addr < TASK_SIZE_64) + end = TASK_SIZE_64; -void ptdump_walk_pgd(struct seq_file *m, struct ptdump_info *info) -{ - struct pg_state st = { - .seq = m, + st = (struct pg_state){ + .seq = s, .marker = info->markers, + .ptdump = { + .note_page = note_page, + .range = (struct ptdump_range[]){ + {info->base_addr, end}, + {0, 0} + } + } }; - walk_pgd(&st, info->mm, info->base_addr); - - note_page(&st, 0, 0, 0); + ptdump_walk_pgd(&st.ptdump, info->mm); } static void ptdump_initialize(void) @@ -398,10 +350,17 @@ void ptdump_check_wx(void) { -1, NULL}, }, .check_wx = true, + .ptdump = { + .note_page = note_page, + .range = (struct ptdump_range[]) { + {PAGE_OFFSET, ~0UL}, + {0, 0} + } + } }; - walk_pgd(&st, &init_mm, PAGE_OFFSET); - note_page(&st, 0, 0, 0); + ptdump_walk_pgd(&st.ptdump, &init_mm); + if (st.wx_pages || st.uxn_pages) pr_warn("Checked W+X mappings: FAILED, %lu W+X pages found, %lu non-UXN pages found\n", st.wx_pages, st.uxn_pages); --- a/arch/arm64/mm/Makefile~arm64-mm-convert-mm-dumpc-to-use-walk_page_range +++ a/arch/arm64/mm/Makefile @@ -4,8 +4,8 @@ obj-y := dma-mapping.o extable.o faul ioremap.o mmap.o pgd.o mmu.o \ context.o proc.o pageattr.o obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o -obj-$(CONFIG_ARM64_PTDUMP_CORE) += dump.o -obj-$(CONFIG_ARM64_PTDUMP_DEBUGFS) += ptdump_debugfs.o +obj-$(CONFIG_PTDUMP_CORE) += dump.o +obj-$(CONFIG_PTDUMP_DEBUGFS) += ptdump_debugfs.o obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o KASAN_SANITIZE_physaddr.o += n --- a/arch/arm64/mm/mmu.c~arm64-mm-convert-mm-dumpc-to-use-walk_page_range +++ a/arch/arm64/mm/mmu.c @@ -943,13 +943,13 @@ int __init arch_ioremap_pud_supported(vo * SW table walks can't handle removal of intermediate entries. */ return IS_ENABLED(CONFIG_ARM64_4K_PAGES) && - !IS_ENABLED(CONFIG_ARM64_PTDUMP_DEBUGFS); + !IS_ENABLED(CONFIG_PTDUMP_DEBUGFS); } int __init arch_ioremap_pmd_supported(void) { /* See arch_ioremap_pud_supported() */ - return !IS_ENABLED(CONFIG_ARM64_PTDUMP_DEBUGFS); + return !IS_ENABLED(CONFIG_PTDUMP_DEBUGFS); } int pud_set_huge(pud_t *pudp, phys_addr_t phys, pgprot_t prot) --- a/arch/arm64/mm/ptdump_debugfs.c~arm64-mm-convert-mm-dumpc-to-use-walk_page_range +++ a/arch/arm64/mm/ptdump_debugfs.c @@ -7,7 +7,7 @@ static int ptdump_show(struct seq_file *m, void *v) { struct ptdump_info *info = m->private; - ptdump_walk_pgd(m, info); + ptdump_walk(m, info); return 0; } DEFINE_SHOW_ATTRIBUTE(ptdump); --- a/drivers/firmware/efi/arm-runtime.c~arm64-mm-convert-mm-dumpc-to-use-walk_page_range +++ a/drivers/firmware/efi/arm-runtime.c @@ -27,7 +27,7 @@ extern u64 efi_system_table; -#ifdef CONFIG_ARM64_PTDUMP_DEBUGFS +#if defined(CONFIG_PTDUMP_DEBUGFS) && defined(CONFIG_ARM64) #include static struct ptdump_info efi_ptdump_info = { From patchwork Sun Dec 1 01:53:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268311 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 63AB6112B for ; Sun, 1 Dec 2019 01:53:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 26D7F208E4 for ; Sun, 1 Dec 2019 01:53:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="PiYuG0yL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 26D7F208E4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0C9F36B02DD; Sat, 30 Nov 2019 20:53:23 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0A08F6B02DF; Sat, 30 Nov 2019 20:53:23 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F21676B02E0; Sat, 30 Nov 2019 20:53:22 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0214.hostedemail.com [216.40.44.214]) by kanga.kvack.org (Postfix) with ESMTP id DA4D66B02DD for ; Sat, 30 Nov 2019 20:53:22 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 91A132C8A for ; Sun, 1 Dec 2019 01:53:22 +0000 (UTC) X-FDA: 76214900244.27.week99_1d6ca53120626 X-Spam-Summary: 2,0,0,26ac0be63ccd435c,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:41:69:355:379:800:9 60:967:9 X-HE-Tag: week99_1d6ca53120626 X-Filterd-Recvd-Size: 5617 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:53:22 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id EEAF422464; Sun, 1 Dec 2019 01:53:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165201; bh=le+052GupQFrXn3tJVaYpMzxv8sUgRRzNQJfbraKxXM=; h=Date:From:To:Subject:From; b=PiYuG0yLMPpO4247PKp+TYdlNrpSfqimxW90FR1Edt6m7wqJ0+OSCR85vx44O0j6W YQFOrJN0L8azsPvOC8Y4wOT9TNd0omH4KH+TSzwmR72i54uZYb4ZL8m2VopvEeIUeo 382LaeHXYKBaQq4+7PnWDvpROXaQfinRz4p5B8ug= Date: Sat, 30 Nov 2019 17:53:19 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 067/158] arm64: mm: display non-present entries in ptdump Message-ID: <20191201015319.jVZT2pWhn%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: arm64: mm: display non-present entries in ptdump Previously the /sys/kernel/debug/kernel_page_tables file would only show lines for entries present in the page tables. However it is useful to also show non-present entries as this makes the size and level of the holes more visible. This aligns the behaviour with x86 which also shows holes. Link: http://lkml.kernel.org/r/20191028135910.33253-23-steven.price@arm.com Signed-off-by: Steven Price Cc: Catalin Marinas Cc: Will Deacon Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Zong Li Signed-off-by: Andrew Morton --- arch/arm64/mm/dump.c | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) --- a/arch/arm64/mm/dump.c~arm64-mm-display-non-present-entries-in-ptdump +++ a/arch/arm64/mm/dump.c @@ -269,21 +269,22 @@ static void note_page(struct ptdump_stat if (st->current_prot) { note_prot_uxn(st, addr); note_prot_wx(st, addr); - pt_dump_seq_printf(st->seq, "0x%016lx-0x%016lx ", + } + + pt_dump_seq_printf(st->seq, "0x%016lx-0x%016lx ", st->start_address, addr); - delta = (addr - st->start_address) >> 10; - while (!(delta & 1023) && unit[1]) { - delta >>= 10; - unit++; - } - pt_dump_seq_printf(st->seq, "%9lu%c %s", delta, *unit, - pg_level[st->level].name); - if (pg_level[st->level].bits) - dump_prot(st, pg_level[st->level].bits, - pg_level[st->level].num); - pt_dump_seq_puts(st->seq, "\n"); + delta = (addr - st->start_address) >> 10; + while (!(delta & 1023) && unit[1]) { + delta >>= 10; + unit++; } + pt_dump_seq_printf(st->seq, "%9lu%c %s", delta, *unit, + pg_level[st->level].name); + if (st->current_prot && pg_level[st->level].bits) + dump_prot(st, pg_level[st->level].bits, + pg_level[st->level].num); + pt_dump_seq_puts(st->seq, "\n"); if (addr >= st->marker[1].start_address) { st->marker++; From patchwork Sun Dec 1 01:53:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268313 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 450A3112B for ; Sun, 1 Dec 2019 01:53:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 055242084E for ; Sun, 1 Dec 2019 01:53:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="YvbmA8Gr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 055242084E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C88516B02DF; Sat, 30 Nov 2019 20:53:27 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C39216B02E1; Sat, 30 Nov 2019 20:53:27 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B76766B02E2; Sat, 30 Nov 2019 20:53:27 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0212.hostedemail.com [216.40.44.212]) by kanga.kvack.org (Postfix) with ESMTP id A18356B02DF for ; Sat, 30 Nov 2019 20:53:27 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 4FE882C7C for ; Sun, 1 Dec 2019 01:53:27 +0000 (UTC) X-FDA: 76214900454.30.flag37_1e1e79d838108 X-Spam-Summary: 2,0,0,e1054388c6af805a,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alex@ghiti.fr:aou@eecs.berkeley.edu:ard.biesheuvel@linaro.org:arnd@arndb.de:aryabinin@virtuozzo.com:benh@kernel.crashing.org:borntraeger@de.ibm.com:bp@alien8.de:catalin.marinas@arm.com:dave.hansen@linux.intel.com:dave.jiang@intel.com:davem@davemloft.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:hpa@zytor.com:james.morse@arm.com:jhogan@kernel.org:kan.liang@linux.intel.com::linux@armlinux.org.uk:luto@kernel.org:mark.rutland@arm.com:mawilcox@microsoft.com:mingo@elte.hu:mm-commits@vger.kernel.org:mpe@ellerman.id.au:n-horiguchi@ah.jp.nec.com:palmer@sifive.com:paul.burton@mips.com:paul.walmsley@sifive.com:paulus@samba.org:peterz@infradead.org:ralf@linux-mips.org:shashim@codeaurora.org:steven.price@arm.com:tglx@linutronix.de:torvalds@linux-foundation.org:vgupta@synopsys.com:will@kernel.org:zong.li@sifive.com,RULES_HIT:2:41:355:379:560:80 0:960:96 X-HE-Tag: flag37_1e1e79d838108 X-Filterd-Recvd-Size: 9778 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:53:26 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C9B4A208E4; Sun, 1 Dec 2019 01:53:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165206; bh=2FS5Bz9FGWzM5m5fT1S+DIq8eZsRmLfBtg9rhvsKqwk=; h=Date:From:To:Subject:From; b=YvbmA8GrGVwcqpxIXRgy92BdDmgpG7nXbF6BBTF+YAzZjt9JF2XMGbaT/Iu1nnnTu eq6f7onVx/oxUZVkJtELbXLqFBD5uXy7ttWYCPe990YhNEA78Eu1XongyKNcahbdaM b5xTjedK5qgNCb0PlXlIT0MVJBO8WfRqBGiIF1OI= Date: Sat, 30 Nov 2019 17:53:24 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alex@ghiti.fr, aou@eecs.berkeley.edu, ard.biesheuvel@linaro.org, arnd@arndb.de, aryabinin@virtuozzo.com, benh@kernel.crashing.org, borntraeger@de.ibm.com, bp@alien8.de, catalin.marinas@arm.com, dave.hansen@linux.intel.com, dave.jiang@intel.com, davem@davemloft.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, hpa@zytor.com, james.morse@arm.com, jhogan@kernel.org, kan.liang@linux.intel.com, linux-mm@kvack.org, linux@armlinux.org.uk, luto@kernel.org, mark.rutland@arm.com, mawilcox@microsoft.com, mingo@elte.hu, mm-commits@vger.kernel.org, mpe@ellerman.id.au, n-horiguchi@ah.jp.nec.com, palmer@sifive.com, paul.burton@mips.com, paul.walmsley@sifive.com, paulus@samba.org, peterz@infradead.org, ralf@linux-mips.org, shashim@codeaurora.org, steven.price@arm.com, tglx@linutronix.de, torvalds@linux-foundation.org, vgupta@synopsys.com, will@kernel.org, zong.li@sifive.com Subject: [patch 068/158] mm: ptdump: reduce level numbers by 1 in note_page() Message-ID: <20191201015324.eXsftVqiI%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Steven Price Subject: mm: ptdump: reduce level numbers by 1 in note_page() Rather than having to increment the 'depth' number by 1 in ptdump_hole(), let's change the meaning of 'level' in note_page() since that makes the code simplier. Note that for x86, the level numbers were previously increased by 1 in commit 45dcd2091363 ("x86/mm/dump_pagetables: Fix printout of p4d level") and the comment "Bit 7 has a different meaning" was not updated, so this change also makes the code match the comment again. Link: http://lkml.kernel.org/r/20191101140942.51554-24-steven.price@arm.com Signed-off-by: Steven Price Reviewed-by: Catalin Marinas Cc: Will Deacon Cc: Albert Ou Cc: Alexander Potapenko Cc: Alexandre Ghiti Cc: Andrey Ryabinin Cc: Andy Lutomirski Cc: Ard Biesheuvel Cc: Arnd Bergmann Cc: Benjamin Herrenschmidt Cc: Borislav Petkov Cc: Christian Borntraeger Cc: Dave Hansen Cc: Dave Jiang Cc: David S. Miller Cc: Dmitry Vyukov Cc: Heiko Carstens Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: James Hogan Cc: James Morse Cc: "Liang, Kan" Cc: Mark Rutland Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naoya Horiguchi Cc: Palmer Dabbelt Cc: Paul Burton Cc: Paul Mackerras Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Ralf Baechle Cc: Russell King Cc: Shiraz Hashim Cc: Thomas Gleixner Cc: Vasily Gorbik Cc: Vineet Gupta Cc: Zong Li Signed-off-by: Andrew Morton --- arch/arm64/mm/dump.c | 6 +++--- arch/x86/mm/dump_pagetables.c | 19 ++++++++++--------- include/linux/ptdump.h | 1 + mm/ptdump.c | 16 ++++++++-------- 4 files changed, 22 insertions(+), 20 deletions(-) --- a/arch/arm64/mm/dump.c~mm-ptdump-reduce-level-numbers-by-1-in-note_page +++ a/arch/arm64/mm/dump.c @@ -175,8 +175,7 @@ struct pg_level { }; static struct pg_level pg_level[] = { - { - }, { /* pgd */ + { /* pgd */ .name = "PGD", .bits = pte_bits, .num = ARRAY_SIZE(pte_bits), @@ -256,7 +255,7 @@ static void note_page(struct ptdump_stat if (level >= 0) prot = val & pg_level[level].mask; - if (!st->level) { + if (st->level == -1) { st->level = level; st->current_prot = prot; st->start_address = addr; @@ -350,6 +349,7 @@ void ptdump_check_wx(void) { 0, NULL}, { -1, NULL}, }, + .level = -1, .check_wx = true, .ptdump = { .note_page = note_page, --- a/arch/x86/mm/dump_pagetables.c~mm-ptdump-reduce-level-numbers-by-1-in-note_page +++ a/arch/x86/mm/dump_pagetables.c @@ -176,7 +176,7 @@ static struct addr_marker address_marker static void printk_prot(struct seq_file *m, pgprotval_t pr, int level, bool dmsg) { static const char * const level_name[] = - { "cr3", "pgd", "p4d", "pud", "pmd", "pte" }; + { "pgd", "p4d", "pud", "pmd", "pte" }; if (!(pr & _PAGE_PRESENT)) { /* Not present */ @@ -200,12 +200,12 @@ static void printk_prot(struct seq_file pt_dump_cont_printf(m, dmsg, " "); /* Bit 7 has a different meaning on level 3 vs 4 */ - if (level <= 4 && pr & _PAGE_PSE) + if (level <= 3 && pr & _PAGE_PSE) pt_dump_cont_printf(m, dmsg, "PSE "); else pt_dump_cont_printf(m, dmsg, " "); - if ((level == 5 && pr & _PAGE_PAT) || - ((level == 4 || level == 3) && pr & _PAGE_PAT_LARGE)) + if ((level == 4 && pr & _PAGE_PAT) || + ((level == 3 || level == 2) && pr & _PAGE_PAT_LARGE)) pt_dump_cont_printf(m, dmsg, "PAT "); else pt_dump_cont_printf(m, dmsg, " "); @@ -267,15 +267,15 @@ static void note_page(struct ptdump_stat new_prot = val & PTE_FLAGS_MASK; - if (level > 1) { - new_eff = effective_prot(st->prot_levels[level - 2], + if (level > 0) { + new_eff = effective_prot(st->prot_levels[level - 1], new_prot); } else { new_eff = new_prot; } - if (level > 0) - st->prot_levels[level - 1] = new_eff; + if (level >= 0) + st->prot_levels[level] = new_eff; /* * If we have a "break" in the series, we need to flush the state that @@ -285,7 +285,7 @@ static void note_page(struct ptdump_stat cur = st->current_prot; eff = st->effective_prot; - if (!st->level) { + if (st->level == -1) { /* First entry */ st->current_prot = new_prot; st->effective_prot = new_eff; @@ -376,6 +376,7 @@ static void ptdump_walk_pgd_level_core(s .note_page = note_page, .range = ptdump_ranges }, + .level = -1, .to_dmesg = dmesg, .check_wx = checkwx, .seq = m --- a/include/linux/ptdump.h~mm-ptdump-reduce-level-numbers-by-1-in-note_page +++ a/include/linux/ptdump.h @@ -11,6 +11,7 @@ struct ptdump_range { }; struct ptdump_state { + /* level is 0:PGD to 4:PTE, or -1 if unknown */ void (*note_page)(struct ptdump_state *st, unsigned long addr, int level, unsigned long val); const struct ptdump_range *range; --- a/mm/ptdump.c~mm-ptdump-reduce-level-numbers-by-1-in-note_page +++ a/mm/ptdump.c @@ -11,7 +11,7 @@ static int ptdump_pgd_entry(pgd_t *pgd, pgd_t val = READ_ONCE(*pgd); if (pgd_leaf(val)) - st->note_page(st, addr, 1, pgd_val(val)); + st->note_page(st, addr, 0, pgd_val(val)); return 0; } @@ -23,7 +23,7 @@ static int ptdump_p4d_entry(p4d_t *p4d, p4d_t val = READ_ONCE(*p4d); if (p4d_leaf(val)) - st->note_page(st, addr, 2, p4d_val(val)); + st->note_page(st, addr, 1, p4d_val(val)); return 0; } @@ -35,7 +35,7 @@ static int ptdump_pud_entry(pud_t *pud, pud_t val = READ_ONCE(*pud); if (pud_leaf(val)) - st->note_page(st, addr, 3, pud_val(val)); + st->note_page(st, addr, 2, pud_val(val)); return 0; } @@ -47,7 +47,7 @@ static int ptdump_pmd_entry(pmd_t *pmd, pmd_t val = READ_ONCE(*pmd); if (pmd_leaf(val)) - st->note_page(st, addr, 4, pmd_val(val)); + st->note_page(st, addr, 3, pmd_val(val)); return 0; } @@ -57,7 +57,7 @@ static int ptdump_pte_entry(pte_t *pte, { struct ptdump_state *st = walk->private; - st->note_page(st, addr, 5, pte_val(READ_ONCE(*pte))); + st->note_page(st, addr, 4, pte_val(READ_ONCE(*pte))); return 0; } @@ -75,7 +75,7 @@ static inline int note_kasan_page_table( { struct ptdump_state *st = walk->private; - st->note_page(st, addr, 5, pte_val(kasan_early_shadow_pte[0])); + st->note_page(st, addr, 4, pte_val(kasan_early_shadow_pte[0])); return 1; } @@ -115,7 +115,7 @@ static int ptdump_hole(unsigned long add { struct ptdump_state *st = walk->private; - st->note_page(st, addr, depth + 1, 0); + st->note_page(st, addr, depth, 0); return 0; } @@ -147,5 +147,5 @@ void ptdump_walk_pgd(struct ptdump_state up_read(&mm->mmap_sem); /* Flush out the last page */ - st->note_page(st, 0, 0, 0); + st->note_page(st, 0, -1, 0); } From patchwork Sun Dec 1 01:53:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268315 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8960F921 for ; Sun, 1 Dec 2019 01:53:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4D2AA2084E for ; Sun, 1 Dec 2019 01:53:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="GEIv0jan" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4D2AA2084E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EAB0A6B02E1; Sat, 30 Nov 2019 20:53:30 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E5C0C6B02E3; Sat, 30 Nov 2019 20:53:30 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D9B7B6B02E4; Sat, 30 Nov 2019 20:53:30 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0053.hostedemail.com [216.40.44.53]) by kanga.kvack.org (Postfix) with ESMTP id C413A6B02E1 for ; Sat, 30 Nov 2019 20:53:30 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 762E440F1 for ; Sun, 1 Dec 2019 01:53:30 +0000 (UTC) X-FDA: 76214900580.08.cat50_1e9751665071f X-Spam-Summary: 2,0,0,09c2f6f1fb19c59a,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hughd@google.com:joel@joelfernandes.org::mm-commits@vger.kernel.org:ngeoffray@google.com:shuah@kernel.org:stable@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1543:1711:1730:1747:1777:1792:1801:2393:2525:2559:2563:2682:2685:2693:2859:2897:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3865:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4605:5007:6261:6653:7576:7903:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12295:12296:12297:12517:12519:12555:12679:12783:12986:13149:13161:13229:13230:13255:13846:14181:14721:14849:21080:21222:21451:21627:21939:30012:30054:30075,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2, LUA_SUMM X-HE-Tag: cat50_1e9751665071f X-Filterd-Recvd-Size: 4498 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:53:30 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 311D6215A5; Sun, 1 Dec 2019 01:53:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165209; bh=FPXma5KIlv3ycKGzXzXGQlKa9YuYJaaq8lBfD5AmEG4=; h=Date:From:To:Subject:From; b=GEIv0janIduchxVrBP9GSBJvrPzosKs7JhBH9KnSd38/ReYVR0SpJ9nreaRXVwqhK f6scz8KwTHCvAM3dDxJ/JAWAuvLlnDA1xSD9yNcO7+StJ5ePCIPllyiKQnJShmG4ci n9vPn3cIJAWN++Caj6PYP9aV4bfmjejqYpiY0RgE= Date: Sat, 30 Nov 2019 17:53:28 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hughd@google.com, joel@joelfernandes.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, ngeoffray@google.com, shuah@kernel.org, stable@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 069/158] mm, memfd: fix COW issue on MAP_PRIVATE and F_SEAL_FUTURE_WRITE mappings Message-ID: <20191201015328.wunf9qxIi%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Nicolas Geoffray Subject: mm, memfd: fix COW issue on MAP_PRIVATE and F_SEAL_FUTURE_WRITE mappings F_SEAL_FUTURE_WRITE has unexpected behavior when used with MAP_PRIVATE: A private mapping created after the memfd file that gets sealed with F_SEAL_FUTURE_WRITE loses the copy-on-write at fork behavior, meaning children and parent share the same memory, even though the mapping is private. The reason for this is due to the code below: static int shmem_mmap(struct file *file, struct vm_area_struct *vma) { struct shmem_inode_info *info = SHMEM_I(file_inode(file)); if (info->seals & F_SEAL_FUTURE_WRITE) { /* * New PROT_WRITE and MAP_SHARED mmaps are not allowed when * "future write" seal active. */ if ((vma->vm_flags & VM_SHARED) && (vma->vm_flags & VM_WRITE)) return -EPERM; /* * Since the F_SEAL_FUTURE_WRITE seals allow for a MAP_SHARED * read-only mapping, take care to not allow mprotect to revert * protections. */ vma->vm_flags &= ~(VM_MAYWRITE); } ... } And for the mm to know if a mapping is copy-on-write: static inline bool is_cow_mapping(vm_flags_t flags) { return (flags & (VM_SHARED | VM_MAYWRITE)) == VM_MAYWRITE; } The patch fixes the issue by making the mprotect revert protection happen only for shared mappings. For private mappings, using mprotect will have no effect on the seal behavior. The F_SEAL_FUTURE_WRITE feature was introduced in v5.1 so v5.3.x stable kernels would need a backport. [akpm@linux-foundation.org: reflow comment, per Christoph] Link: http://lkml.kernel.org/r/20191107195355.80608-1-joel@joelfernandes.org Fixes: ab3948f58ff84 ("mm/memfd: add an F_SEAL_FUTURE_WRITE seal to memfd") Signed-off-by: Nicolas Geoffray Signed-off-by: Joel Fernandes (Google) Cc: Hugh Dickins Cc: Shuah Khan Cc: Signed-off-by: Andrew Morton --- mm/shmem.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) --- a/mm/shmem.c~memfd-fix-cow-issue-on-map_private-and-f_seal_future_write-mappings +++ a/mm/shmem.c @@ -2214,11 +2214,14 @@ static int shmem_mmap(struct file *file, return -EPERM; /* - * Since the F_SEAL_FUTURE_WRITE seals allow for a MAP_SHARED - * read-only mapping, take care to not allow mprotect to revert - * protections. + * Since an F_SEAL_FUTURE_WRITE sealed memfd can be mapped as + * MAP_SHARED and read-only, take care to not allow mprotect to + * revert protections on such mappings. Do this only for shared + * mappings. For private mappings, don't need to mask + * VM_MAYWRITE as we still want them to be COW-writable. */ - vma->vm_flags &= ~(VM_MAYWRITE); + if (vma->vm_flags & VM_SHARED) + vma->vm_flags &= ~(VM_MAYWRITE); } file_accessed(file); From patchwork Sun Dec 1 01:53:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268317 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DB930921 for ; Sun, 1 Dec 2019 01:53:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A8CA32084E for ; Sun, 1 Dec 2019 01:53:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="Y9YWcmAp" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A8CA32084E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 499F26B02E3; Sat, 30 Nov 2019 20:53:34 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 448D06B02E5; Sat, 30 Nov 2019 20:53:34 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 361CC6B02E6; Sat, 30 Nov 2019 20:53:34 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 1B57A6B02E3 for ; Sat, 30 Nov 2019 20:53:34 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id CB8E640DD for ; Sun, 1 Dec 2019 01:53:33 +0000 (UTC) X-FDA: 76214900706.13.meat35_1f13e2097644e X-Spam-Summary: 2,0,0,16ac82092ec0ca7a,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hughd@google.com:joel@joelfernandes.org::mm-commits@vger.kernel.org:ngeoffray@google.com:shuah@kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6261:6653:7576:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13069:13161:13229:13255:13311:13357:13846:14181:14384:14721:14849:21080:21324:21451:21627:21939:30054:30056:30079,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: meat35_1f13e2097644e X-Filterd-Recvd-Size: 3259 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:53:33 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 86C26215A5; Sun, 1 Dec 2019 01:53:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165212; bh=TR+yiD40D3MCjLPheH38cBiYAg3XeUkqG7RUIjTXBPE=; h=Date:From:To:Subject:From; b=Y9YWcmApWXBCgVAkuqsPjMunARhBzSXvHtYffvbmOJzyaRBWL1OOMRLZutOSEBHHe 5pVbqjirakXKDMg1udq47qVi/7FFVT0Cu37dXofTV68TF2v0rtucEfq4FMglerTdxQ at+sPZ/UY9ZBXbQhy7e3P8BCDBHY/E7cmIB9RyUY= Date: Sat, 30 Nov 2019 17:53:31 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hughd@google.com, joel@joelfernandes.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, ngeoffray@google.com, shuah@kernel.org, torvalds@linux-foundation.org Subject: [patch 070/158] memfd: add test for COW on MAP_PRIVATE and F_SEAL_FUTURE_WRITE mappings Message-ID: <20191201015331.lZ47gCN7K%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Joel Fernandes (Google)" Subject: memfd: add test for COW on MAP_PRIVATE and F_SEAL_FUTURE_WRITE mappings In this test, the parent and child both have writable private mappings. The test shows that without the patch in this series, the parent and child shared the same memory which is incorrect. In other words, COW needs to be triggered so any writes to child's copy stays local to the child. Link: http://lkml.kernel.org/r/20191107195355.80608-2-joel@joelfernandes.org Signed-off-by: Joel Fernandes (Google) Cc: Hugh Dickins Cc: Nicolas Geoffray Cc: Shuah Khan Signed-off-by: Andrew Morton --- tools/testing/selftests/memfd/memfd_test.c | 36 +++++++++++++++++++ 1 file changed, 36 insertions(+) --- a/tools/testing/selftests/memfd/memfd_test.c~memfd-add-test-for-cow-on-map_private-and-f_seal_future_write-mappings +++ a/tools/testing/selftests/memfd/memfd_test.c @@ -290,6 +290,40 @@ static void mfd_assert_read_shared(int f munmap(p, mfd_def_size); } +static void mfd_assert_fork_private_write(int fd) +{ + int *p; + pid_t pid; + + p = mmap(NULL, + mfd_def_size, + PROT_READ | PROT_WRITE, + MAP_PRIVATE, + fd, + 0); + if (p == MAP_FAILED) { + printf("mmap() failed: %m\n"); + abort(); + } + + p[0] = 22; + + pid = fork(); + if (pid == 0) { + p[0] = 33; + exit(0); + } else { + waitpid(pid, NULL, 0); + + if (p[0] != 22) { + printf("MAP_PRIVATE copy-on-write failed: %m\n"); + abort(); + } + } + + munmap(p, mfd_def_size); +} + static void mfd_assert_write(int fd) { ssize_t l; @@ -760,6 +794,8 @@ static void test_seal_future_write(void) mfd_assert_read_shared(fd2); mfd_fail_write(fd2); + mfd_assert_fork_private_write(fd); + munmap(p, mfd_def_size); close(fd2); close(fd); From patchwork Sun Dec 1 01:53:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268319 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9D42817E0 for ; Sun, 1 Dec 2019 01:53:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6133A2245C for ; Sun, 1 Dec 2019 01:53:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="uZn1Kkmq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6133A2245C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1D9776B02E5; Sat, 30 Nov 2019 20:53:38 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 189E96B02E7; Sat, 30 Nov 2019 20:53:38 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0A04C6B02E8; Sat, 30 Nov 2019 20:53:38 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0228.hostedemail.com [216.40.44.228]) by kanga.kvack.org (Postfix) with ESMTP id E80696B02E5 for ; Sat, 30 Nov 2019 20:53:37 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 3168B181AEF0B for ; Sun, 1 Dec 2019 01:53:37 +0000 (UTC) X-FDA: 76214900874.08.drain60_1f8c4de73eb2a X-Spam-Summary: 2,0,0,3a2f53e4f8c45e5c,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:dan.j.williams@intel.com:jane.chu@oracle.com::mhocko@kernel.org:mm-commits@vger.kernel.org:n-horiguchi@ah.jp.nec.com:torvalds@linux-foundation.org,RULES_HIT:41:69:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1535:1544:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2689:2859:2898:2899:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4605:5007:6119:6261:6653:7576:7901:7903:8957:9025:9391:9545:9592:10004:10913:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13153:13228:13846:14181:14721:14849:21060:21080:21433:21451:21627:21819:21939:30012:30054:30064:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:n eutral,C X-HE-Tag: drain60_1f8c4de73eb2a X-Filterd-Recvd-Size: 5519 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:53:36 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B827E215A5; Sun, 1 Dec 2019 01:53:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165216; bh=92umU60NzFfSA81sojuUWtdYtACego7Uesl4wID/dnE=; h=Date:From:To:Subject:From; b=uZn1KkmqFppOTV4r9i1Tx81vuwrVtr+4kLRqTUCk8D9Bj+hDAlYv1Rh5IywM//+GK 6dB46Q84XOuc1ur/Z0rlXrcubs/VJzT5kvF+keLYxwhla9OnjUhEOyW53wz251YuVh EUc6y9BRDr3MGwwUMQrHNY4LvSdz47dn1effDbrg= Date: Sat, 30 Nov 2019 17:53:35 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, dan.j.williams@intel.com, jane.chu@oracle.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, n-horiguchi@ah.jp.nec.com, torvalds@linux-foundation.org Subject: [patch 071/158] mm/memory-failure.c clean up around tk pre-allocation Message-ID: <20191201015335.4OS4YEWMH%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jane Chu Subject: mm/memory-failure.c clean up around tk pre-allocation add_to_kill() expects the first 'tk' to be pre-allocated, it makes subsequent allocations on need basis, this makes the code a bit difficult to read. Move all the allocation internal to add_to_kill() and drop the **tk argument. Link: http://lkml.kernel.org/r/1565112345-28754-2-git-send-email-jane.chu@oracle.com Signed-off-by: Jane Chu Reviewed-by: Dan Williams Acked-by: Naoya Horiguchi Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/memory-failure.c | 40 +++++++++++++--------------------------- 1 file changed, 13 insertions(+), 27 deletions(-) --- a/mm/memory-failure.c~mm-memory-failurec-clean-up-around-tk-pre-allocation +++ a/mm/memory-failure.c @@ -303,25 +303,19 @@ static unsigned long dev_pagemap_mapping /* * Schedule a process for later kill. * Uses GFP_ATOMIC allocations to avoid potential recursions in the VM. - * TBD would GFP_NOIO be enough? */ static void add_to_kill(struct task_struct *tsk, struct page *p, struct vm_area_struct *vma, - struct list_head *to_kill, - struct to_kill **tkc) + struct list_head *to_kill) { struct to_kill *tk; - if (*tkc) { - tk = *tkc; - *tkc = NULL; - } else { - tk = kmalloc(sizeof(struct to_kill), GFP_ATOMIC); - if (!tk) { - pr_err("Memory failure: Out of memory while machine check handling\n"); - return; - } + tk = kmalloc(sizeof(struct to_kill), GFP_ATOMIC); + if (!tk) { + pr_err("Memory failure: Out of memory while machine check handling\n"); + return; } + tk->addr = page_address_in_vma(p, vma); if (is_zone_device_page(p)) tk->size_shift = dev_pagemap_mapping_shift(p, vma); @@ -345,6 +339,7 @@ static void add_to_kill(struct task_stru kfree(tk); return; } + get_task_struct(tsk); tk->tsk = tsk; list_add_tail(&tk->nd, to_kill); @@ -436,7 +431,7 @@ static struct task_struct *task_early_ki * Collect processes when the error hit an anonymous page. */ static void collect_procs_anon(struct page *page, struct list_head *to_kill, - struct to_kill **tkc, int force_early) + int force_early) { struct vm_area_struct *vma; struct task_struct *tsk; @@ -461,7 +456,7 @@ static void collect_procs_anon(struct pa if (!page_mapped_in_vma(page, vma)) continue; if (vma->vm_mm == t->mm) - add_to_kill(t, page, vma, to_kill, tkc); + add_to_kill(t, page, vma, to_kill); } } read_unlock(&tasklist_lock); @@ -472,7 +467,7 @@ static void collect_procs_anon(struct pa * Collect processes when the error hit a file mapped page. */ static void collect_procs_file(struct page *page, struct list_head *to_kill, - struct to_kill **tkc, int force_early) + int force_early) { struct vm_area_struct *vma; struct task_struct *tsk; @@ -496,7 +491,7 @@ static void collect_procs_file(struct pa * to be informed of all such data corruptions. */ if (vma->vm_mm == t->mm) - add_to_kill(t, page, vma, to_kill, tkc); + add_to_kill(t, page, vma, to_kill); } } read_unlock(&tasklist_lock); @@ -505,26 +500,17 @@ static void collect_procs_file(struct pa /* * Collect the processes who have the corrupted page mapped to kill. - * This is done in two steps for locking reasons. - * First preallocate one tokill structure outside the spin locks, - * so that we can kill at least one process reasonably reliable. */ static void collect_procs(struct page *page, struct list_head *tokill, int force_early) { - struct to_kill *tk; - if (!page->mapping) return; - tk = kmalloc(sizeof(struct to_kill), GFP_NOIO); - if (!tk) - return; if (PageAnon(page)) - collect_procs_anon(page, tokill, &tk, force_early); + collect_procs_anon(page, tokill, force_early); else - collect_procs_file(page, tokill, &tk, force_early); - kfree(tk); + collect_procs_file(page, tokill, force_early); } static const char *action_name[] = { From patchwork Sun Dec 1 01:53:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268321 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 673AA112B for ; Sun, 1 Dec 2019 01:53:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 34D522084E for ; Sun, 1 Dec 2019 01:53:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="VzU/3jIj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 34D522084E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BC9C06B02E7; Sat, 30 Nov 2019 20:53:40 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B754A6B02E9; Sat, 30 Nov 2019 20:53:40 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A8B1B6B02EA; Sat, 30 Nov 2019 20:53:40 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0187.hostedemail.com [216.40.44.187]) by kanga.kvack.org (Postfix) with ESMTP id 88FB76B02E7 for ; Sat, 30 Nov 2019 20:53:40 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 4DBDF824999B for ; Sun, 1 Dec 2019 01:53:40 +0000 (UTC) X-FDA: 76214901000.17.root96_20026f2797430 X-Spam-Summary: 2,0,0,b9711d374f619884,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:david@redhat.com::mhocko@kernel.org:mm-commits@vger.kernel.org:n-horiguchi@ah.jp.nec.com:nao.horiguchi@gmail.com:osalvador@suse.de:torvalds@linux-foundation.org,RULES_HIT:41:69:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1535:1543:1711:1730:1747:1777:1792:2194:2196:2199:2200:2393:2525:2559:2563:2682:2685:2741:2859:2898:2899:2901:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3865:3866:3867:3868:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4470:4605:5007:6119:6261:6653:7514:7576:7875:7903:8599:9025:9036:9545:10004:10913:11026:11473:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12696:12737:12783:12986:14181:14721:14849:21060:21080:21433:21451:21627:21740:21939:30003:30054:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL: neutral, X-HE-Tag: root96_20026f2797430 X-Filterd-Recvd-Size: 5090 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:53:39 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D6DDE2084E; Sun, 1 Dec 2019 01:53:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165219; bh=fzemeXhdVOovyLpiO2Ypi6rjTdBpgJRXT0ytmrktCFc=; h=Date:From:To:Subject:From; b=VzU/3jIj9uWx63LndHAtkEV/Bx5P5LAjBTWXSWPwYi4w7yXQIClASgqc6PosTPuPi bSWfSS2qCuMm+c/lhFixrvPn93dc7UV2r5ZmbeUJ4mFJCSHPl3Ofre4e9vff8uCruT p+s6jhFD03aG6wI+aYrXcIe1KhOvqUxhyMr0etfg= Date: Sat, 30 Nov 2019 17:53:38 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, david@redhat.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, n-horiguchi@ah.jp.nec.com, nao.horiguchi@gmail.com, osalvador@suse.de, torvalds@linux-foundation.org Subject: [patch 072/158] mm, soft-offline: convert parameter to pfn Message-ID: <20191201015338.lzlT2O1FE%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Subject: mm, soft-offline: convert parameter to pfn Currently soft_offline_page() receives struct page, and its sibling memory_failure() receives pfn. This discrepancy looks weird and makes precheck on pfn validity tricky. So let's align them. Link: http://lkml.kernel.org/r/20191016234706.GA5493@www9186uo.sakura.ne.jp Signed-off-by: Naoya Horiguchi Acked-by: Andrew Morton Cc: David Hildenbrand Cc: Michal Hocko Cc: Oscar Salvador Signed-off-by: Andrew Morton --- drivers/base/memory.c | 7 +------ include/linux/mm.h | 2 +- mm/madvise.c | 2 +- mm/memory-failure.c | 19 +++++++++---------- 4 files changed, 12 insertions(+), 18 deletions(-) --- a/drivers/base/memory.c~mm-soft-offline-convert-parameter-to-pfn +++ a/drivers/base/memory.c @@ -538,12 +538,7 @@ static ssize_t soft_offline_page_store(s if (kstrtoull(buf, 0, &pfn) < 0) return -EINVAL; pfn >>= PAGE_SHIFT; - if (!pfn_valid(pfn)) - return -ENXIO; - /* Only online pages can be soft-offlined (esp., not ZONE_DEVICE). */ - if (!pfn_to_online_page(pfn)) - return -EIO; - ret = soft_offline_page(pfn_to_page(pfn), 0); + ret = soft_offline_page(pfn, 0); return ret == 0 ? count : ret; } --- a/include/linux/mm.h~mm-soft-offline-convert-parameter-to-pfn +++ a/include/linux/mm.h @@ -2773,7 +2773,7 @@ extern int sysctl_memory_failure_early_k extern int sysctl_memory_failure_recovery; extern void shake_page(struct page *p, int access); extern atomic_long_t num_poisoned_pages __read_mostly; -extern int soft_offline_page(struct page *page, int flags); +extern int soft_offline_page(unsigned long pfn, int flags); /* --- a/mm/madvise.c~mm-soft-offline-convert-parameter-to-pfn +++ a/mm/madvise.c @@ -895,7 +895,7 @@ static int madvise_inject_error(int beha pr_info("Soft offlining pfn %#lx at process virtual address %#lx\n", pfn, start); - ret = soft_offline_page(page, MF_COUNT_INCREASED); + ret = soft_offline_page(pfn, MF_COUNT_INCREASED); if (ret) return ret; continue; --- a/mm/memory-failure.c~mm-soft-offline-convert-parameter-to-pfn +++ a/mm/memory-failure.c @@ -1476,7 +1476,7 @@ static void memory_failure_work_func(str if (!gotten) break; if (entry.flags & MF_SOFT_OFFLINE) - soft_offline_page(pfn_to_page(entry.pfn), entry.flags); + soft_offline_page(entry.pfn, entry.flags); else memory_failure(entry.pfn, entry.flags); } @@ -1857,7 +1857,7 @@ static int soft_offline_free_page(struct /** * soft_offline_page - Soft offline a page. - * @page: page to offline + * @pfn: pfn to soft-offline * @flags: flags. Same as memory_failure(). * * Returns 0 on success, otherwise negated errno. @@ -1877,18 +1877,17 @@ static int soft_offline_free_page(struct * This is not a 100% solution for all memory, but tries to be * ``good enough'' for the majority of memory. */ -int soft_offline_page(struct page *page, int flags) +int soft_offline_page(unsigned long pfn, int flags) { int ret; - unsigned long pfn = page_to_pfn(page); + struct page *page; - if (is_zone_device_page(page)) { - pr_debug_ratelimited("soft_offline: %#lx page is device page\n", - pfn); - if (flags & MF_COUNT_INCREASED) - put_page(page); + if (!pfn_valid(pfn)) + return -ENXIO; + /* Only online pages can be soft-offlined (esp., not ZONE_DEVICE). */ + page = pfn_to_online_page(pfn); + if (!page) return -EIO; - } if (PageHWPoison(page)) { pr_info("soft offline: %#lx page already poisoned\n", pfn); From patchwork Sun Dec 1 01:53:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268323 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5F337921 for ; Sun, 1 Dec 2019 01:53:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2BB832084E for ; Sun, 1 Dec 2019 01:53:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="JULIYyro" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2BB832084E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CAE146B02E9; Sat, 30 Nov 2019 20:53:43 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C63426B02EB; Sat, 30 Nov 2019 20:53:43 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B4DB96B02EC; Sat, 30 Nov 2019 20:53:43 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0066.hostedemail.com [216.40.44.66]) by kanga.kvack.org (Postfix) with ESMTP id 97C776B02E9 for ; Sat, 30 Nov 2019 20:53:43 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 4C3EE180AD817 for ; Sun, 1 Dec 2019 01:53:43 +0000 (UTC) X-FDA: 76214901126.15.teeth80_20761bccb5a25 X-Spam-Summary: 2,0,0,c14771bcc99b3cf7,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:david@redhat.com::mm-commits@vger.kernel.org:n-horiguchi@ah.jp.nec.com:torvalds@linux-foundation.org:yeyunfeng@huawei.com,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1714:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2859:2899:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3350:3865:3870:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6261:6653:7576:9025:9545:10004:11026:11257:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:13069:13311:13357:13846:14181:14384:14721:14849:21060:21080:21451:21627:21939:30054:30083,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: teeth80_20761bccb5a25 X-Filterd-Recvd-Size: 2379 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:53:42 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1BF8B2084D; Sun, 1 Dec 2019 01:53:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165222; bh=WCSq6nU1RXQkhKDbnm3wW98OMUY4M9iXPJ62k06Ku8M=; h=Date:From:To:Subject:From; b=JULIYyroUa+BfPDAipaeI0Kggh3aKG8nhIb35ElYqdKPtykt2RFhb4+qRwhzy/VMm PPkJ2R1oi+pOVyptSQrJQ3mOrL2gmaAFlxt8qJQp0x3/Az8gIVowuvjxAAxTinQsUd vzgYuR2Q1vR18oAUhBDL0g/YX3lRTOZkmjC3dKgc= Date: Sat, 30 Nov 2019 17:53:41 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, david@redhat.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, n-horiguchi@ah.jp.nec.com, torvalds@linux-foundation.org, yeyunfeng@huawei.com Subject: [patch 073/158] mm/memory-failure.c: use page_shift() in add_to_kill() Message-ID: <20191201015341.uq80Vd14O%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yunfeng Ye Subject: mm/memory-failure.c: use page_shift() in add_to_kill() page_shift() is supported after the commit 94ad9338109f ("mm: introduce page_shift()"). So replace with page_shift() in add_to_kill() for readability. Link: http://lkml.kernel.org/r/543d8bc9-f2e7-3023-7c35-2e7ed67c0e82@huawei.com Signed-off-by: Yunfeng Ye Reviewed-by: David Hildenbrand Acked-by: Naoya Horiguchi Signed-off-by: Andrew Morton --- mm/memory-failure.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/memory-failure.c~mm-memory-failurec-replace-with-page_shift-in-add_to_kill +++ a/mm/memory-failure.c @@ -320,7 +320,7 @@ static void add_to_kill(struct task_stru if (is_zone_device_page(p)) tk->size_shift = dev_pagemap_mapping_shift(p, vma); else - tk->size_shift = compound_order(compound_head(p)) + PAGE_SHIFT; + tk->size_shift = page_shift(compound_head(p)); /* * Send SIGKILL if "tk->addr == -EFAULT". Also, as From patchwork Sun Dec 1 01:53:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268325 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CCFFB112B for ; Sun, 1 Dec 2019 01:53:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 977BE20880 for ; Sun, 1 Dec 2019 01:53:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="j3Yt8O/R" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 977BE20880 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5D0F86B02EB; Sat, 30 Nov 2019 20:53:47 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 581896B02ED; Sat, 30 Nov 2019 20:53:47 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4BEA36B02EE; Sat, 30 Nov 2019 20:53:47 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0228.hostedemail.com [216.40.44.228]) by kanga.kvack.org (Postfix) with ESMTP id 344456B02EB for ; Sat, 30 Nov 2019 20:53:47 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id ED550824999B for ; Sun, 1 Dec 2019 01:53:46 +0000 (UTC) X-FDA: 76214901252.30.cough30_20facbad0d029 X-Spam-Summary: 2,0,0,bf706a4a322a7032,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:anshuman.khandual@arm.com:dan.j.williams@intel.com:david@redhat.com::mhocko@suse.com:mm-commits@vger.kernel.org:osalvador@suse.de:pasha.tatashin@soleen.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:560:800:960:965:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1543:1711:1730:1747:1777:1792:1963:2196:2198:2199:2200:2393:2525:2559:2564:2682:2685:2693:2731:2859:2899:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3865:3866:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4390:4395:4470:5007:6261:6653:7576:8603:9025:9389:9545:10004:10128:10913:11026:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12783:12986:13255:13846:13972:14181:14721:14849:21080:21212:21433:21451:21627:21740:21819:21939:30054:30064,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache: 0,MSF:no X-HE-Tag: cough30_20facbad0d029 X-Filterd-Recvd-Size: 4817 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:53:46 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 6FB4A2084E; Sun, 1 Dec 2019 01:53:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165226; bh=dSdGAOyfZukaqerH4ZucoYOgrsA7zvJqL0j1sMrN1k0=; h=Date:From:To:Subject:From; b=j3Yt8O/R1CqLrYI0ygFnDpC1k5586x3pkTxMHklmXzcC3dPRipXelDcWFEKsVBosH IxMywnUxLNLhpESLCkzc15quddoVa48drCBXfnvPQJJtb4sjBQ2SsDfXr1assojfJL 078cNwiZOu9UA8bQLQAD5YcDVu04ejqhq71NhaEs= Date: Sat, 30 Nov 2019 17:53:44 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, anshuman.khandual@arm.com, dan.j.williams@intel.com, david@redhat.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, osalvador@suse.de, pasha.tatashin@soleen.com, torvalds@linux-foundation.org Subject: [patch 074/158] mm/hotplug: reorder memblock_[free|remove]() calls in try_remove_memory() Message-ID: <20191201015344.HHVU5QQZn%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Anshuman Khandual Subject: mm/hotplug: reorder memblock_[free|remove]() calls in try_remove_memory() Currently during memory hot add procedure, memory gets into memblock before calling arch_add_memory() which creates its linear mapping. add_memory_resource() { .................. memblock_add_node() .................. arch_add_memory() .................. } But during memory hot remove procedure, removal from memblock happens first before its linear mapping gets teared down with arch_remove_memory() which is not consistent. Resource removal should happen in reverse order as they were added. However this does not pose any problem for now, unless there is an assumption regarding linear mapping. One example was a subtle failure on arm64 platform [1]. Though this has now found a different solution. try_remove_memory() { .................. memblock_free() memblock_remove() .................. arch_remove_memory() .................. } This changes the sequence of resource removal including memblock and linear mapping tear down during memory hot remove which will now be the reverse order in which they were added during memory hot add. The changed removal order looks like the following. try_remove_memory() { .................. arch_remove_memory() .................. memblock_free() memblock_remove() .................. } [1] https://patchwork.kernel.org/patch/11127623/ Memory hot remove now works on arm64 without this because a recent commit 60bb462fc7ad ("drivers/base/node.c: simplify unregister_memory_block_under_nodes()"). This does not fix a serious problem. It just removes an inconsistency while freeing resources during memory hot remove which for now does not pose a real problem. David mentioned that re-ordering should still make sense for consistency purpose (removing stuff in the reverse order they were added). This patch is now detached from arm64 hot-remove series. Michal: : I would just a note that the inconsistency doesn't pose any problem now : but if somebody makes any assumptions about linear mappings then it could : get subtly broken like your example for arm64 which has found a different : solution in the meantime. Link: http://lkml.kernel.org/r/1569380273-7708-1-git-send-email-anshuman.khandual@arm.com Signed-off-by: Anshuman Khandual Acked-by: Michal Hocko Reviewed-by: David Hildenbrand Cc: Oscar Salvador Cc: Pavel Tatashin Cc: Dan Williams Signed-off-by: Andrew Morton --- mm/memory_hotplug.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/mm/memory_hotplug.c~mm-hotplug-reorder-memblock_-calls-in-try_remove_memory +++ a/mm/memory_hotplug.c @@ -1750,13 +1750,13 @@ static int __ref try_remove_memory(int n /* remove memmap entry */ firmware_map_remove(start, start + size, "System RAM"); - memblock_free(start, size); - memblock_remove(start, size); /* remove memory block devices before removing memory */ remove_memory_block_devices(start, size); arch_remove_memory(nid, start, size, NULL); + memblock_free(start, size); + memblock_remove(start, size); __release_memory_resource(start, size); try_offline_node(nid); From patchwork Sun Dec 1 01:53:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268327 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2F176112B for ; Sun, 1 Dec 2019 01:53:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E70292084D for ; Sun, 1 Dec 2019 01:53:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="YJ48AZQY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E70292084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B28EA6B02ED; Sat, 30 Nov 2019 20:53:50 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AD9A96B02EF; Sat, 30 Nov 2019 20:53:50 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C8816B02F0; Sat, 30 Nov 2019 20:53:50 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0188.hostedemail.com [216.40.44.188]) by kanga.kvack.org (Postfix) with ESMTP id 854DF6B02ED for ; Sat, 30 Nov 2019 20:53:50 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 34B7B2A87 for ; Sun, 1 Dec 2019 01:53:50 +0000 (UTC) X-FDA: 76214901420.17.boy21_2178007d0a432 X-Spam-Summary: 2,0,0,479f7036e0244441,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alastair@d-silva.org:dan.j.williams@intel.com:david@redhat.com::mhocko@suse.com:mm-commits@vger.kernel.org:osalvador@suse.de:pasha.tatashin@soleen.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:1981:2194:2198:2199:2200:2393:2525:2553:2559:2564:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4031:4321:5007:6119:6261:6653:7576:8599:9025:9036:9545:10004:10913:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12682:12783:12986:13146:13149:13230:14181:14721:14849:21080:21451:21627:21740:21939:30003:30054:30064:30069:30070:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:n ot bulk, X-HE-Tag: boy21_2178007d0a432 X-Filterd-Recvd-Size: 3994 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:53:49 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D895E20880; Sun, 1 Dec 2019 01:53:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165229; bh=vIVhnu/2QjD24HNck87Y8PGybwHkefCgntuNBnLC4pI=; h=Date:From:To:Subject:From; b=YJ48AZQYhf9+hMBQzyt0MJNdaVLlxUzquhwba2jg/3Mofjlkb+oGbRmRsSl6RpW7b enbPW8CjRRnBYaikD0sNcYOsRroHjvT+p/xB1DoOWTiQESXMq/vU9ZSGkPdUvSQXaf iTpv+jfDgkqjLEgpiNF2go8EcHLaVZuoksZ0Tao4= Date: Sat, 30 Nov 2019 17:53:48 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alastair@d-silva.org, dan.j.williams@intel.com, david@redhat.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, osalvador@suse.de, pasha.tatashin@soleen.com, torvalds@linux-foundation.org Subject: [patch 075/158] mm/memory_hotplug.c: add a bounds check to __add_pages() Message-ID: <20191201015348.Twz4gBPZs%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alastair D'Silva Subject: mm/memory_hotplug.c: add a bounds check to __add_pages() On PowerPC, the address ranges allocated to OpenCAPI LPC memory are allocated from firmware. These address ranges may be higher than what older kernels permit, as we increased the maximum permissable address in commit 4ffe713b7587 ("powerpc/mm: Increase the max addressable memory to 2PB"). It is possible that the addressable range may change again in the future. In this scenario, we end up with a bogus section returned from __section_nr (see the discussion on the thread "mm: Trigger bug on if a section is not found in __section_nr"). Adding a check here means that we fail early and have an opportunity to handle the error gracefully, rather than rumbling on and potentially accessing an incorrect section. Further discussion is also on the thread ("powerpc: Perform a bounds check in arch_add_memory") http://lkml.kernel.org/r/20190827052047.31547-1-alastair@au1.ibm.com Link: http://lkml.kernel.org/r/20191001004617.7536-2-alastair@au1.ibm.com Signed-off-by: Alastair D'Silva Reviewed-by: David Hildenbrand Acked-by: Michal Hocko Cc: Oscar Salvador Cc: Pavel Tatashin Cc: Dan Williams Signed-off-by: Andrew Morton --- mm/memory_hotplug.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) --- a/mm/memory_hotplug.c~memory_hotplug-add-a-bounds-check-to-__add_pages +++ a/mm/memory_hotplug.c @@ -278,6 +278,22 @@ static int check_pfn_span(unsigned long return 0; } +static int check_hotplug_memory_addressable(unsigned long pfn, + unsigned long nr_pages) +{ + const u64 max_addr = PFN_PHYS(pfn + nr_pages) - 1; + + if (max_addr >> MAX_PHYSMEM_BITS) { + const u64 max_allowed = (1ull << (MAX_PHYSMEM_BITS + 1)) - 1; + WARN(1, + "Hotplugged memory exceeds maximum addressable address, range=%#llx-%#llx, maximum=%#llx\n", + (u64)PFN_PHYS(pfn), max_addr, max_allowed); + return -E2BIG; + } + + return 0; +} + /* * Reasonably generic function for adding memory. It is * expected that archs that support memory hotplug will @@ -291,6 +307,10 @@ int __ref __add_pages(int nid, unsigned unsigned long nr, start_sec, end_sec; struct vmem_altmap *altmap = restrictions->altmap; + err = check_hotplug_memory_addressable(pfn, nr_pages); + if (err) + return err; + if (altmap) { /* * Validate altmap is within bounds of the total request From patchwork Sun Dec 1 01:53:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268329 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 82E3A921 for ; Sun, 1 Dec 2019 01:53:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4FEFC2084D for ; Sun, 1 Dec 2019 01:53:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="NTV2F+09" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4FEFC2084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2509F6B02EF; Sat, 30 Nov 2019 20:53:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 230876B02F1; Sat, 30 Nov 2019 20:53:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 163996B02F2; Sat, 30 Nov 2019 20:53:54 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0124.hostedemail.com [216.40.44.124]) by kanga.kvack.org (Postfix) with ESMTP id 008246B02EF for ; Sat, 30 Nov 2019 20:53:53 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id AADC4181AEF0B for ; Sun, 1 Dec 2019 01:53:53 +0000 (UTC) X-FDA: 76214901546.08.death40_21f815bd3d73c X-Spam-Summary: 2,0,0,7b511a7cf4155468,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:cai@lca.pw:dan.j.williams@intel.com:david@redhat.com:haiyangz@microsoft.com:kys@microsoft.com::mhocko@suse.com:mm-commits@vger.kernel.org:osalvador@suse.com:pasha.tatashin@soleen.com:richard.weiyang@gmail.com:sashal@kernel.org:sthemmin@microsoft.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:1981:2194:2196:2198:2199:2200:2201:2393:2525:2559:2563:2682:2685:2731:2741:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4605:5007:6261:6653:6737:7514:7576:7688:7903:8599:9025:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12114:12297:12438:12517:12519:12555:12679:12783:12986:13846:14096:14181:14721:14849:21080:21251:21451:21611:21627:21939:30054:30064:30 069,0,RB X-HE-Tag: death40_21f815bd3d73c X-Filterd-Recvd-Size: 4224 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:53:53 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2BA7B208E4; Sun, 1 Dec 2019 01:53:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165232; bh=DELTp0hAw/1OSTKXwxXO9joXbTpcMCh6EjTSDgDOu2c=; h=Date:From:To:Subject:From; b=NTV2F+09xGpLweWQkzynjeNzTsop/NdGrEiq4Xhd71dczeSO5tXOyq+OrNuR7rYlR jIIHe2NYAYGCn+ok4iEgCz8pNSx4EZodYXt/dL8UWSgxSTZ93XKINRCaEKeWEYxrfZ RvlblEckaCuI3SFU+/1TYGtJme627Z7cd74ti6DE= Date: Sat, 30 Nov 2019 17:53:51 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, cai@lca.pw, dan.j.williams@intel.com, david@redhat.com, haiyangz@microsoft.com, kys@microsoft.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, osalvador@suse.com, pasha.tatashin@soleen.com, richard.weiyang@gmail.com, sashal@kernel.org, sthemmin@microsoft.com, torvalds@linux-foundation.org Subject: [patch 076/158] mm/memory_hotplug: export generic_online_page() Message-ID: <20191201015351.qnEi-XxC0%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: mm/memory_hotplug: export generic_online_page() Patch series "mm/memory_hotplug: Export generic_online_page()". Let's replace the __online_page...() functions by generic_online_page(). Hyper-V only wants to delay the actual onlining of un-backed pages, so we can simpy re-use the generic function. This patch (of 3): Let's expose generic_online_page() so online_page_callback users can simply fall back to the generic implementation when actually deciding to online the pages. Link: http://lkml.kernel.org/r/20190909114830.662-2-david@redhat.com Signed-off-by: David Hildenbrand Acked-by: Michal Hocko Cc: Oscar Salvador Cc: Pavel Tatashin Cc: Dan Williams Cc: Wei Yang Cc: Qian Cai Cc: Haiyang Zhang Cc: "K. Y. Srinivasan" Cc: Sasha Levin Cc: Stephen Hemminger Signed-off-by: Andrew Morton --- include/linux/memory_hotplug.h | 1 + mm/memory_hotplug.c | 5 ++--- 2 files changed, 3 insertions(+), 3 deletions(-) --- a/include/linux/memory_hotplug.h~mm-memory_hotplug-export-generic_online_page +++ a/include/linux/memory_hotplug.h @@ -102,6 +102,7 @@ extern unsigned long __offline_isolated_ typedef void (*online_page_callback_t)(struct page *page, unsigned int order); +extern void generic_online_page(struct page *page, unsigned int order); extern int set_online_page_callback(online_page_callback_t callback); extern int restore_online_page_callback(online_page_callback_t callback); --- a/mm/memory_hotplug.c~mm-memory_hotplug-export-generic_online_page +++ a/mm/memory_hotplug.c @@ -49,8 +49,6 @@ * and restore_online_page_callback() for generic callback restore. */ -static void generic_online_page(struct page *page, unsigned int order); - static online_page_callback_t online_page_callback = generic_online_page; static DEFINE_MUTEX(online_page_callback_lock); @@ -617,7 +615,7 @@ void __online_page_free(struct page *pag } EXPORT_SYMBOL_GPL(__online_page_free); -static void generic_online_page(struct page *page, unsigned int order) +void generic_online_page(struct page *page, unsigned int order) { kernel_map_pages(page, 1 << order, 1); __free_pages_core(page, order); @@ -627,6 +625,7 @@ static void generic_online_page(struct p totalhigh_pages_add(1UL << order); #endif } +EXPORT_SYMBOL_GPL(generic_online_page); static int online_pages_range(unsigned long start_pfn, unsigned long nr_pages, void *arg) From patchwork Sun Dec 1 01:53:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268331 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 06DB9112B for ; Sun, 1 Dec 2019 01:53:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BE242208C3 for ; Sun, 1 Dec 2019 01:53:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="eeBSCU0N" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BE242208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9F1226B02F1; Sat, 30 Nov 2019 20:53:57 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9C6D46B02F3; Sat, 30 Nov 2019 20:53:57 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DC6C6B02F4; Sat, 30 Nov 2019 20:53:57 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0231.hostedemail.com [216.40.44.231]) by kanga.kvack.org (Postfix) with ESMTP id 7693F6B02F1 for ; Sat, 30 Nov 2019 20:53:57 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 2A438824999B for ; Sun, 1 Dec 2019 01:53:57 +0000 (UTC) X-FDA: 76214901714.17.beef69_227656ece543f X-Spam-Summary: 2,0,0,bd5d8e1a29f475c0,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:cai@lca.pw:dan.j.williams@intel.com:david@redhat.com:haiyangz@microsoft.com:kys@microsoft.com::mhocko@suse.com:mm-commits@vger.kernel.org:osalvador@suse.com:pasha.tatashin@soleen.com:richard.weiyang@gmail.com:sashal@kernel.org:sthemmin@microsoft.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:1981:2194:2196:2199:2200:2393:2525:2559:2563:2682:2685:2693:2741:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:5007:6261:6653:6737:7514:7576:7688:7875:8599:9025:9545:10004:10913:11026:11658:11914:12043:12048:12114:12297:12438:12517:12519:12555:12679:12783:12986:13069:13311:13357:13846:14181:14384:14721:14849:21080:21251:21451:21627:21939:30054:30064,0,RBL:error,CacheIP:n one,Baye X-HE-Tag: beef69_227656ece543f X-Filterd-Recvd-Size: 2760 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:53:56 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9633D2084D; Sun, 1 Dec 2019 01:53:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165236; bh=uzkjNw7nlXKxuYMEBOjO5vWqtzK10nvWinwiOo3AsJw=; h=Date:From:To:Subject:From; b=eeBSCU0NanfGUGGuKuqBcY+o284BV/PbNOVKFYIzwszR0attMvN8Gf7Sk2gE2jq/W al0f4uTeRogFCJgsBaHBZvrOCtqiOeWZV06iT06NPtKDZhdG4+fQZk5AcOtU+vbduf PI9G2v2I/Vcacdt1728f1p+fKKofHz7RXhulPg24= Date: Sat, 30 Nov 2019 17:53:55 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, cai@lca.pw, dan.j.williams@intel.com, david@redhat.com, haiyangz@microsoft.com, kys@microsoft.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, osalvador@suse.com, pasha.tatashin@soleen.com, richard.weiyang@gmail.com, sashal@kernel.org, sthemmin@microsoft.com, torvalds@linux-foundation.org Subject: [patch 077/158] hv_balloon: use generic_online_page() Message-ID: <20191201015355.jHjwCMvVx%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: hv_balloon: use generic_online_page() Let's use the generic onlining function - which will now also take care of calling kernel_map_pages(). Link: http://lkml.kernel.org/r/20190909114830.662-3-david@redhat.com Signed-off-by: David Hildenbrand Acked-by: Michal Hocko Cc: "K. Y. Srinivasan" Cc: Haiyang Zhang Cc: Stephen Hemminger Cc: Sasha Levin Cc: Dan Williams Cc: Oscar Salvador Cc: Pavel Tatashin Cc: Qian Cai Cc: Wei Yang Signed-off-by: Andrew Morton --- drivers/hv/hv_balloon.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) --- a/drivers/hv/hv_balloon.c~hv_balloon-use-generic_online_page +++ a/drivers/hv/hv_balloon.c @@ -681,8 +681,7 @@ static void hv_page_online_one(struct hv /* This frame is currently backed; online the page. */ __online_page_set_limits(pg); - __online_page_increment_counters(pg); - __online_page_free(pg); + generic_online_page(pg, 0); lockdep_assert_held(&dm_device.ha_lock); dm_device.num_pages_onlined++; From patchwork Sun Dec 1 01:54:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268333 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F2DBA112B for ; Sun, 1 Dec 2019 01:54:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C00F6215A5 for ; Sun, 1 Dec 2019 01:54:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="sJ1yQe+A" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C00F6215A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A8D5B6B02F3; Sat, 30 Nov 2019 20:54:02 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A62D16B02F5; Sat, 30 Nov 2019 20:54:02 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A0D56B02F6; Sat, 30 Nov 2019 20:54:02 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0116.hostedemail.com [216.40.44.116]) by kanga.kvack.org (Postfix) with ESMTP id 826A56B02F3 for ; Sat, 30 Nov 2019 20:54:02 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 27346180AD817 for ; Sun, 1 Dec 2019 01:54:02 +0000 (UTC) X-FDA: 76214901924.13.sand39_2332a8cd54544 X-Spam-Summary: 2,0,0,50b1c735fb23b2e1,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:cai@lca.pw:dan.j.williams@intel.com:david@redhat.com:haiyangz@microsoft.com:kys@microsoft.com::mhocko@suse.com:mm-commits@vger.kernel.org:osalvador@suse.com:pasha.tatashin@soleen.com:richard.weiyang@gmail.com:sashal@kernel.org:sthemmin@microsoft.com:torvalds@linux-foundation.org,RULES_HIT:41:69:355:379:800:960:965:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2525:2559:2563:2682:2685:2731:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3868:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4390:4395:4605:5007:6261:6653:6737:7514:7576:7688:8599:9025:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12114:12297:12438:12517:12519:12555:12679:12783:12986:13255:13846:14181:14721:14849:21080:21251:21451:21627:21939:30054:30064,0,RBL:error, CacheIP: X-HE-Tag: sand39_2332a8cd54544 X-Filterd-Recvd-Size: 3581 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:01 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9589E208C3; Sun, 1 Dec 2019 01:54:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165241; bh=pA/6Vh8W0ylJQiDaMFkvtwsH+6mKZ9c1ZMfoY6uBIg4=; h=Date:From:To:Subject:From; b=sJ1yQe+AOXIW212Q5gSsPl3Ni39kW8XdMbs883e2hg7LZvaPAPGjwA3dXj+0vYBUu JyIO7JGWe0wFKTSPAsSp5/nEiHCBlkFQ4InZowJkkkjhzeni1g16sQOOpkXFaojbpM MtmNNY/dvfKg0GKhgkdnypIq99ZRwYlqBFAcLjDU= Date: Sat, 30 Nov 2019 17:54:00 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, cai@lca.pw, dan.j.williams@intel.com, david@redhat.com, haiyangz@microsoft.com, kys@microsoft.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, osalvador@suse.com, pasha.tatashin@soleen.com, richard.weiyang@gmail.com, sashal@kernel.org, sthemmin@microsoft.com, torvalds@linux-foundation.org Subject: [patch 078/158] mm/memory_hotplug: remove __online_page_free() and __online_page_increment_counters() Message-ID: <20191201015400.uPMnfcLpt%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: mm/memory_hotplug: remove __online_page_free() and __online_page_increment_counters() Let's drop the now unused functions. Link: http://lkml.kernel.org/r/20190909114830.662-4-david@redhat.com Signed-off-by: David Hildenbrand Acked-by: Michal Hocko Cc: Oscar Salvador Cc: Pavel Tatashin Cc: Wei Yang Cc: Dan Williams Cc: Qian Cai Cc: Haiyang Zhang Cc: "K. Y. Srinivasan" Cc: Sasha Levin Cc: Stephen Hemminger Signed-off-by: Andrew Morton --- include/linux/memory_hotplug.h | 2 -- mm/memory_hotplug.c | 12 ------------ 2 files changed, 14 deletions(-) --- a/include/linux/memory_hotplug.h~mm-memory_hotplug-remove-__online_page_free-and-__online_page_increment_counters +++ a/include/linux/memory_hotplug.h @@ -107,8 +107,6 @@ extern int set_online_page_callback(onli extern int restore_online_page_callback(online_page_callback_t callback); extern void __online_page_set_limits(struct page *page); -extern void __online_page_increment_counters(struct page *page); -extern void __online_page_free(struct page *page); extern int try_online_node(int nid); --- a/mm/memory_hotplug.c~mm-memory_hotplug-remove-__online_page_free-and-__online_page_increment_counters +++ a/mm/memory_hotplug.c @@ -603,18 +603,6 @@ void __online_page_set_limits(struct pag } EXPORT_SYMBOL_GPL(__online_page_set_limits); -void __online_page_increment_counters(struct page *page) -{ - adjust_managed_page_count(page, 1); -} -EXPORT_SYMBOL_GPL(__online_page_increment_counters); - -void __online_page_free(struct page *page) -{ - __free_reserved_page(page); -} -EXPORT_SYMBOL_GPL(__online_page_free); - void generic_online_page(struct page *page, unsigned int order) { kernel_map_pages(page, 1 << order, 1); From patchwork Sun Dec 1 01:54:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268335 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A5A42921 for ; Sun, 1 Dec 2019 01:54:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7339A206E1 for ; Sun, 1 Dec 2019 01:54:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="VdEUw5bg" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7339A206E1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5D0DE6B02F5; Sat, 30 Nov 2019 20:54:06 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 581E36B02F7; Sat, 30 Nov 2019 20:54:06 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4986E6B02F8; Sat, 30 Nov 2019 20:54:06 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0088.hostedemail.com [216.40.44.88]) by kanga.kvack.org (Postfix) with ESMTP id 30B276B02F5 for ; Sat, 30 Nov 2019 20:54:06 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id CE3C72A87 for ; Sun, 1 Dec 2019 01:54:05 +0000 (UTC) X-FDA: 76214902050.04.yard73_23b83eb6f190b X-Spam-Summary: 2,0,0,4a3ab7d74be5bbcf,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alexander.h.duyck@linux.intel.com:anshuman.khandual@arm.com:cai@lca.pw:dan.j.williams@intel.com:david@redhat.com:kernelfans@gmail.com::mgorman@techsingularity.net:mhocko@suse.com:mm-commits@vger.kernel.org:osalvador@suse.de:pasha.tatashin@soleen.com:pavel.tatashin@microsoft.com:richard.weiyang@gmail.com:rppt@linux.ibm.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:vbabka@suse.cz,RULES_HIT:41:355:379:800:960:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1535:1543:1711:1730:1747:1777:1792:1963:2194:2196:2198:2199:2200:2201:2393:2525:2559:2563:2682:2685:2693:2731:2859:2898:2901:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:5007:6119:6261:6653:6737:7514:7576:7688:8599:8660:8957:9025:9545:10004:10913:11026:11232:11658:11914 :12043:1 X-HE-Tag: yard73_23b83eb6f190b X-Filterd-Recvd-Size: 5160 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:05 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2838A215A5; Sun, 1 Dec 2019 01:54:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165244; bh=nV2mhqNm5IkpJ30bu72iv80jMk4XXMP0vH2LExHC5oM=; h=Date:From:To:Subject:From; b=VdEUw5bg2IV1rxVGy0Fg8IeTh44JvI7CK95mN4z8aXXEkg+199VvlT7qaPf1Ojhgt T4fqJQGAeTn2+JLVajZgym0tPBNbrtkw7z0s86HDGt5xxNr9I5xPqRpc2SNhS4r0Y2 0tYVdkeD7LCrdBUkQ02KSYNj9K29NdJDKqMBhfZQ= Date: Sat, 30 Nov 2019 17:54:03 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alexander.h.duyck@linux.intel.com, anshuman.khandual@arm.com, cai@lca.pw, dan.j.williams@intel.com, david@redhat.com, kernelfans@gmail.com, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@suse.com, mm-commits@vger.kernel.org, osalvador@suse.de, pasha.tatashin@soleen.com, pavel.tatashin@microsoft.com, richard.weiyang@gmail.com, rppt@linux.ibm.com, rppt@linux.vnet.ibm.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 079/158] mm/page_alloc.c: don't set pages PageReserved() when offlining Message-ID: <20191201015403.828G0Qt_w%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: mm/page_alloc.c: don't set pages PageReserved() when offlining Patch series "mm: Memory offlining + page isolation cleanups", v2. This patch (of 2): We call __offline_isolated_pages() from __offline_pages() after all pages were isolated and are either free (PageBuddy()) or PageHWPoison. Nothing can stop us from offlining memory at this point. In __offline_isolated_pages() we first set all affected memory sections offline (offline_mem_sections(pfn, end_pfn)), to mark the memmap as invalid (pfn_to_online_page() will no longer succeed), and then walk over all pages to pull the free pages from the free lists (to the isolated free lists, to be precise). Note that re-onlining a memory block will result in the whole memmap getting reinitialized, overwriting any old state. We already poision the memmap when offlining is complete to find any access to stale/uninitialized memmaps. So, setting the pages PageReserved() is not helpful. The memap is marked offline and all pageblocks are isolated. As soon as offline, the memmap is stale either way. This looks like a leftover from ancient times where we initialized the memmap when adding memory and not when onlining it (the pages were set PageReserved so re-onling would work as expected). Link: http://lkml.kernel.org/r/20191021172353.3056-2-david@redhat.com Signed-off-by: David Hildenbrand Acked-by: Michal Hocko Cc: Vlastimil Babka Cc: Oscar Salvador Cc: Mel Gorman Cc: Mike Rapoport Cc: Dan Williams Cc: Wei Yang Cc: Alexander Duyck Cc: Anshuman Khandual Cc: Pavel Tatashin Cc: Mike Rapoport Cc: Pavel Tatashin Cc: Pingfan Liu Cc: Qian Cai Signed-off-by: Andrew Morton --- mm/memory_hotplug.c | 4 +--- mm/page_alloc.c | 5 +---- 2 files changed, 2 insertions(+), 7 deletions(-) --- a/mm/memory_hotplug.c~mm-page_allocc-dont-set-pages-pagereserved-when-offlining +++ a/mm/memory_hotplug.c @@ -1384,9 +1384,7 @@ do_migrate_range(unsigned long start_pfn return ret; } -/* - * remove from free_area[] and mark all as Reserved. - */ +/* Mark all sections offline and remove all free pages from the buddy. */ static int offline_isolated_pages_cb(unsigned long start, unsigned long nr_pages, void *data) --- a/mm/page_alloc.c~mm-page_allocc-dont-set-pages-pagereserved-when-offlining +++ a/mm/page_alloc.c @@ -8560,7 +8560,7 @@ __offline_isolated_pages(unsigned long s { struct page *page; struct zone *zone; - unsigned int order, i; + unsigned int order; unsigned long pfn; unsigned long flags; unsigned long offlined_pages = 0; @@ -8588,7 +8588,6 @@ __offline_isolated_pages(unsigned long s */ if (unlikely(!PageBuddy(page) && PageHWPoison(page))) { pfn++; - SetPageReserved(page); offlined_pages++; continue; } @@ -8602,8 +8601,6 @@ __offline_isolated_pages(unsigned long s pfn, 1 << order, end_pfn); #endif del_page_from_free_area(page, &zone->free_area[order]); - for (i = 0; i < (1 << order); i++) - SetPageReserved((page+i)); pfn += (1 << order); } spin_unlock_irqrestore(&zone->lock, flags); From patchwork Sun Dec 1 01:54:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268337 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 98240921 for ; Sun, 1 Dec 2019 01:54:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 57BCF215E5 for ; Sun, 1 Dec 2019 01:54:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="sQWtT7Gu" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 57BCF215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 275E36B02F7; Sat, 30 Nov 2019 20:54:10 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 226626B02F9; Sat, 30 Nov 2019 20:54:10 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 114706B02FA; Sat, 30 Nov 2019 20:54:10 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0236.hostedemail.com [216.40.44.236]) by kanga.kvack.org (Postfix) with ESMTP id EFA656B02F7 for ; Sat, 30 Nov 2019 20:54:09 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id A9F6A2C8A for ; Sun, 1 Dec 2019 01:54:09 +0000 (UTC) X-FDA: 76214902218.01.wax56_24466d88abc45 X-Spam-Summary: 2,0,0,2ddb96505f5f5496,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:alexander.h.duyck@linux.intel.com:anshuman.khandual@arm.com:cai@lca.pw:dan.j.williams@intel.com:david@redhat.com:kernelfans@gmail.com::mgorman@techsingularity.net:mhocko@suse.com:mm-commits@vger.kernel.org:osalvador@suse.de:pasha.tatashin@soleen.com:pavel.tatashin@microsoft.com:richard.weiyang@gmail.com:rppt@linux.ibm.com:rppt@linux.vnet.ibm.com:torvalds@linux-foundation.org:vbabka@suse.cz,RULES_HIT:2:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1535:1730:1747:1777:1792:1963:2196:2198:2199:2200:2393:2525:2553:2559:2563:2682:2685:2859:2898:2899:2902:2903:2915:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4049:4119:4321:4385:4605:5007:6117:6119:6261:6653:6737:7514:7576:7688:7875:8599:8784:8957:9025:9121:9545:10004:10913:11026:1123 3:11473: X-HE-Tag: wax56_24466d88abc45 X-Filterd-Recvd-Size: 8155 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:09 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id ECBE3206E1; Sun, 1 Dec 2019 01:54:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165248; bh=DcxjY0kYqJ+q+htorjgyojrX640TKiospGUeJNAv6Jc=; h=Date:From:To:Subject:From; b=sQWtT7Gu+d4TxhaVGPtA7BZ4gS7bghHwz0Q75oxpJWJ5r6IEYiDP4IPWB/762do1I LuOwepHzu97DBsAuA7GH/rQI1ZuHYKL3dghdCLEOU9HyxR/h58gXrIl1YMj3l7Rypb s8GQbg5a75/ji44EoS4Am8J4ktVAISOm/jGBOMtY= Date: Sat, 30 Nov 2019 17:54:07 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, alexander.h.duyck@linux.intel.com, anshuman.khandual@arm.com, cai@lca.pw, dan.j.williams@intel.com, david@redhat.com, kernelfans@gmail.com, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@suse.com, mm-commits@vger.kernel.org, osalvador@suse.de, pasha.tatashin@soleen.com, pavel.tatashin@microsoft.com, richard.weiyang@gmail.com, rppt@linux.ibm.com, rppt@linux.vnet.ibm.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 080/158] mm/page_isolation.c: convert SKIP_HWPOISON to MEMORY_OFFLINE Message-ID: <20191201015407.BoZ_1bSSb%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: mm/page_isolation.c: convert SKIP_HWPOISON to MEMORY_OFFLINE We have two types of users of page isolation: 1. Memory offlining: Offline memory so it can be unplugged. Memory won't be touched. 2. Memory allocation: Allocate memory (e.g., alloc_contig_range()) to become the owner of the memory and make use of it. For example, in case we want to offline memory, we can ignore (skip over) PageHWPoison() pages, as the memory won't get used. We can allow to offline memory. In contrast, we don't want to allow to allocate such memory. Let's generalize the approach so we can special case other types of pages we want to skip over in case we offline memory. While at it, also pass the same flags to test_pages_isolated(). Link: http://lkml.kernel.org/r/20191021172353.3056-3-david@redhat.com Signed-off-by: David Hildenbrand Suggested-by: Michal Hocko Acked-by: Michal Hocko Cc: Oscar Salvador Cc: Anshuman Khandual Cc: David Hildenbrand Cc: Pingfan Liu Cc: Qian Cai Cc: Pavel Tatashin Cc: Dan Williams Cc: Vlastimil Babka Cc: Mel Gorman Cc: Mike Rapoport Cc: Alexander Duyck Cc: Mike Rapoport Cc: Pavel Tatashin Cc: Wei Yang Signed-off-by: Andrew Morton --- include/linux/page-isolation.h | 4 ++-- mm/memory_hotplug.c | 8 +++++--- mm/page_alloc.c | 4 ++-- mm/page_isolation.c | 12 ++++++------ 4 files changed, 15 insertions(+), 13 deletions(-) --- a/include/linux/page-isolation.h~mm-page_isolationc-convert-skip_hwpoison-to-memory_offline +++ a/include/linux/page-isolation.h @@ -30,7 +30,7 @@ static inline bool is_migrate_isolate(in } #endif -#define SKIP_HWPOISON 0x1 +#define MEMORY_OFFLINE 0x1 #define REPORT_FAILURE 0x2 bool has_unmovable_pages(struct zone *zone, struct page *page, int count, @@ -58,7 +58,7 @@ undo_isolate_page_range(unsigned long st * Test all pages in [start_pfn, end_pfn) are isolated or not. */ int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn, - bool skip_hwpoisoned_pages); + int isol_flags); struct page *alloc_migrate_target(struct page *page, unsigned long private); --- a/mm/memory_hotplug.c~mm-page_isolationc-convert-skip_hwpoison-to-memory_offline +++ a/mm/memory_hotplug.c @@ -1187,7 +1187,8 @@ static bool is_pageblock_removable_noloc if (!zone_spans_pfn(zone, pfn)) return false; - return !has_unmovable_pages(zone, page, 0, MIGRATE_MOVABLE, SKIP_HWPOISON); + return !has_unmovable_pages(zone, page, 0, MIGRATE_MOVABLE, + MEMORY_OFFLINE); } /* Checks if this range of memory is likely to be hot-removable. */ @@ -1402,7 +1403,8 @@ static int check_pages_isolated_cb(unsigned long start_pfn, unsigned long nr_pages, void *data) { - return test_pages_isolated(start_pfn, start_pfn + nr_pages, true); + return test_pages_isolated(start_pfn, start_pfn + nr_pages, + MEMORY_OFFLINE); } static int __init cmdline_parse_movable_node(char *p) @@ -1513,7 +1515,7 @@ static int __ref __offline_pages(unsigne /* set above range as isolated */ ret = start_isolate_page_range(start_pfn, end_pfn, MIGRATE_MOVABLE, - SKIP_HWPOISON | REPORT_FAILURE); + MEMORY_OFFLINE | REPORT_FAILURE); if (ret < 0) { reason = "failure to isolate range"; goto failed_removal; --- a/mm/page_alloc.c~mm-page_isolationc-convert-skip_hwpoison-to-memory_offline +++ a/mm/page_alloc.c @@ -8261,7 +8261,7 @@ bool has_unmovable_pages(struct zone *zo * The HWPoisoned page may be not in buddy system, and * page_count() is not 0. */ - if ((flags & SKIP_HWPOISON) && PageHWPoison(page)) + if ((flags & MEMORY_OFFLINE) && PageHWPoison(page)) continue; if (__PageMovable(page)) @@ -8477,7 +8477,7 @@ int alloc_contig_range(unsigned long sta } /* Make sure the range is really isolated. */ - if (test_pages_isolated(outer_start, end, false)) { + if (test_pages_isolated(outer_start, end, 0)) { pr_info_ratelimited("%s: [%lx, %lx) PFNs busy\n", __func__, outer_start, end); ret = -EBUSY; --- a/mm/page_isolation.c~mm-page_isolationc-convert-skip_hwpoison-to-memory_offline +++ a/mm/page_isolation.c @@ -168,7 +168,8 @@ __first_valid_page(unsigned long pfn, un * @migratetype: Migrate type to set in error recovery. * @flags: The following flags are allowed (they can be combined in * a bit mask) - * SKIP_HWPOISON - ignore hwpoison pages + * MEMORY_OFFLINE - isolate to offline (!allocate) memory + * e.g., skip over PageHWPoison() pages * REPORT_FAILURE - report details about the failure to * isolate the range * @@ -257,7 +258,7 @@ void undo_isolate_page_range(unsigned lo */ static unsigned long __test_page_isolated_in_pageblock(unsigned long pfn, unsigned long end_pfn, - bool skip_hwpoisoned_pages) + int flags) { struct page *page; @@ -274,7 +275,7 @@ __test_page_isolated_in_pageblock(unsign * simple way to verify that as VM_BUG_ON(), though. */ pfn += 1 << page_order(page); - else if (skip_hwpoisoned_pages && PageHWPoison(page)) + else if ((flags & MEMORY_OFFLINE) && PageHWPoison(page)) /* A HWPoisoned page cannot be also PageBuddy */ pfn++; else @@ -286,7 +287,7 @@ __test_page_isolated_in_pageblock(unsign /* Caller should ensure that requested range is in a single zone */ int test_pages_isolated(unsigned long start_pfn, unsigned long end_pfn, - bool skip_hwpoisoned_pages) + int isol_flags) { unsigned long pfn, flags; struct page *page; @@ -308,8 +309,7 @@ int test_pages_isolated(unsigned long st /* Check all pages are free or marked as ISOLATED */ zone = page_zone(page); spin_lock_irqsave(&zone->lock, flags); - pfn = __test_page_isolated_in_pageblock(start_pfn, end_pfn, - skip_hwpoisoned_pages); + pfn = __test_page_isolated_in_pageblock(start_pfn, end_pfn, isol_flags); spin_unlock_irqrestore(&zone->lock, flags); trace_test_pages_isolated(start_pfn, end_pfn, pfn); From patchwork Sun Dec 1 01:54:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268339 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AAE28112B for ; Sun, 1 Dec 2019 01:54:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6E4B0215E5 for ; Sun, 1 Dec 2019 01:54:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="CnToV0st" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6E4B0215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2C3426B02F9; Sat, 30 Nov 2019 20:54:13 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 275D76B02FB; Sat, 30 Nov 2019 20:54:13 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 140516B02FC; Sat, 30 Nov 2019 20:54:13 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0236.hostedemail.com [216.40.44.236]) by kanga.kvack.org (Postfix) with ESMTP id F138F6B02F9 for ; Sat, 30 Nov 2019 20:54:12 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id B6490181AEF0B for ; Sun, 1 Dec 2019 01:54:12 +0000 (UTC) X-FDA: 76214902344.30.wheel91_24bf9d2e29011 X-Spam-Summary: 2,0,0,cf9042314448dbbf,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:ben.dooks@codethink.co.uk:david@redhat.com::mhocko@suse.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2194:2196:2199:2200:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3867:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:5007:6261:6653:7576:8599:8603:9025:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12114:12297:12438:12517:12519:12555:12679:12783:12986:13069:13161:13229:13255:13311:13357:14181:14384:14721:14849:21080:21451:21627:21939:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: wheel91_24bf9d2e29011 X-Filterd-Recvd-Size: 3145 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:12 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 85670222D9; Sun, 1 Dec 2019 01:54:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165251; bh=UYV41vDYqczQVAG5OTU5oyq4obo8Xo6RAHkSfOow6Ps=; h=Date:From:To:Subject:From; b=CnToV0st2t0QLz/sIXRg05hXxyQdto9Chn1zWAdLlH0NeNXDICmJnqzPLIBRV5ZKc R+nbep/MXQglxX96iDQAPRWUf7g1qYCq/J4N+YrZ+WvQTznw+hL5ldz9+cRbKgnr4v TjNVZeDhU/FL3XKZt2Av2qTNXRkJkC0m3P4fmN6s= Date: Sat, 30 Nov 2019 17:54:10 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, ben.dooks@codethink.co.uk, david@redhat.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 081/158] include/linux/memory_hotplug.h: move definitions of {set,clear}_zone_contiguous Message-ID: <20191201015410.n9Q8plnnT%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Ben Dooks (Codethink)" Subject: include/linux/memory_hotplug.h: move definitions of {set,clear}_zone_contiguous The {set,clear}_zone_contiguous are built whatever the configuratoon so move the definitions outside the current ifdef to avoid the following compiler warnings: mm/page_alloc.c:1550:6: warning: no previous prototype for 'set_zone_contiguous' [-Wmissing-prototypes] mm/page_alloc.c:1571:6: warning: no previous prototype for 'clear_zone_contiguous' [-Wmissing-prototypes] Link: http://lkml.kernel.org/r/20191106123911.7435-1-ben.dooks@codethink.co.uk Signed-off-by: Ben Dooks (Codethink) Acked-by: Michal Hocko Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- include/linux/memory_hotplug.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) --- a/include/linux/memory_hotplug.h~mm-memory_hotplug-move-definitions-of-setclear_zone_contiguous +++ a/include/linux/memory_hotplug.h @@ -228,9 +228,6 @@ void put_online_mems(void); void mem_hotplug_begin(void); void mem_hotplug_done(void); -extern void set_zone_contiguous(struct zone *zone); -extern void clear_zone_contiguous(struct zone *zone); - #else /* ! CONFIG_MEMORY_HOTPLUG */ #define pfn_to_online_page(pfn) \ ({ \ @@ -338,6 +335,9 @@ static inline int remove_memory(int nid, static inline void __remove_memory(int nid, u64 start, u64 size) {} #endif /* CONFIG_MEMORY_HOTREMOVE */ +extern void set_zone_contiguous(struct zone *zone); +extern void clear_zone_contiguous(struct zone *zone); + extern void __ref free_area_init_core_hotplug(int nid); extern int __add_memory(int nid, u64 start, u64 size); extern int add_memory(int nid, u64 start, u64 size); From patchwork Sun Dec 1 01:54:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268341 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3ACD1112B for ; Sun, 1 Dec 2019 01:54:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EF89A215E5 for ; Sun, 1 Dec 2019 01:54:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="deXkOS6n" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EF89A215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A3E726B02FB; Sat, 30 Nov 2019 20:54:16 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9F2C26B02FD; Sat, 30 Nov 2019 20:54:16 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9044A6B02FE; Sat, 30 Nov 2019 20:54:16 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0164.hostedemail.com [216.40.44.164]) by kanga.kvack.org (Postfix) with ESMTP id 7AF496B02FB for ; Sat, 30 Nov 2019 20:54:16 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 2A1DC180AD817 for ; Sun, 1 Dec 2019 01:54:16 +0000 (UTC) X-FDA: 76214902512.23.bean16_2539023dcb24e X-Spam-Summary: 2,0,0,5b112e75f3c2321f,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:david@redhat.com:gregkh@linuxfoundation.org::mhocko@kernel.org:mm-commits@vger.kernel.org:osalvador@suse.de:rafael@kernel.org:torvalds@linux-foundation.org,RULES_HIT:2:41:69:355:379:800:960:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1535:1605:1730:1747:1777:1792:2194:2199:2393:2525:2559:2563:2682:2685:2693:2859:2897:2898:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4049:4118:4250:4321:4605:5007:6261:6653:7576:7875:7903:8599:8603:8660:9025:9389:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13148:13161:13200:13229:13230:13846:13972:21080:21324:21433:21451:21627:21740:21939:30054:30069:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk ,SPF:fp, X-HE-Tag: bean16_2539023dcb24e X-Filterd-Recvd-Size: 7449 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:15 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B9992215E5; Sun, 1 Dec 2019 01:54:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165255; bh=elwkYVOk8NfEg8EycBSx9jvMs2CPaiHvyOt5dSoucKc=; h=Date:From:To:Subject:From; b=deXkOS6n1KzOGCd87BzZYm5CtvSmddA+UzkAp4zES/D/tvQbn5jSKMHBcQmPgo/lB chRfi/d8NfJsuCLqkKPFO0uHyWNycfemhYbyFcNN3nQg/sHr3+SJBu0xBsgkD5JPPB Y1SQcyKeOHPuhtZrMW96iCN1sIVo/0hzR/KmCUqI= Date: Sat, 30 Nov 2019 17:54:14 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, david@redhat.com, gregkh@linuxfoundation.org, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, osalvador@suse.de, rafael@kernel.org, torvalds@linux-foundation.org Subject: [patch 082/158] drivers/base/memory.c: drop the mem_sysfs_mutex Message-ID: <20191201015414.1DORDFq_C%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: drivers/base/memory.c: drop the mem_sysfs_mutex The mem_sysfs_mutex isn't really helpful. Also, it's not really clear what the mutex protects at all. The device lists of the memory subsystem are protected separately. We don't need that mutex when looking up. creating, or removing independent devices. find_memory_block_by_id() will perform locking on its own and grab a reference of the returned device. At the time memory_dev_init() is called, we cannot have concurrent hot(un)plug operations yet - we're still fairly early during boot. We don't need any locking. The creation/removal of memory block devices should be protected on a higher level - especially using the device hotplug lock to avoid documented issues (see Documentation/core-api/memory-hotplug.rst) - or if that is reworked, using similar locking. Protecting in the context of these functions only doesn't really make sense. Especially, if we would have a situation where the same memory blocks are created/deleted at the same time, there is something horribly going wrong (imagining adding/removing a DIMM at the same time from two call paths) - after the functions succeeded something else in the callers would blow up (e.g., create_memory_block_devices() succeeded but there are no memory block devices anymore). All relevant call paths (except when adding memory early during boot via ACPI, which is now documented) hold the device hotplug lock when adding memory, and when removing memory. Let's document that instead. Add a simple safety net to create_memory_block_devices() in case we would actually remove memory blocks while adding them, so we'll never dereference a NULL pointer. Simplify memory_dev_init() now that the lock is gone. Link: http://lkml.kernel.org/r/20190925082621.4927-1-david@redhat.com Signed-off-by: David Hildenbrand Reviewed-by: Andrew Morton Cc: Greg Kroah-Hartman Cc: "Rafael J. Wysocki" Cc: Michal Hocko Cc: Oscar Salvador Signed-off-by: Andrew Morton --- drivers/base/memory.c | 33 ++++++++++++++------------------- 1 file changed, 14 insertions(+), 19 deletions(-) --- a/drivers/base/memory.c~drivers-base-memoryc-drop-the-mem_sysfs_mutex +++ a/drivers/base/memory.c @@ -19,15 +19,12 @@ #include #include #include -#include #include #include #include #include -static DEFINE_MUTEX(mem_sysfs_mutex); - #define MEMORY_CLASS_NAME "memory" #define to_memory_block(dev) container_of(dev, struct memory_block, dev) @@ -700,6 +697,8 @@ static void unregister_memory(struct mem * Create memory block devices for the given memory area. Start and size * have to be aligned to memory block granularity. Memory block devices * will be initialized as offline. + * + * Called under device_hotplug_lock. */ int create_memory_block_devices(unsigned long start, unsigned long size) { @@ -713,7 +712,6 @@ int create_memory_block_devices(unsigned !IS_ALIGNED(size, memory_block_size_bytes()))) return -EINVAL; - mutex_lock(&mem_sysfs_mutex); for (block_id = start_block_id; block_id != end_block_id; block_id++) { ret = init_memory_block(&mem, block_id, MEM_OFFLINE); if (ret) @@ -725,11 +723,12 @@ int create_memory_block_devices(unsigned for (block_id = start_block_id; block_id != end_block_id; block_id++) { mem = find_memory_block_by_id(block_id); + if (WARN_ON_ONCE(!mem)) + continue; mem->section_count = 0; unregister_memory(mem); } } - mutex_unlock(&mem_sysfs_mutex); return ret; } @@ -737,6 +736,8 @@ int create_memory_block_devices(unsigned * Remove memory block devices for the given memory area. Start and size * have to be aligned to memory block granularity. Memory block devices * have to be offline. + * + * Called under device_hotplug_lock. */ void remove_memory_block_devices(unsigned long start, unsigned long size) { @@ -749,7 +750,6 @@ void remove_memory_block_devices(unsigne !IS_ALIGNED(size, memory_block_size_bytes()))) return; - mutex_lock(&mem_sysfs_mutex); for (block_id = start_block_id; block_id != end_block_id; block_id++) { mem = find_memory_block_by_id(block_id); if (WARN_ON_ONCE(!mem)) @@ -758,7 +758,6 @@ void remove_memory_block_devices(unsigne unregister_memory_block_under_nodes(mem); unregister_memory(mem); } - mutex_unlock(&mem_sysfs_mutex); } /* return true if the memory block is offlined, otherwise, return false */ @@ -792,12 +791,13 @@ static const struct attribute_group *mem }; /* - * Initialize the sysfs support for memory devices... + * Initialize the sysfs support for memory devices. At the time this function + * is called, we cannot have concurrent creation/deletion of memory block + * devices, the device_hotplug_lock is not needed. */ void __init memory_dev_init(void) { int ret; - int err; unsigned long block_sz, nr; /* Validate the configured memory block size */ @@ -808,24 +808,19 @@ void __init memory_dev_init(void) ret = subsys_system_register(&memory_subsys, memory_root_attr_groups); if (ret) - goto out; + panic("%s() failed to register subsystem: %d\n", __func__, ret); /* * Create entries for memory sections that were found * during boot and have been initialized */ - mutex_lock(&mem_sysfs_mutex); for (nr = 0; nr <= __highest_present_section_nr; nr += sections_per_block) { - err = add_memory_block(nr); - if (!ret) - ret = err; + ret = add_memory_block(nr); + if (ret) + panic("%s() failed to add memory block: %d\n", __func__, + ret); } - mutex_unlock(&mem_sysfs_mutex); - -out: - if (ret) - panic("%s() failed: %d\n", __func__, ret); } /** From patchwork Sun Dec 1 01:54:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268343 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9164E921 for ; Sun, 1 Dec 2019 01:54:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 554542245C for ; Sun, 1 Dec 2019 01:54:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="t5QkMLee" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 554542245C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 02AF26B02FD; Sat, 30 Nov 2019 20:54:20 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F1DFC6B02FF; Sat, 30 Nov 2019 20:54:19 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0EB16B0300; Sat, 30 Nov 2019 20:54:19 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0124.hostedemail.com [216.40.44.124]) by kanga.kvack.org (Postfix) with ESMTP id CD54D6B02FD for ; Sat, 30 Nov 2019 20:54:19 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 8A6EF2C7C for ; Sun, 1 Dec 2019 01:54:19 +0000 (UTC) X-FDA: 76214902638.13.bean20_25b916d2dce4e X-Spam-Summary: 50,0,0,896eb2e04c58497a,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:anshuman.khandual@arm.com:dan.j.williams@intel.com:david@redhat.com::mhocko@suse.com:mm-commits@vger.kernel.org:nao.horiguchi@gmail.com:osalvador@suse.de:pasha.tatashin@soleen.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1535:1544:1605:1711:1730:1747:1777:1792:1963:2194:2196:2198:2199:2200:2201:2393:2525:2561:2565:2682:2685:2693:2736:2741:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4605:5007:6261:6653:6737:7514:7576:7875:7903:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12291:12294:12296:12297:12438:12517:12519:12555:12679:12740:12783:12895:12986:13161:13180:13229:13255:13846:14096:14181:14721:14849:21060:21080:21094:21251:21323:21451:21627:21740:217 88:21939 X-HE-Tag: bean20_25b916d2dce4e X-Filterd-Recvd-Size: 5996 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:19 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1EC392231B; Sun, 1 Dec 2019 01:54:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165258; bh=cE1+JNvJqh5u/VQt+rXTw7yhPupAZIMAYKGYaWCtnoo=; h=Date:From:To:Subject:From; b=t5QkMLee6Rg1tx2sf8BLbaUr6y/BtWhLabg+rMIY+ovDzLrQ8A8i3+kUaZt+xohWb jKbtcCQOXSvBxY5WuzfTHTfKdDisduWnFDYvaawbAPun9iG+TugXTLxmrmbRwgGTd4 AL/hs24Ygx1NLyOcTEnSGIt5as8Fgy8M1BD13Re0= Date: Sat, 30 Nov 2019 17:54:17 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, anshuman.khandual@arm.com, dan.j.williams@intel.com, david@redhat.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, nao.horiguchi@gmail.com, osalvador@suse.de, pasha.tatashin@soleen.com, torvalds@linux-foundation.org Subject: [patch 083/158] mm/memory_hotplug.c: don't allow to online/offline memory blocks with holes Message-ID: <20191201015417.c9-W09fyc%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: mm/memory_hotplug.c: don't allow to online/offline memory blocks with holes Our onlining/offlining code is unnecessarily complicated. Only memory blocks added during boot can have holes (a range that is not IORESOURCE_SYSTEM_RAM). Hotplugged memory never has holes (e.g., see add_memory_resource()). All memory blocks that belong to boot memory are already online. Note that boot memory can have holes and the memmap of the holes is marked PG_reserved. However, also memory allocated early during boot is PG_reserved - basically every page of boot memory that is not given to the buddy is PG_reserved. Therefore, when we stop allowing to offline memory blocks with holes, we implicitly no longer have to deal with onlining memory blocks with holes. E.g., online_pages() will do a walk_system_ram_range(..., online_pages_range), whereby online_pages_range() will effectively only free the memory holes not falling into a hole to the buddy. The other pages (holes) are kept PG_reserved (via move_pfn_range_to_zone()->memmap_init_zone()). This allows to simplify the code. For example, we no longer have to worry about marking pages that fall into memory holes PG_reserved when onlining memory. We can stop setting pages PG_reserved completely in memmap_init_zone(). Offlining memory blocks added during boot is usually not guaranteed to work either way (unmovable data might have easily ended up on that memory during boot). So stopping to do that should not really hurt. Also, people are not even aware of a setup where onlining/offlining of memory blocks with holes used to work reliably (see [1] and [2] especially regarding the hotplug path) - I doubt it worked reliably. For the use case of offlining memory to unplug DIMMs, we should see no change. (holes on DIMMs would be weird). Please note that hardware errors (PG_hwpoison) are not memory holes and are not affected by this change when offlining. [1] https://lkml.org/lkml/2019/10/22/135 [2] https://lkml.org/lkml/2019/8/14/1365 Link: http://lkml.kernel.org/r/20191119115237.6662-1-david@redhat.com Reviewed-by: Dan Williams Signed-off-by: David Hildenbrand Acked-by: Michal Hocko Cc: Oscar Salvador Cc: Pavel Tatashin Cc: Dan Williams Cc: Anshuman Khandual Cc: Naoya Horiguchi Signed-off-by: Andrew Morton --- mm/memory_hotplug.c | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) --- a/mm/memory_hotplug.c~mm-memory_hotplug-dont-allow-to-online-offline-memory-blocks-with-holes +++ a/mm/memory_hotplug.c @@ -1485,10 +1485,19 @@ static void node_states_clear_node(int n node_clear_state(node, N_MEMORY); } +static int count_system_ram_pages_cb(unsigned long start_pfn, + unsigned long nr_pages, void *data) +{ + unsigned long *nr_system_ram_pages = data; + + *nr_system_ram_pages += nr_pages; + return 0; +} + static int __ref __offline_pages(unsigned long start_pfn, unsigned long end_pfn) { - unsigned long pfn, nr_pages; + unsigned long pfn, nr_pages = 0; unsigned long offlined_pages = 0; int ret, node, nr_isolate_pageblock; unsigned long flags; @@ -1499,6 +1508,22 @@ static int __ref __offline_pages(unsigne mem_hotplug_begin(); + /* + * Don't allow to offline memory blocks that contain holes. + * Consequently, memory blocks with holes can never get onlined + * via the hotplug path - online_pages() - as hotplugged memory has + * no holes. This way, we e.g., don't have to worry about marking + * memory holes PG_reserved, don't need pfn_valid() checks, and can + * avoid using walk_system_ram_range() later. + */ + walk_system_ram_range(start_pfn, end_pfn - start_pfn, &nr_pages, + count_system_ram_pages_cb); + if (nr_pages != end_pfn - start_pfn) { + ret = -EINVAL; + reason = "memory holes"; + goto failed_removal; + } + /* This makes hotplug much easier...and readable. we assume this for now. .*/ if (!test_pages_in_a_zone(start_pfn, end_pfn, &valid_start, @@ -1510,7 +1535,6 @@ static int __ref __offline_pages(unsigne zone = page_zone(pfn_to_page(valid_start)); node = zone_to_nid(zone); - nr_pages = end_pfn - start_pfn; /* set above range as isolated */ ret = start_isolate_page_range(start_pfn, end_pfn, From patchwork Sun Dec 1 01:54:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268345 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BCF8F112B for ; Sun, 1 Dec 2019 01:54:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 807FE215E5 for ; Sun, 1 Dec 2019 01:54:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="pnKsRNUX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 807FE215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 275236B02FF; Sat, 30 Nov 2019 20:54:23 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 24D416B0301; Sat, 30 Nov 2019 20:54:23 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13FA76B0302; Sat, 30 Nov 2019 20:54:23 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0042.hostedemail.com [216.40.44.42]) by kanga.kvack.org (Postfix) with ESMTP id E161C6B02FF for ; Sat, 30 Nov 2019 20:54:22 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id A4AE9180AD817 for ; Sun, 1 Dec 2019 01:54:22 +0000 (UTC) X-FDA: 76214902764.25.show26_262e57d0fe358 X-Spam-Summary: 2,0,0,b2c2db244036b8c9,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:david@redhat.com::mhocko@suse.com:mm-commits@vger.kernel.org:osalvador@suse.de:pasha.tatashin@soleen.com:torvalds@linux-foundation.org:vincent.whitchurch@axis.com,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6261:6653:7576:7903:8599:8603:8660:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12783:12986:13148:13230:13846:13869:14181:14721:14849:21063:21067:21080:21451:21627:21740:21795:21939:30005:30012:30051:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules: 0:0:0,LF X-HE-Tag: show26_262e57d0fe358 X-Filterd-Recvd-Size: 3348 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:22 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 503BE2084E; Sun, 1 Dec 2019 01:54:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165261; bh=GUtZ281BGBLSSwvJdttO1ArPz6dJmsly+Xp3hMbIG9M=; h=Date:From:To:Subject:From; b=pnKsRNUXqHoAW8YDzi39TvMI5k1r7/+Bz24NmeCPboDKxEI3e8Py+F15jXEssI77j qRBQ6pfQ/WFYV4PNqSolNzZTsk/CFKW9JANrfFmEpqaGzWgA6dNyxDayotBXWFI+4h V/DyZhFn6gbjioyhWck4ZKzWt5ksTqaPimuzcpMw= Date: Sat, 30 Nov 2019 17:54:20 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, david@redhat.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, osalvador@suse.de, pasha.tatashin@soleen.com, torvalds@linux-foundation.org, vincent.whitchurch@axis.com Subject: [patch 084/158] mm/sparse: consistently do not zero memmap Message-ID: <20191201015420.L3ynd5Uvl%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vincent Whitchurch Subject: mm/sparse: consistently do not zero memmap sparsemem without VMEMMAP has two allocation paths to allocate the memory needed for its memmap (done in sparse_mem_map_populate()). In one allocation path (sparse_buffer_alloc() succeeds), the memory is not zeroed (since it was previously allocated with memblock_alloc_try_nid_raw()). In the other allocation path (sparse_buffer_alloc() fails and sparse_mem_map_populate() falls back to memblock_alloc_try_nid()), the memory is zeroed. AFAICS this difference does not appear to be on purpose. If the code is supposed to work with non-initialized memory (__init_single_page() takes care of zeroing the struct pages which are actually used), we should consistently not zero the memory, to avoid masking bugs. (I noticed this because on my ARM64 platform, with 1 GiB of memory the first [and only] section is allocated from the zeroing path while with 2 GiB of memory the first 1 GiB section is allocated from the non-zeroing path.) Michal: "the main user visible problem is a memory wastage. The overal amount of memory should be small. I wouldn't call it stable material." Link: http://lkml.kernel.org/r/20191030131122.8256-1-vincent.whitchurch@axis.com Signed-off-by: Vincent Whitchurch Acked-by: Michal Hocko Acked-by: David Hildenbrand Reviewed-by: Oscar Salvador Reviewed-by: Pavel Tatashin Signed-off-by: Andrew Morton --- mm/sparse.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/sparse.c~mm-sparse-consistently-do-not-zero-memmap +++ a/mm/sparse.c @@ -458,7 +458,7 @@ struct page __init *__populate_section_m if (map) return map; - map = memblock_alloc_try_nid(size, + map = memblock_alloc_try_nid_raw(size, PAGE_SIZE, addr, MEMBLOCK_ALLOC_ACCESSIBLE, nid); if (!map) From patchwork Sun Dec 1 01:54:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268347 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DA6B4921 for ; Sun, 1 Dec 2019 01:54:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A6F76215E5 for ; Sun, 1 Dec 2019 01:54:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZrgoLsS9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A6F76215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 276D66B0301; Sat, 30 Nov 2019 20:54:26 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 226B16B0303; Sat, 30 Nov 2019 20:54:26 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 164246B0304; Sat, 30 Nov 2019 20:54:26 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0057.hostedemail.com [216.40.44.57]) by kanga.kvack.org (Postfix) with ESMTP id F3D716B0301 for ; Sat, 30 Nov 2019 20:54:25 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id BBC35181AEF0B for ; Sun, 1 Dec 2019 01:54:25 +0000 (UTC) X-FDA: 76214902890.21.pot77_26a1d6e54bd5b X-Spam-Summary: 2,0,0,32ed19b00ca0f867,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:david@redhat.com:gor@linux.ibm.com:heiko.carstens@de.ibm.com:iii@linux.ibm.com::mm-commits@vger.kernel.org:osalvador@suse.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2564:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3870:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:5007:6261:6653:7576:7903:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:14096:14181:14721:14849:21080:21433:21451:21627:21939:30054:30056,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:26,LUA_SUMMARY:none X-HE-Tag: pot77_26a1d6e54bd5b X-Filterd-Recvd-Size: 3432 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:25 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 8625C22464; Sun, 1 Dec 2019 01:54:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165264; bh=gAuw9+tb0KYsz6wtfIc4f3j8CApzYBV8iwhMPVTAMRA=; h=Date:From:To:Subject:From; b=ZrgoLsS9l1nCqNqabha25B9E5FXAcBX758f57dNpJ0iSyQS0LinDr8LVgY/AHaMNj svyrpV5Cx+BFxKZTpae+dudiaYyyIJBFebRuPJcnJkak1JJMdKBXdq5GIMq6q1BrU1 aS+fXoGwEwDk3eE+frNBzitrtljWHBf9a4C4JnCo= Date: Sat, 30 Nov 2019 17:54:24 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, david@redhat.com, gor@linux.ibm.com, heiko.carstens@de.ibm.com, iii@linux.ibm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, OSalvador@suse.com, torvalds@linux-foundation.org Subject: [patch 085/158] mm/sparse.c: mark populate_section_memmap as __meminit Message-ID: <20191201015424.NWY-fe-8T%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Ilya Leoshkevich Subject: mm/sparse.c: mark populate_section_memmap as __meminit Building the kernel on s390 with -Og produces the following warning: WARNING: vmlinux.o(.text+0x28dabe): Section mismatch in reference from the function populate_section_memmap() to the function .meminit.text:__populate_section_memmap() The function populate_section_memmap() references the function __meminit __populate_section_memmap(). This is often because populate_section_memmap lacks a __meminit annotation or the annotation of __populate_section_memmap is wrong. While -Og is not supported, in theory this might still happen with another compiler or on another architecture. So fix this by using the correct section annotations. [iii@linux.ibm.com: v2] Link: http://lkml.kernel.org/r/20191030151639.41486-1-iii@linux.ibm.com Link: http://lkml.kernel.org/r/20191028165549.14478-1-iii@linux.ibm.com Signed-off-by: Ilya Leoshkevich Acked-by: David Hildenbrand Cc: Heiko Carstens Cc: Vasily Gorbik Cc: Oscar Salvador Signed-off-by: Andrew Morton --- mm/sparse.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/mm/sparse.c~mm-sparsec-mark-populate_section_memmap-as-__meminit +++ a/mm/sparse.c @@ -647,7 +647,7 @@ void offline_mem_sections(unsigned long #endif #ifdef CONFIG_SPARSEMEM_VMEMMAP -static struct page *populate_section_memmap(unsigned long pfn, +static struct page * __meminit populate_section_memmap(unsigned long pfn, unsigned long nr_pages, int nid, struct vmem_altmap *altmap) { return __populate_section_memmap(pfn, nr_pages, nid, altmap); @@ -669,7 +669,7 @@ static void free_map_bootmem(struct page vmemmap_free(start, end, NULL); } #else -struct page *populate_section_memmap(unsigned long pfn, +struct page * __meminit populate_section_memmap(unsigned long pfn, unsigned long nr_pages, int nid, struct vmem_altmap *altmap) { struct page *page, *ret; From patchwork Sun Dec 1 01:54:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268349 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3BE67921 for ; Sun, 1 Dec 2019 01:54:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F389420880 for ; Sun, 1 Dec 2019 01:54:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="HdzMEFIe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F389420880 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ADA136B0303; Sat, 30 Nov 2019 20:54:29 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A897F6B0305; Sat, 30 Nov 2019 20:54:29 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C7D76B0306; Sat, 30 Nov 2019 20:54:29 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0131.hostedemail.com [216.40.44.131]) by kanga.kvack.org (Postfix) with ESMTP id 8768E6B0303 for ; Sat, 30 Nov 2019 20:54:29 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 3FB1A180AD817 for ; Sun, 1 Dec 2019 01:54:29 +0000 (UTC) X-FDA: 76214903058.05.sand27_272229111323b X-Spam-Summary: 2,0,0,cb1d80c444da9965,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:david@redhat.com::mhocko@suse.com:mm-commits@vger.kernel.org:osalvador@suse.com:pasha.tatashin@soleen.com:torvalds@linux-foundation.org:vincent.whitchurch@axis.com,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2393:2525:2559:2564:2682:2685:2859:2902:2919:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6117:6261:6653:7576:7875:7903:8599:8603:8660:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13148:13161:13221:13229:13230:13846:14181:14721:14849:21067:21080:21451:21627:21939:30054:30056:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_r ules:0:0 X-HE-Tag: sand27_272229111323b X-Filterd-Recvd-Size: 4280 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:28 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D4BD221774; Sun, 1 Dec 2019 01:54:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165268; bh=fAREv1unVnWXEyDlXNS++09m8uVAdR/fTkl6IrPJae8=; h=Date:From:To:Subject:From; b=HdzMEFIeYZHahnFcsFhh9E1grVXIFidv6ER6naunVHcp3OoUfkw9aV23WTtqS8HGd sCpkyuAfFy/eY6DdVX+XjFddpqUxmGP3URiMPVh1L40uhDrpGCGCvYSQWeJBd1yvsI TNgDwMjUFF1IxreVIQybNFQoXgrmqNERNOw8NwjY= Date: Sat, 30 Nov 2019 17:54:27 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, david@redhat.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, OSalvador@suse.com, pasha.tatashin@soleen.com, torvalds@linux-foundation.org, vincent.whitchurch@axis.com Subject: [patch 086/158] mm/sparse.c: do not waste pre allocated memmap space Message-ID: <20191201015427.E9NNqefxs%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Michal Hocko Subject: mm/sparse.c: do not waste pre allocated memmap space Vincent has noticed [1] that there is something unusual with the memmap allocations going on on his platform : I noticed this because on my ARM64 platform, with 1 GiB of memory the : first [and only] section is allocated from the zeroing path while with : 2 GiB of memory the first 1 GiB section is allocated from the : non-zeroing path. The underlying problem is that although sparse_buffer_init allocates enough memory for all sections on the node sparse_buffer_alloc is not able to consume them due to mismatch in the expected allocation alignement. While sparse_buffer_init preallocation uses the PAGE_SIZE alignment the real memmap has to be aligned to section_map_size() this results in a wasted initial chunk of the preallocated memmap and unnecessary fallback allocation for a section. While we are at it also change __populate_section_memmap to align to the requested size because at least VMEMMAP has constrains to have memmap properly aligned. [1] http://lkml.kernel.org/r/20191030131122.8256-1-vincent.whitchurch@axis.com [akpm@linux-foundation.org: tweak layout, per David] Link: http://lkml.kernel.org/r/20191119092642.31799-1-mhocko@kernel.org Fixes: 35fd1eb1e821 ("mm/sparse: abstract sparse buffer allocations") Signed-off-by: Michal Hocko Reported-by: Vincent Whitchurch Debugged-by: Vincent Whitchurch Acked-by: David Hildenbrand Cc: Pavel Tatashin Cc: Oscar Salvador Signed-off-by: Andrew Morton --- mm/sparse.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) --- a/mm/sparse.c~mm-sparse-do-not-waste-pre-allocated-memmap-space +++ a/mm/sparse.c @@ -458,8 +458,7 @@ struct page __init *__populate_section_m if (map) return map; - map = memblock_alloc_try_nid_raw(size, - PAGE_SIZE, addr, + map = memblock_alloc_try_nid_raw(size, size, addr, MEMBLOCK_ALLOC_ACCESSIBLE, nid); if (!map) panic("%s: Failed to allocate %lu bytes align=0x%lx nid=%d from=%pa\n", @@ -482,10 +481,13 @@ static void __init sparse_buffer_init(un { phys_addr_t addr = __pa(MAX_DMA_ADDRESS); WARN_ON(sparsemap_buf); /* forgot to call sparse_buffer_fini()? */ - sparsemap_buf = - memblock_alloc_try_nid_raw(size, PAGE_SIZE, - addr, - MEMBLOCK_ALLOC_ACCESSIBLE, nid); + /* + * Pre-allocated buffer is mainly used by __populate_section_memmap + * and we want it to be properly aligned to the section size - this is + * especially the case for VMEMMAP which maps memmap to PMDs + */ + sparsemap_buf = memblock_alloc_try_nid_raw(size, section_map_size(), + addr, MEMBLOCK_ALLOC_ACCESSIBLE, nid); sparsemap_buf_end = sparsemap_buf + size; } From patchwork Sun Dec 1 01:54:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268351 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 54813921 for ; Sun, 1 Dec 2019 01:54:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1677A20880 for ; Sun, 1 Dec 2019 01:54:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="GGBJfiVj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1677A20880 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A69876B0305; Sat, 30 Nov 2019 20:54:32 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9A2BB6B0307; Sat, 30 Nov 2019 20:54:32 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B5756B0308; Sat, 30 Nov 2019 20:54:32 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0172.hostedemail.com [216.40.44.172]) by kanga.kvack.org (Postfix) with ESMTP id 760B76B0305 for ; Sat, 30 Nov 2019 20:54:32 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 2D1BC40DD for ; Sun, 1 Dec 2019 01:54:32 +0000 (UTC) X-FDA: 76214903184.11.hook75_2793d84c95e12 X-Spam-Summary: 2,0,0,16b7fe5968754c0b,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org::liuxiang_1999@126.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1540:1568:1711:1714:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3151:3866:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6114:6261:6642:6653:7576:9025:9545:10004:10913:11026:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:12555:12679:12783:12986:13069:13255:13311:13357:13846:14094:14181:14384:14721:14777:14849:21080:21433:21451:21600:21627:21819:21939:30054,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:26,LUA_SUMMARY:none X-HE-Tag: hook75_2793d84c95e12 X-Filterd-Recvd-Size: 2258 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:31 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 06F4A215E5; Sun, 1 Dec 2019 01:54:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165271; bh=JB2xGGYgsgcF47SdM4MvYQUJbFXz0uQkLNDF9ZBQMKg=; h=Date:From:To:Subject:From; b=GGBJfiVjZXRllDgpvzl74H8RAXdyqp/uaWkdP2yiT9o6K3F8wWSrp9brd40TqYeWC I3TOpq46BbwmW7frNpuvZGYqTDmPPwWxzBKNmDX4d6T4sKO+9dhMd/A0/FrbYhr1Pi RQ8jds5bUTcdZTQwFF7hKvRRxiY2mBLSbjRFYG54= Date: Sat, 30 Nov 2019 17:54:30 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, linux-mm@kvack.org, liuxiang_1999@126.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 087/158] mm/vmalloc.c: remove unnecessary highmem_mask from parameter of gfpflags_allow_blocking() Message-ID: <20191201015430.tX0WIdRt9%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Liu Xiang Subject: mm/vmalloc.c: remove unnecessary highmem_mask from parameter of gfpflags_allow_blocking() gfpflags_allow_blocking() does not care about __GFP_HIGHMEM, so highmem_mask can be removed. Link: http://lkml.kernel.org/r/1568812319-3467-1-git-send-email-liuxiang_1999@126.com Signed-off-by: Liu Xiang Reviewed-by: Andrew Morton Signed-off-by: Andrew Morton --- mm/vmalloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/vmalloc.c~mm-vmalloc-remove-unnecessary-highmem_mask-from-parameter-of-gfpflags_allow_blocking +++ a/mm/vmalloc.c @@ -2440,7 +2440,7 @@ static void *__vmalloc_area_node(struct goto fail; } area->pages[i] = page; - if (gfpflags_allow_blocking(gfp_mask|highmem_mask)) + if (gfpflags_allow_blocking(gfp_mask)) cond_resched(); } atomic_long_add(area->nr_pages, &nr_vmalloc_pages); From patchwork Sun Dec 1 01:54:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268353 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B0A87112B for ; Sun, 1 Dec 2019 01:54:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 73CAF20880 for ; Sun, 1 Dec 2019 01:54:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="w/RYgsD6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 73CAF20880 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 33A6D6B0307; Sat, 30 Nov 2019 20:54:36 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 310736B0309; Sat, 30 Nov 2019 20:54:36 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24CB36B030A; Sat, 30 Nov 2019 20:54:36 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0201.hostedemail.com [216.40.44.201]) by kanga.kvack.org (Postfix) with ESMTP id 104A66B0307 for ; Sat, 30 Nov 2019 20:54:36 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id BBB1E824999B for ; Sun, 1 Dec 2019 01:54:35 +0000 (UTC) X-FDA: 76214903310.20.dolls76_2810f5e627901 X-Spam-Summary: 2,0,0,69ff1056bbbd97a7,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:bigeasy@linutronix.de:dwagner@suse.de:hdanton@sina.com::mhocko@suse.com:mm-commits@vger.kernel.org:oleksiy.avramchenko@sonymobile.com:peterz@infradead.org:rostedt@goodmis.org:tglx@linutronix.de:torvalds@linux-foundation.org:urezki@gmail.com:willy@infradead.org,RULES_HIT:41:69:355:379:800:960:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1535:1544:1605:1711:1730:1747:1777:1792:2195:2196:2199:2200:2393:2525:2559:2563:2682:2685:2693:2732:2859:2902:2903:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3151:3503:3504:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:5007:6261:6653:6737:7514:7576:7903:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12291:12295:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13161:13229:13846:14096:14181:14721:14819:14849:21080:21433 :21451:2 X-HE-Tag: dolls76_2810f5e627901 X-Filterd-Recvd-Size: 5903 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:35 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2AAD6217AB; Sun, 1 Dec 2019 01:54:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165274; bh=IVJpqWQYhswBsHF9el9MGqjNB8khvPveAEIB7B/vAr8=; h=Date:From:To:Subject:From; b=w/RYgsD6ZVertR5BgR0PQNbzPn10Z4eOuRnCji6IIodKAURssbLGrNoYhGc4QGgAY wnmKyfWyW5l/kIFgEMgDtF4dyIFKr227NhA8bDov4rOpzo9ecs3FS29FKQH9RfpKD9 3wIACaGubpuGoLhLAqwP3uEGBIuem19bk4py3ci4= Date: Sat, 30 Nov 2019 17:54:33 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, bigeasy@linutronix.de, dwagner@suse.de, hdanton@sina.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, oleksiy.avramchenko@sonymobile.com, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, torvalds@linux-foundation.org, urezki@gmail.com, willy@infradead.org Subject: [patch 088/158] mm/vmalloc: remove preempt_disable/enable when doing preloading Message-ID: <20191201015433.nDqmkbg4_%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Uladzislau Rezki (Sony)" Subject: mm/vmalloc: remove preempt_disable/enable when doing preloading Some background. The preemption was disabled before to guarantee that a preloaded object is available for a CPU, it was stored for. That was achieved by combining the disabling the preemption and taking the spin lock while the ne_fit_preload_node is checked. The aim was to not allocate in atomic context when spinlock is taken later, for regular vmap allocations. But that approach conflicts with CONFIG_PREEMPT_RT philosophy. It means that calling spin_lock() with disabled preemption is forbidden in the CONFIG_PREEMPT_RT kernel. Therefore, get rid of preempt_disable() and preempt_enable() when the preload is done for splitting purpose. As a result we do not guarantee now that a CPU is preloaded, instead we minimize the case when it is not, with this change, by populating the per cpu preload pointer under the vmap_area_lock. This implies that at least each caller that has done the preallocation will not fallback to an atomic allocation later. It is possible that the preallocation would be pointless or that no preallocation is done because of the race but the data shows that this is really rare. For example i run the special test case that follows the preload pattern and path. 20 "unbind" threads run it and each does 1000000 allocations. Only 3.5 times among 1000000 a CPU was not preloaded. So it can happen but the number is negligible. [mhocko@suse.com: changelog additions] Link: http://lkml.kernel.org/r/20191016095438.12391-1-urezki@gmail.com Fixes: 82dd23e84be3 ("mm/vmalloc.c: preload a CPU with one object for split purpose") Signed-off-by: Uladzislau Rezki (Sony) Reviewed-by: Steven Rostedt (VMware) Acked-by: Sebastian Andrzej Siewior Acked-by: Daniel Wagner Acked-by: Michal Hocko Cc: Hillf Danton Cc: Matthew Wilcox Cc: Oleksiy Avramchenko Cc: Peter Zijlstra Cc: Thomas Gleixner Signed-off-by: Andrew Morton --- mm/vmalloc.c | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) --- a/mm/vmalloc.c~mm-vmalloc-remove-preempt_disable-enable-when-do-preloading +++ a/mm/vmalloc.c @@ -1077,31 +1077,34 @@ static struct vmap_area *alloc_vmap_area retry: /* - * Preload this CPU with one extra vmap_area object to ensure - * that we have it available when fit type of free area is - * NE_FIT_TYPE. + * Preload this CPU with one extra vmap_area object. It is used + * when fit type of free area is NE_FIT_TYPE. Please note, it + * does not guarantee that an allocation occurs on a CPU that + * is preloaded, instead we minimize the case when it is not. + * It can happen because of cpu migration, because there is a + * race until the below spinlock is taken. * * The preload is done in non-atomic context, thus it allows us * to use more permissive allocation masks to be more stable under - * low memory condition and high memory pressure. + * low memory condition and high memory pressure. In rare case, + * if not preloaded, GFP_NOWAIT is used. * - * Even if it fails we do not really care about that. Just proceed - * as it is. "overflow" path will refill the cache we allocate from. + * Set "pva" to NULL here, because of "retry" path. */ - preempt_disable(); - if (!__this_cpu_read(ne_fit_preload_node)) { - preempt_enable(); - pva = kmem_cache_alloc_node(vmap_area_cachep, GFP_KERNEL, node); - preempt_disable(); + pva = NULL; - if (__this_cpu_cmpxchg(ne_fit_preload_node, NULL, pva)) { - if (pva) - kmem_cache_free(vmap_area_cachep, pva); - } - } + if (!this_cpu_read(ne_fit_preload_node)) + /* + * Even if it fails we do not really care about that. + * Just proceed as it is. If needed "overflow" path + * will refill the cache we allocate from. + */ + pva = kmem_cache_alloc_node(vmap_area_cachep, GFP_KERNEL, node); spin_lock(&vmap_area_lock); - preempt_enable(); + + if (pva && __this_cpu_cmpxchg(ne_fit_preload_node, NULL, pva)) + kmem_cache_free(vmap_area_cachep, pva); /* * If an allocation fails, the "vend" address is From patchwork Sun Dec 1 01:54:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268355 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 00B3A112B for ; Sun, 1 Dec 2019 01:54:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C283B2084D for ; Sun, 1 Dec 2019 01:54:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="m6IZnxgN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C283B2084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9A1356B0309; Sat, 30 Nov 2019 20:54:39 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 953096B030B; Sat, 30 Nov 2019 20:54:39 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 88E496B030C; Sat, 30 Nov 2019 20:54:39 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0052.hostedemail.com [216.40.44.52]) by kanga.kvack.org (Postfix) with ESMTP id 744526B0309 for ; Sat, 30 Nov 2019 20:54:39 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 23FDB2C7C for ; Sun, 1 Dec 2019 01:54:39 +0000 (UTC) X-FDA: 76214903478.08.list48_2895bd6a58b00 X-Spam-Summary: 2,0,0,84dcb638752b98ea,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:bigeasy@linutronix.de:dwagner@suse.de:hdanton@sina.com::mhocko@suse.com:mm-commits@vger.kernel.org:oleksiy.avramchenko@sonymobile.com:peterz@infradead.org:rostedt@goodmis.org:tglx@linutronix.de:torvalds@linux-foundation.org:urezki@gmail.com:willy@infradead.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2195:2198:2199:2200:2393:2525:2559:2563:2682:2685:2693:2732:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3151:3353:3503:3504:3865:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:6737:7514:7576:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13846:14096:14181:14721:14849:21080:21451:21600:21627:21939:30012:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5 ,0.5,Net X-HE-Tag: list48_2895bd6a58b00 X-Filterd-Recvd-Size: 3808 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:38 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 86444215E5; Sun, 1 Dec 2019 01:54:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165278; bh=SfdAgLZQJYSXSNdrbz57eEb0GnyRJ87tQK66DkgHKoI=; h=Date:From:To:Subject:From; b=m6IZnxgN2kVLd5N/j+Im2tcPqgAPgN+EOYHxg6hjuQip6TmX8Ak6GaQuDRELkktX7 Y4uPrkLyfT3AmilNcV76l8pcT+Hhz5j3nMENoxvNogU+eQr4DSm0czhdwUW1d7LXtD aDSbSdsWoaM1MIbkBQHUVkj5t5MsTGmTaOGZnz2k= Date: Sat, 30 Nov 2019 17:54:37 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, bigeasy@linutronix.de, dwagner@suse.de, hdanton@sina.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, oleksiy.avramchenko@sonymobile.com, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, torvalds@linux-foundation.org, urezki@gmail.com, willy@infradead.org Subject: [patch 089/158] mm/vmalloc: respect passed gfp_mask when doing preloading Message-ID: <20191201015437._pPJWeQrf%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Uladzislau Rezki (Sony)" Subject: mm/vmalloc: respect passed gfp_mask when doing preloading Allocation functions should comply with the given gfp_mask as much as possible. The preallocation code in alloc_vmap_area doesn't follow that pattern and it is using a hardcoded GFP_KERNEL. Although this doesn't really make much difference because vmalloc is not GFP_NOWAIT compliant in general (e.g. page table allocations are GFP_KERNEL) there is no reason to spread that bad habit and it is good to fix the antipattern. [mhocko@suse.com: rewrite changelog] Link: http://lkml.kernel.org/r/20191016095438.12391-2-urezki@gmail.com Signed-off-by: Uladzislau Rezki (Sony) Acked-by: Michal Hocko Cc: Daniel Wagner Cc: Hillf Danton Cc: Matthew Wilcox Cc: Oleksiy Avramchenko Cc: Peter Zijlstra Cc: Sebastian Andrzej Siewior Cc: Steven Rostedt Cc: Thomas Gleixner Signed-off-by: Andrew Morton --- mm/vmalloc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) --- a/mm/vmalloc.c~mm-vmalloc-respect-passed-gfp_mask-when-do-preloading +++ a/mm/vmalloc.c @@ -1063,9 +1063,9 @@ static struct vmap_area *alloc_vmap_area return ERR_PTR(-EBUSY); might_sleep(); + gfp_mask = gfp_mask & GFP_RECLAIM_MASK; - va = kmem_cache_alloc_node(vmap_area_cachep, - gfp_mask & GFP_RECLAIM_MASK, node); + va = kmem_cache_alloc_node(vmap_area_cachep, gfp_mask, node); if (unlikely(!va)) return ERR_PTR(-ENOMEM); @@ -1073,7 +1073,7 @@ static struct vmap_area *alloc_vmap_area * Only scan the relevant parts containing pointers to other objects * to avoid false negatives. */ - kmemleak_scan_area(&va->rb_node, SIZE_MAX, gfp_mask & GFP_RECLAIM_MASK); + kmemleak_scan_area(&va->rb_node, SIZE_MAX, gfp_mask); retry: /* @@ -1099,7 +1099,7 @@ retry: * Just proceed as it is. If needed "overflow" path * will refill the cache we allocate from. */ - pva = kmem_cache_alloc_node(vmap_area_cachep, GFP_KERNEL, node); + pva = kmem_cache_alloc_node(vmap_area_cachep, gfp_mask, node); spin_lock(&vmap_area_lock); From patchwork Sun Dec 1 01:54:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268357 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5CDBC921 for ; Sun, 1 Dec 2019 01:54:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 28BA520880 for ; Sun, 1 Dec 2019 01:54:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="ofMpfTGs" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 28BA520880 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EF9486B030B; Sat, 30 Nov 2019 20:54:42 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EA8436B030D; Sat, 30 Nov 2019 20:54:42 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0D096B030E; Sat, 30 Nov 2019 20:54:42 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0189.hostedemail.com [216.40.44.189]) by kanga.kvack.org (Postfix) with ESMTP id CF7526B030B for ; Sat, 30 Nov 2019 20:54:42 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 7B18E181AEF0B for ; Sun, 1 Dec 2019 01:54:42 +0000 (UTC) X-FDA: 76214903604.23.pig94_291323d99361e X-Spam-Summary: 2,0,0,612f007409ebfd89,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:bigeasy@linutronix.de:dwagner@suse.de:hdanton@sina.com::mhocko@suse.com:mm-commits@vger.kernel.org:oleksiy.avramchenko@sonymobile.com:peterz@infradead.org:rostedt@goodmis.org:tglx@linutronix.de:torvalds@linux-foundation.org:urezki@gmail.com:willy@infradead.org,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3151:3353:3503:3504:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:5007:6261:6653:6737:6755:7514:7576:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12114:12219:12296:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:13846:14096:14181:14721:14849:21080:21324:21433:21451:21600:21627:21939:30054:30074,0 ,RBL:err X-HE-Tag: pig94_291323d99361e X-Filterd-Recvd-Size: 3733 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:42 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 14D5D2084D; Sun, 1 Dec 2019 01:54:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165281; bh=4Ei9u6EdIshRDgIIneK55ObFablUeWubKpy9n77vs/g=; h=Date:From:To:Subject:From; b=ofMpfTGsCGFZF/AVZezu8SMqk1C/qNktKLQXM2BG5re3fdIZQPnmge8NFtLXCcBg/ Rf8lXYkd2LXkISpki0n4GG9hOr2u4ISBVEmKrTiK+Un99/65tLSXyVsgQjqt2oaOfu Biyl2VglYHqkUPKW9YeqnacFGfSbeD+tnxiIlsaI= Date: Sat, 30 Nov 2019 17:54:40 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, bigeasy@linutronix.de, dwagner@suse.de, hdanton@sina.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, oleksiy.avramchenko@sonymobile.com, peterz@infradead.org, rostedt@goodmis.org, tglx@linutronix.de, torvalds@linux-foundation.org, urezki@gmail.com, willy@infradead.org Subject: [patch 090/158] mm/vmalloc: add more comments to the adjust_va_to_fit_type() Message-ID: <20191201015440.p_aqbgBSe%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Uladzislau Rezki (Sony)" Subject: mm/vmalloc: add more comments to the adjust_va_to_fit_type() When fit type is NE_FIT_TYPE there is a need in one extra object. Usually the "ne_fit_preload_node" per-CPU variable has it and there is no need in GFP_NOWAIT allocation, but there are exceptions. This commit just adds more explanations, as a result giving answers on questions like when it can occur, how often, under which conditions and what happens if GFP_NOWAIT gets failed. Link: http://lkml.kernel.org/r/20191016095438.12391-3-urezki@gmail.com Signed-off-by: Uladzislau Rezki (Sony) Acked-by: Michal Hocko Cc: Daniel Wagner Cc: Sebastian Andrzej Siewior Cc: Thomas Gleixner Cc: Peter Zijlstra Cc: Uladzislau Rezki Cc: Hillf Danton Cc: Michal Hocko Cc: Matthew Wilcox Cc: Oleksiy Avramchenko Cc: Steven Rostedt Signed-off-by: Andrew Morton --- mm/vmalloc.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) --- a/mm/vmalloc.c~mm-vmalloc-add-more-comments-to-the-adjust_va_to_fit_type +++ a/mm/vmalloc.c @@ -968,6 +968,19 @@ adjust_va_to_fit_type(struct vmap_area * * There are a few exceptions though, as an example it is * a first allocation (early boot up) when we have "one" * big free space that has to be split. + * + * Also we can hit this path in case of regular "vmap" + * allocations, if "this" current CPU was not preloaded. + * See the comment in alloc_vmap_area() why. If so, then + * GFP_NOWAIT is used instead to get an extra object for + * split purpose. That is rare and most time does not + * occur. + * + * What happens if an allocation gets failed. Basically, + * an "overflow" path is triggered to purge lazily freed + * areas to free some memory, then, the "retry" path is + * triggered to repeat one more time. See more details + * in alloc_vmap_area() function. */ lva = kmem_cache_alloc(vmap_area_cachep, GFP_NOWAIT); if (!lva) From patchwork Sun Dec 1 01:54:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268359 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9A89F921 for ; Sun, 1 Dec 2019 01:54:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 67EBC20880 for ; Sun, 1 Dec 2019 01:54:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="w+6XyIJJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 67EBC20880 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1B2B46B030D; Sat, 30 Nov 2019 20:54:46 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 162ED6B030F; Sat, 30 Nov 2019 20:54:46 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 052866B0310; Sat, 30 Nov 2019 20:54:45 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0086.hostedemail.com [216.40.44.86]) by kanga.kvack.org (Postfix) with ESMTP id DED146B030D for ; Sat, 30 Nov 2019 20:54:45 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 96A142A87 for ; Sun, 1 Dec 2019 01:54:45 +0000 (UTC) X-FDA: 76214903730.17.plane93_2984a1a21590b X-Spam-Summary: 2,0,0,00a87259e97ebc5d,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:anders.roxell@linaro.org::mm-commits@vger.kernel.org:shuah@kernel.org:torvalds@linux-foundation.org:urezki@gmail.com,RULES_HIT:41:355:379:800:960:967:973:982:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1714:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3350:3503:3504:3865:3867:3870:3871:3873:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7514:7576:7619:8599:8784:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13069:13311:13357:13846:14094:14181:14384:14721:14849:21080:21451:21627:21740:21939:30045:30054:30056,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: plane93_2984a1a21590b X-Filterd-Recvd-Size: 2318 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:45 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5E120208C3; Sun, 1 Dec 2019 01:54:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165284; bh=cfwWM4D6//1Kp4fIIRpbu0qwHo3h5dZ9VxyRbnoYkWs=; h=Date:From:To:Subject:From; b=w+6XyIJJEa5d2C1lVTFc0wuLA6dm8Hs/Dx0/aSpnfMwvf6HOkYH8AzcO0+PAuu1aQ 9ddUSJk98GSdck/19t29CmqD6UvUf+vhb3s8HWYrbWHfi9f3sZtTa/9lWPgJSa87bV sWjNVniT+V86sVX0OTN/z6cni7axN1kUS29mkFi0= Date: Sat, 30 Nov 2019 17:54:43 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, anders.roxell@linaro.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, shuah@kernel.org, torvalds@linux-foundation.org, urezki@gmail.com Subject: [patch 091/158] selftests: vm: add fragment CONFIG_TEST_VMALLOC Message-ID: <20191201015443.WX4wQj4ID%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Anders Roxell Subject: selftests: vm: add fragment CONFIG_TEST_VMALLOC When running test_vmalloc.sh smoke the following print out states that the fragment is missing. # ./test_vmalloc.sh: You must have the following enabled in your kernel: # CONFIG_TEST_VMALLOC=m Rework to add the fragment 'CONFIG_TEST_VMALLOC=m' to the config file. Link: http://lkml.kernel.org/r/20190916095217.19665-1-anders.roxell@linaro.org Fixes: a05ef00c9790 ("selftests/vm: add script helper for CONFIG_TEST_VMALLOC_MODULE") Signed-off-by: Anders Roxell Cc: Shuah Khan Cc: "Uladzislau Rezki (Sony)" Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/config | 1 + 1 file changed, 1 insertion(+) --- a/tools/testing/selftests/vm/config~selftests-vm-add-fragment-config_test_vmalloc +++ a/tools/testing/selftests/vm/config @@ -1,2 +1,3 @@ CONFIG_SYSVIPC=y CONFIG_USERFAULTFD=y +CONFIG_TEST_VMALLOC=m From patchwork Sun Dec 1 01:54:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268361 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1F785921 for ; Sun, 1 Dec 2019 01:54:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D449820880 for ; Sun, 1 Dec 2019 01:54:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="wTvEdHl5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D449820880 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A64FD6B030F; Sat, 30 Nov 2019 20:54:49 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9C7026B0311; Sat, 30 Nov 2019 20:54:49 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 868846B0312; Sat, 30 Nov 2019 20:54:49 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0004.hostedemail.com [216.40.44.4]) by kanga.kvack.org (Postfix) with ESMTP id 62FB76B030F for ; Sat, 30 Nov 2019 20:54:49 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 16A5F2A87 for ; Sun, 1 Dec 2019 01:54:49 +0000 (UTC) X-FDA: 76214903898.05.pie32_29ffc963c7561 X-Spam-Summary: 2,0,0,c88ab517e447a1e4,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hdanton@sina.com::mhocko@suse.com:mm-commits@vger.kernel.org:oleksiy.avramchenko@sonymobile.com:rostedt@goodmis.org:torvalds@linux-foundation.org:urezki@gmail.com:willy@infradead.org,RULES_HIT:1:2:41:69:355:379:800:960:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1605:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2525:2559:2563:2682:2685:2693:2731:2741:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3151:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4051:4250:4321:4385:4605:5007:6261:6653:7514:7576:7903:8599:8603:8957:9010:9025:9036:9545:9592:9707:10004:10913:11026:11473:11658:11914:12043:12048:12291:12294:12296:12297:12438:12517:12519:12555:12679:12683:12739:12783:12986:13846:13972:14096:21080:21451:21600:21611:21627:21740:21790:21939:30054,0,RBL:error,CacheIP:none,Bayesian:0 .5,0.5,0 X-HE-Tag: pie32_29ffc963c7561 X-Filterd-Recvd-Size: 10025 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:48 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9310A208E4; Sun, 1 Dec 2019 01:54:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165287; bh=5PqCjY/r/tdFtFBsYkLcQgKU8NLH28o+Z0uktE9D728=; h=Date:From:To:Subject:From; b=wTvEdHl5Vi7BsP4KN+qehjlHLr90PfIyF2Gn7q3+GKkRSeAJ1eD6VdlNJH+Foedl3 HzrVdjaMEkKzEhnp2rnHWMZiW/j1MZyZATqFxTLxiwfE+gRWaz38Nbcfx0oP8zXyKB NA21HGaAcTaR/HyDIGxA6FYWpu8Ja4yborFUEjTQ= Date: Sat, 30 Nov 2019 17:54:47 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hdanton@sina.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, oleksiy.avramchenko@sonymobile.com, rostedt@goodmis.org, torvalds@linux-foundation.org, urezki@gmail.com, willy@infradead.org Subject: [patch 092/158] mm/vmalloc: rework vmap_area_lock Message-ID: <20191201015447.bPXjuzW7E%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Uladzislau Rezki (Sony)" Subject: mm/vmalloc: rework vmap_area_lock With the new allocation approach introduced in the 5.2 kernel, it becomes possible to get rid of one global spinlock. By doing that we can further improve the KVA from the performance point of view. Basically we can have two independent locks, one for allocation part and another one for deallocation, because of two different entities: "free data structures" and "busy data structures". As a result, allocation/deallocation operations can still interfere between each other in case of running simultaneously on different CPUs, it means there is still dependency, but with two locks it becomes lower. Summarizing: - it reduces the high lock contention - it allows to perform operations on "free" and "busy" trees in parallel on different CPUs. Please note it does not solve scalability issue. Test results: In order to evaluate this patch, we can run "vmalloc test driver" to see how many CPU cycles it takes to complete all test cases running sequentially. All online CPUs run it so it will cause a high lock contention. HiKey 960, ARM64, 8xCPUs, big.LITTLE: sudo ./test_vmalloc.sh sequential_test_order=1 [ 390.950557] All test took CPU0=457126382 cycles [ 391.046690] All test took CPU1=454763452 cycles [ 391.128586] All test took CPU2=454539334 cycles [ 391.222669] All test took CPU3=455649517 cycles [ 391.313946] All test took CPU4=388272196 cycles [ 391.410425] All test took CPU5=384036264 cycles [ 391.492219] All test took CPU6=387432964 cycles [ 391.578433] All test took CPU7=387201996 cycles [ 304.721224] All test took CPU0=391521310 cycles [ 304.821219] All test took CPU1=393533002 cycles [ 304.917120] All test took CPU2=392243032 cycles [ 305.008986] All test took CPU3=392353853 cycles [ 305.108944] All test took CPU4=297630721 cycles [ 305.196406] All test took CPU5=297548736 cycles [ 305.288602] All test took CPU6=297092392 cycles [ 305.381088] All test took CPU7=297293597 cycles ~14%-23% patched variant is better. Link: http://lkml.kernel.org/r/20191022155800.20468-1-urezki@gmail.com Signed-off-by: Uladzislau Rezki (Sony) Acked-by: Andrew Morton Cc: Hillf Danton Cc: Michal Hocko Cc: Matthew Wilcox Cc: Oleksiy Avramchenko Cc: Steven Rostedt Signed-off-by: Andrew Morton --- mm/vmalloc.c | 80 ++++++++++++++++++++++++++++++------------------- 1 file changed, 50 insertions(+), 30 deletions(-) --- a/mm/vmalloc.c~mm-vmalloc-rework-vmap_area_lock +++ a/mm/vmalloc.c @@ -331,6 +331,7 @@ EXPORT_SYMBOL(vmalloc_to_pfn); static DEFINE_SPINLOCK(vmap_area_lock); +static DEFINE_SPINLOCK(free_vmap_area_lock); /* Export for kexec only */ LIST_HEAD(vmap_area_list); static LLIST_HEAD(vmap_purge_list); @@ -1114,7 +1115,7 @@ retry: */ pva = kmem_cache_alloc_node(vmap_area_cachep, gfp_mask, node); - spin_lock(&vmap_area_lock); + spin_lock(&free_vmap_area_lock); if (pva && __this_cpu_cmpxchg(ne_fit_preload_node, NULL, pva)) kmem_cache_free(vmap_area_cachep, pva); @@ -1124,14 +1125,17 @@ retry: * returned. Therefore trigger the overflow path. */ addr = __alloc_vmap_area(size, align, vstart, vend); + spin_unlock(&free_vmap_area_lock); + if (unlikely(addr == vend)) goto overflow; va->va_start = addr; va->va_end = addr + size; va->vm = NULL; - insert_vmap_area(va, &vmap_area_root, &vmap_area_list); + spin_lock(&vmap_area_lock); + insert_vmap_area(va, &vmap_area_root, &vmap_area_list); spin_unlock(&vmap_area_lock); BUG_ON(!IS_ALIGNED(va->va_start, align)); @@ -1141,7 +1145,6 @@ retry: return va; overflow: - spin_unlock(&vmap_area_lock); if (!purged) { purge_vmap_area_lazy(); purged = 1; @@ -1177,28 +1180,25 @@ int unregister_vmap_purge_notifier(struc } EXPORT_SYMBOL_GPL(unregister_vmap_purge_notifier); -static void __free_vmap_area(struct vmap_area *va) +/* + * Free a region of KVA allocated by alloc_vmap_area + */ +static void free_vmap_area(struct vmap_area *va) { /* * Remove from the busy tree/list. */ + spin_lock(&vmap_area_lock); unlink_va(va, &vmap_area_root); + spin_unlock(&vmap_area_lock); /* - * Merge VA with its neighbors, otherwise just add it. + * Insert/Merge it back to the free tree/list. */ + spin_lock(&free_vmap_area_lock); merge_or_add_vmap_area(va, &free_vmap_area_root, &free_vmap_area_list); -} - -/* - * Free a region of KVA allocated by alloc_vmap_area - */ -static void free_vmap_area(struct vmap_area *va) -{ - spin_lock(&vmap_area_lock); - __free_vmap_area(va); - spin_unlock(&vmap_area_lock); + spin_unlock(&free_vmap_area_lock); } /* @@ -1291,7 +1291,7 @@ static bool __purge_vmap_area_lazy(unsig flush_tlb_kernel_range(start, end); resched_threshold = lazy_max_pages() << 1; - spin_lock(&vmap_area_lock); + spin_lock(&free_vmap_area_lock); llist_for_each_entry_safe(va, n_va, valist, purge_list) { unsigned long nr = (va->va_end - va->va_start) >> PAGE_SHIFT; @@ -1306,9 +1306,9 @@ static bool __purge_vmap_area_lazy(unsig atomic_long_sub(nr, &vmap_lazy_nr); if (atomic_long_read(&vmap_lazy_nr) < resched_threshold) - cond_resched_lock(&vmap_area_lock); + cond_resched_lock(&free_vmap_area_lock); } - spin_unlock(&vmap_area_lock); + spin_unlock(&free_vmap_area_lock); return true; } @@ -2030,15 +2030,21 @@ int map_vm_area(struct vm_struct *area, } EXPORT_SYMBOL_GPL(map_vm_area); -static void setup_vmalloc_vm(struct vm_struct *vm, struct vmap_area *va, - unsigned long flags, const void *caller) +static inline void setup_vmalloc_vm_locked(struct vm_struct *vm, + struct vmap_area *va, unsigned long flags, const void *caller) { - spin_lock(&vmap_area_lock); vm->flags = flags; vm->addr = (void *)va->va_start; vm->size = va->va_end - va->va_start; vm->caller = caller; va->vm = vm; +} + +static void setup_vmalloc_vm(struct vm_struct *vm, struct vmap_area *va, + unsigned long flags, const void *caller) +{ + spin_lock(&vmap_area_lock); + setup_vmalloc_vm_locked(vm, va, flags, caller); spin_unlock(&vmap_area_lock); } @@ -3298,7 +3304,7 @@ struct vm_struct **pcpu_get_vm_areas(con goto err_free; } retry: - spin_lock(&vmap_area_lock); + spin_lock(&free_vmap_area_lock); /* start scanning - we scan from the top, begin with the last area */ area = term_area = last_area; @@ -3380,29 +3386,38 @@ retry: va = vas[area]; va->va_start = start; va->va_end = start + size; - - insert_vmap_area(va, &vmap_area_root, &vmap_area_list); } - spin_unlock(&vmap_area_lock); + spin_unlock(&free_vmap_area_lock); /* insert all vm's */ - for (area = 0; area < nr_vms; area++) - setup_vmalloc_vm(vms[area], vas[area], VM_ALLOC, + spin_lock(&vmap_area_lock); + for (area = 0; area < nr_vms; area++) { + insert_vmap_area(vas[area], &vmap_area_root, &vmap_area_list); + + setup_vmalloc_vm_locked(vms[area], vas[area], VM_ALLOC, pcpu_get_vm_areas); + } + spin_unlock(&vmap_area_lock); kfree(vas); return vms; recovery: - /* Remove previously inserted areas. */ + /* + * Remove previously allocated areas. There is no + * need in removing these areas from the busy tree, + * because they are inserted only on the final step + * and when pcpu_get_vm_areas() is success. + */ while (area--) { - __free_vmap_area(vas[area]); + merge_or_add_vmap_area(vas[area], + &free_vmap_area_root, &free_vmap_area_list); vas[area] = NULL; } overflow: - spin_unlock(&vmap_area_lock); + spin_unlock(&free_vmap_area_lock); if (!purged) { purge_vmap_area_lazy(); purged = true; @@ -3453,9 +3468,12 @@ void pcpu_free_vm_areas(struct vm_struct #ifdef CONFIG_PROC_FS static void *s_start(struct seq_file *m, loff_t *pos) + __acquires(&vmap_purge_lock) __acquires(&vmap_area_lock) { + mutex_lock(&vmap_purge_lock); spin_lock(&vmap_area_lock); + return seq_list_start(&vmap_area_list, *pos); } @@ -3465,8 +3483,10 @@ static void *s_next(struct seq_file *m, } static void s_stop(struct seq_file *m, void *p) + __releases(&vmap_purge_lock) __releases(&vmap_area_lock) { + mutex_unlock(&vmap_purge_lock); spin_unlock(&vmap_area_lock); } From patchwork Sun Dec 1 01:54:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268363 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 33492921 for ; Sun, 1 Dec 2019 01:54:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CC85A206E1 for ; Sun, 1 Dec 2019 01:54:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="qw+B7Srw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CC85A206E1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6BBB96B0311; Sat, 30 Nov 2019 20:54:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 61E016B0313; Sat, 30 Nov 2019 20:54:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4BECA6B0316; Sat, 30 Nov 2019 20:54:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0156.hostedemail.com [216.40.44.156]) by kanga.kvack.org (Postfix) with ESMTP id 2ABF56B0311 for ; Sat, 30 Nov 2019 20:54:53 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id D291E824999B for ; Sun, 1 Dec 2019 01:54:52 +0000 (UTC) X-FDA: 76214904024.29.range77_2a8215bbb4b4e X-Spam-Summary: 50,0,0,5df22acc28db1aa3,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:aryabinin@virtuozzo.com:cai@lca.pw:christophe.leroy@c-s.fr:dja@axtens.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com::mark.rutland@arm.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:69:327:355:379:800:960:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1605:1730:1747:1777:1792:1801:1981:2194:2196:2198:2199:2200:2201:2393:2525:2553:2561:2568:2627:2682:2685:2689:2693:2731:2740:2859:2901:2902:2903:2904:2918:2933:2937:2939:2942:2945:2947:2951:2954:2987:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4605:5007:6119:6261:6653:6691:6737:7576:7875:7903:7904:8599:8603:8660:8784:9010:9025:9036:9040:9121:9545:10004:10913:11026:11233:11473:11658:11914:12043:12048:12219:12291:12296:12297:12438:12517:12519:12555:12663:12679:12683:12698:12737:12783:1289 5:12986: X-HE-Tag: range77_2a8215bbb4b4e X-Filterd-Recvd-Size: 31356 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:52 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0B3782464B; Sun, 1 Dec 2019 01:54:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165291; bh=YTlk6GMfSIEbdZFk+G3tay/WaU6LAGFptQUtqfjbhxI=; h=Date:From:To:Subject:From; b=qw+B7Srw9rvqzTOHi7Tftb1aA9dUQP/TCbdxwIYev1ACpON8xo4n4FGWxLLIvDsSS EIMHm3Bb5+cL5j2jwiOSOSaFbyRpOz+MOwUDaWWVeuwSxD3H8uOmnS0VMBPpBq4v13 gtnuS6BWKNtkmpwijm5uJyQV2S9PdRp/HuhBH8g4= Date: Sat, 30 Nov 2019 17:54:50 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, aryabinin@virtuozzo.com, cai@lca.pw, christophe.leroy@c-s.fr, dja@axtens.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, linux-mm@kvack.org, mark.rutland@arm.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 093/158] kasan: support backing vmalloc space with real shadow memory Message-ID: <20191201015450.z1NgbdP0R%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Daniel Axtens Subject: kasan: support backing vmalloc space with real shadow memory Patch series "kasan: support backing vmalloc space with real shadow memory", v11. Currently, vmalloc space is backed by the early shadow page. This means that kasan is incompatible with VMAP_STACK. This series provides a mechanism to back vmalloc space with real, dynamically allocated memory. I have only wired up x86, because that's the only currently supported arch I can work with easily, but it's very easy to wire up other architectures, and it appears that there is some work-in-progress code to do this on arm64 and s390. This has been discussed before in the context of VMAP_STACK: - https://bugzilla.kernel.org/show_bug.cgi?id=202009 - https://lkml.org/lkml/2018/7/22/198 - https://lkml.org/lkml/2019/7/19/822 In terms of implementation details: Most mappings in vmalloc space are small, requiring less than a full page of shadow space. Allocating a full shadow page per mapping would therefore be wasteful. Furthermore, to ensure that different mappings use different shadow pages, mappings would have to be aligned to KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE. Instead, share backing space across multiple mappings. Allocate a backing page when a mapping in vmalloc space uses a particular page of the shadow region. This page can be shared by other vmalloc mappings later on. We hook in to the vmap infrastructure to lazily clean up unused shadow memory. Testing with test_vmalloc.sh on an x86 VM with 2 vCPUs shows that: - Turning on KASAN, inline instrumentation, without vmalloc, introuduces a 4.1x-4.2x slowdown in vmalloc operations. - Turning this on introduces the following slowdowns over KASAN: * ~1.76x slower single-threaded (test_vmalloc.sh performance) * ~2.18x slower when both cpus are performing operations simultaneously (test_vmalloc.sh sequential_test_order=1) This is unfortunate but given that this is a debug feature only, not the end of the world. The benchmarks are also a stress-test for the vmalloc subsystem: they're not indicative of an overall 2x slowdown! This patch (of 4): Hook into vmalloc and vmap, and dynamically allocate real shadow memory to back the mappings. Most mappings in vmalloc space are small, requiring less than a full page of shadow space. Allocating a full shadow page per mapping would therefore be wasteful. Furthermore, to ensure that different mappings use different shadow pages, mappings would have to be aligned to KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE. Instead, share backing space across multiple mappings. Allocate a backing page when a mapping in vmalloc space uses a particular page of the shadow region. This page can be shared by other vmalloc mappings later on. We hook in to the vmap infrastructure to lazily clean up unused shadow memory. To avoid the difficulties around swapping mappings around, this code expects that the part of the shadow region that covers the vmalloc space will not be covered by the early shadow page, but will be left unmapped. This will require changes in arch-specific code. This allows KASAN with VMAP_STACK, and may be helpful for architectures that do not have a separate module space (e.g. powerpc64, which I am currently working on). It also allows relaxing the module alignment back to PAGE_SIZE. Testing with test_vmalloc.sh on an x86 VM with 2 vCPUs shows that: - Turning on KASAN, inline instrumentation, without vmalloc, introuduces a 4.1x-4.2x slowdown in vmalloc operations. - Turning this on introduces the following slowdowns over KASAN: * ~1.76x slower single-threaded (test_vmalloc.sh performance) * ~2.18x slower when both cpus are performing operations simultaneously (test_vmalloc.sh sequential_test_order=3D1) This is unfortunate but given that this is a debug feature only, not the end of the world. The full benchmark results are: Performance No KASAN KASAN original x baseline KASAN vmalloc x baseline x KASAN fix_size_alloc_test 662004 11404956 17.23 19144610 28.92 1.68 full_fit_alloc_test 710950 12029752 16.92 13184651 18.55 1.10 long_busy_list_alloc_test 9431875 43990172 4.66 82970178 8.80 1.89 random_size_alloc_test 5033626 23061762 4.58 47158834 9.37 2.04 fix_align_alloc_test 1252514 15276910 12.20 31266116 24.96 2.05 random_size_align_alloc_te 1648501 14578321 8.84 25560052 15.51 1.75 align_shift_alloc_test 147 830 5.65 5692 38.72 6.86 pcpu_alloc_test 80732 125520 1.55 140864 1.74 1.12 Total Cycles 119240774314 763211341128 6.40 1390338696894 11.66 1.82 Sequential, 2 cpus No KASAN KASAN original x baseline KASAN vmalloc x baseline x KASAN fix_size_alloc_test 1423150 14276550 10.03 27733022 19.49 1.94 full_fit_alloc_test 1754219 14722640 8.39 15030786 8.57 1.02 long_busy_list_alloc_test 11451858 52154973 4.55 107016027 9.34 2.05 random_size_alloc_test 5989020 26735276 4.46 68885923 11.50 2.58 fix_align_alloc_test 2050976 20166900 9.83 50491675 24.62 2.50 random_size_align_alloc_te 2858229 17971700 6.29 38730225 13.55 2.16 align_shift_alloc_test 405 6428 15.87 26253 64.82 4.08 pcpu_alloc_test 127183 151464 1.19 216263 1.70 1.43 Total Cycles 54181269392 308723699764 5.70 650772566394 12.01 2.11 fix_size_alloc_test 1420404 14289308 10.06 27790035 19.56 1.94 full_fit_alloc_test 1736145 14806234 8.53 15274301 8.80 1.03 long_busy_list_alloc_test 11404638 52270785 4.58 107550254 9.43 2.06 random_size_alloc_test 6017006 26650625 4.43 68696127 11.42 2.58 fix_align_alloc_test 2045504 20280985 9.91 50414862 24.65 2.49 random_size_align_alloc_te 2845338 17931018 6.30 38510276 13.53 2.15 align_shift_alloc_test 472 3760 7.97 9656 20.46 2.57 pcpu_alloc_test 118643 132732 1.12 146504 1.23 1.10 Total Cycles 54040011688 309102805492 5.72 651325675652 12.05 2.11 [dja@axtens.net: fixups] Link: http://lkml.kernel.org/r/20191120052719.7201-1-dja@axtens.net Link: https://bugzilla.kernel.org/show_bug.cgi?id=3D202009 Link: http://lkml.kernel.org/r/20191031093909.9228-2-dja@axtens.net Signed-off-by: Mark Rutland [shadow rework] Signed-off-by: Daniel Axtens Co-developed-by: Mark Rutland Acked-by: Vasily Gorbik Reviewed-by: Andrey Ryabinin Cc: Alexander Potapenko Cc: Dmitry Vyukov Cc: Christophe Leroy Cc: Qian Cai Signed-off-by: Andrew Morton --- Documentation/dev-tools/kasan.rst | 63 +++++++ include/linux/kasan.h | 31 +++ include/linux/moduleloader.h | 2 include/linux/vmalloc.h | 12 + lib/Kconfig.kasan | 16 + mm/kasan/common.c | 233 ++++++++++++++++++++++++++++ mm/kasan/generic_report.c | 3 mm/kasan/kasan.h | 1 mm/vmalloc.c | 56 +++++- 9 files changed, 408 insertions(+), 9 deletions(-) --- a/Documentation/dev-tools/kasan.rst~kasan-support-backing-vmalloc-space-with-real-shadow-memory +++ a/Documentation/dev-tools/kasan.rst @@ -218,3 +218,66 @@ brk handler is used to print bug reports A potential expansion of this mode is a hardware tag-based mode, which would use hardware memory tagging support instead of compiler instrumentation and manual shadow memory manipulation. + +What memory accesses are sanitised by KASAN? +-------------------------------------------- + +The kernel maps memory in a number of different parts of the address +space. This poses something of a problem for KASAN, which requires +that all addresses accessed by instrumented code have a valid shadow +region. + +The range of kernel virtual addresses is large: there is not enough +real memory to support a real shadow region for every address that +could be accessed by the kernel. + +By default +~~~~~~~~~~ + +By default, architectures only map real memory over the shadow region +for the linear mapping (and potentially other small areas). For all +other areas - such as vmalloc and vmemmap space - a single read-only +page is mapped over the shadow area. This read-only shadow page +declares all memory accesses as permitted. + +This presents a problem for modules: they do not live in the linear +mapping, but in a dedicated module space. By hooking in to the module +allocator, KASAN can temporarily map real shadow memory to cover +them. This allows detection of invalid accesses to module globals, for +example. + +This also creates an incompatibility with ``VMAP_STACK``: if the stack +lives in vmalloc space, it will be shadowed by the read-only page, and +the kernel will fault when trying to set up the shadow data for stack +variables. + +CONFIG_KASAN_VMALLOC +~~~~~~~~~~~~~~~~~~~~ + +With ``CONFIG_KASAN_VMALLOC``, KASAN can cover vmalloc space at the +cost of greater memory usage. Currently this is only supported on x86. + +This works by hooking into vmalloc and vmap, and dynamically +allocating real shadow memory to back the mappings. + +Most mappings in vmalloc space are small, requiring less than a full +page of shadow space. Allocating a full shadow page per mapping would +therefore be wasteful. Furthermore, to ensure that different mappings +use different shadow pages, mappings would have to be aligned to +``KASAN_SHADOW_SCALE_SIZE * PAGE_SIZE``. + +Instead, we share backing space across multiple mappings. We allocate +a backing page when a mapping in vmalloc space uses a particular page +of the shadow region. This page can be shared by other vmalloc +mappings later on. + +We hook in to the vmap infrastructure to lazily clean up unused shadow +memory. + +To avoid the difficulties around swapping mappings around, we expect +that the part of the shadow region that covers the vmalloc space will +not be covered by the early shadow page, but will be left +unmapped. This will require changes in arch-specific code. + +This allows ``VMAP_STACK`` support on x86, and can simplify support of +architectures that do not have a fixed module region. --- a/include/linux/kasan.h~kasan-support-backing-vmalloc-space-with-real-shadow-memory +++ a/include/linux/kasan.h @@ -70,8 +70,18 @@ struct kasan_cache { int free_meta_offset; }; +/* + * These functions provide a special case to support backing module + * allocations with real shadow memory. With KASAN vmalloc, the special + * case is unnecessary, as the work is handled in the generic case. + */ +#ifndef CONFIG_KASAN_VMALLOC int kasan_module_alloc(void *addr, size_t size); void kasan_free_shadow(const struct vm_struct *vm); +#else +static inline int kasan_module_alloc(void *addr, size_t size) { return 0; } +static inline void kasan_free_shadow(const struct vm_struct *vm) {} +#endif int kasan_add_zero_shadow(void *start, unsigned long size); void kasan_remove_zero_shadow(void *start, unsigned long size); @@ -194,4 +204,25 @@ static inline void *kasan_reset_tag(cons #endif /* CONFIG_KASAN_SW_TAGS */ +#ifdef CONFIG_KASAN_VMALLOC +int kasan_populate_vmalloc(unsigned long requested_size, + struct vm_struct *area); +void kasan_poison_vmalloc(void *start, unsigned long size); +void kasan_release_vmalloc(unsigned long start, unsigned long end, + unsigned long free_region_start, + unsigned long free_region_end); +#else +static inline int kasan_populate_vmalloc(unsigned long requested_size, + struct vm_struct *area) +{ + return 0; +} + +static inline void kasan_poison_vmalloc(void *start, unsigned long size) {} +static inline void kasan_release_vmalloc(unsigned long start, + unsigned long end, + unsigned long free_region_start, + unsigned long free_region_end) {} +#endif + #endif /* LINUX_KASAN_H */ --- a/include/linux/moduleloader.h~kasan-support-backing-vmalloc-space-with-real-shadow-memory +++ a/include/linux/moduleloader.h @@ -91,7 +91,7 @@ void module_arch_cleanup(struct module * /* Any cleanup before freeing mod->module_init */ void module_arch_freeing_init(struct module *mod); -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN) && !defined(CONFIG_KASAN_VMALLOC) #include #define MODULE_ALIGN (PAGE_SIZE << KASAN_SHADOW_SCALE_SHIFT) #else --- a/include/linux/vmalloc.h~kasan-support-backing-vmalloc-space-with-real-shadow-memory +++ a/include/linux/vmalloc.h @@ -22,6 +22,18 @@ struct notifier_block; /* in notifier.h #define VM_UNINITIALIZED 0x00000020 /* vm_struct is not fully initialized */ #define VM_NO_GUARD 0x00000040 /* don't add guard page */ #define VM_KASAN 0x00000080 /* has allocated kasan shadow memory */ + +/* + * VM_KASAN is used slighly differently depending on CONFIG_KASAN_VMALLOC. + * + * If IS_ENABLED(CONFIG_KASAN_VMALLOC), VM_KASAN is set on a vm_struct after + * shadow memory has been mapped. It's used to handle allocation errors so that + * we don't try to poision shadow on free if it was never allocated. + * + * Otherwise, VM_KASAN is set for kasan_module_alloc() allocations and used to + * determine which allocations need the module shadow freed. + */ + /* * Memory with VM_FLUSH_RESET_PERMS cannot be freed in an interrupt or with * vfree_atomic(). --- a/lib/Kconfig.kasan~kasan-support-backing-vmalloc-space-with-real-shadow-memory +++ a/lib/Kconfig.kasan @@ -6,6 +6,9 @@ config HAVE_ARCH_KASAN config HAVE_ARCH_KASAN_SW_TAGS bool +config HAVE_ARCH_KASAN_VMALLOC + bool + config CC_HAS_KASAN_GENERIC def_bool $(cc-option, -fsanitize=kernel-address) @@ -142,6 +145,19 @@ config KASAN_SW_TAGS_IDENTIFY (use-after-free or out-of-bounds) at the cost of increased memory consumption. +config KASAN_VMALLOC + bool "Back mappings in vmalloc space with real shadow memory" + depends on KASAN && HAVE_ARCH_KASAN_VMALLOC + help + By default, the shadow region for vmalloc space is the read-only + zero page. This means that KASAN cannot detect errors involving + vmalloc space. + + Enabling this option will hook in to vmap/vmalloc and back those + mappings with real shadow memory allocated on demand. This allows + for KASAN to detect more sorts of errors (and to support vmapped + stacks), but at the cost of higher memory usage. + config TEST_KASAN tristate "Module for testing KASAN for bug detection" depends on m && KASAN --- a/mm/kasan/common.c~kasan-support-backing-vmalloc-space-with-real-shadow-memory +++ a/mm/kasan/common.c @@ -36,6 +36,8 @@ #include #include +#include + #include "kasan.h" #include "../slab.h" @@ -590,6 +592,7 @@ void kasan_kfree_large(void *ptr, unsign /* The object will be poisoned by page_alloc. */ } +#ifndef CONFIG_KASAN_VMALLOC int kasan_module_alloc(void *addr, size_t size) { void *ret; @@ -625,6 +628,7 @@ void kasan_free_shadow(const struct vm_s if (vm->flags & VM_KASAN) vfree(kasan_mem_to_shadow(vm->addr)); } +#endif extern void __kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); @@ -744,3 +748,232 @@ static int __init kasan_memhotplug_init( core_initcall(kasan_memhotplug_init); #endif + +#ifdef CONFIG_KASAN_VMALLOC +static int kasan_populate_vmalloc_pte(pte_t *ptep, unsigned long addr, + void *unused) +{ + unsigned long page; + pte_t pte; + + if (likely(!pte_none(*ptep))) + return 0; + + page = __get_free_page(GFP_KERNEL); + if (!page) + return -ENOMEM; + + memset((void *)page, KASAN_VMALLOC_INVALID, PAGE_SIZE); + pte = pfn_pte(PFN_DOWN(__pa(page)), PAGE_KERNEL); + + spin_lock(&init_mm.page_table_lock); + if (likely(pte_none(*ptep))) { + set_pte_at(&init_mm, addr, ptep, pte); + page = 0; + } + spin_unlock(&init_mm.page_table_lock); + if (page) + free_page(page); + return 0; +} + +int kasan_populate_vmalloc(unsigned long requested_size, struct vm_struct *area) +{ + unsigned long shadow_start, shadow_end; + int ret; + + shadow_start = (unsigned long)kasan_mem_to_shadow(area->addr); + shadow_start = ALIGN_DOWN(shadow_start, PAGE_SIZE); + shadow_end = (unsigned long)kasan_mem_to_shadow(area->addr + + area->size); + shadow_end = ALIGN(shadow_end, PAGE_SIZE); + + ret = apply_to_page_range(&init_mm, shadow_start, + shadow_end - shadow_start, + kasan_populate_vmalloc_pte, NULL); + if (ret) + return ret; + + flush_cache_vmap(shadow_start, shadow_end); + + kasan_unpoison_shadow(area->addr, requested_size); + + area->flags |= VM_KASAN; + + /* + * We need to be careful about inter-cpu effects here. Consider: + * + * CPU#0 CPU#1 + * WRITE_ONCE(p, vmalloc(100)); while (x = READ_ONCE(p)) ; + * p[99] = 1; + * + * With compiler instrumentation, that ends up looking like this: + * + * CPU#0 CPU#1 + * // vmalloc() allocates memory + * // let a = area->addr + * // we reach kasan_populate_vmalloc + * // and call kasan_unpoison_shadow: + * STORE shadow(a), unpoison_val + * ... + * STORE shadow(a+99), unpoison_val x = LOAD p + * // rest of vmalloc process + * STORE p, a LOAD shadow(x+99) + * + * If there is no barrier between the end of unpoisioning the shadow + * and the store of the result to p, the stores could be committed + * in a different order by CPU#0, and CPU#1 could erroneously observe + * poison in the shadow. + * + * We need some sort of barrier between the stores. + * + * In the vmalloc() case, this is provided by a smp_wmb() in + * clear_vm_uninitialized_flag(). In the per-cpu allocator and in + * get_vm_area() and friends, the caller gets shadow allocated but + * doesn't have any pages mapped into the virtual address space that + * has been reserved. Mapping those pages in will involve taking and + * releasing a page-table lock, which will provide the barrier. + */ + + return 0; +} + +/* + * Poison the shadow for a vmalloc region. Called as part of the + * freeing process at the time the region is freed. + */ +void kasan_poison_vmalloc(void *start, unsigned long size) +{ + size = round_up(size, KASAN_SHADOW_SCALE_SIZE); + kasan_poison_shadow(start, size, KASAN_VMALLOC_INVALID); +} + +static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr, + void *unused) +{ + unsigned long page; + + page = (unsigned long)__va(pte_pfn(*ptep) << PAGE_SHIFT); + + spin_lock(&init_mm.page_table_lock); + + if (likely(!pte_none(*ptep))) { + pte_clear(&init_mm, addr, ptep); + free_page(page); + } + spin_unlock(&init_mm.page_table_lock); + + return 0; +} + +/* + * Release the backing for the vmalloc region [start, end), which + * lies within the free region [free_region_start, free_region_end). + * + * This can be run lazily, long after the region was freed. It runs + * under vmap_area_lock, so it's not safe to interact with the vmalloc/vmap + * infrastructure. + * + * How does this work? + * ------------------- + * + * We have a region that is page aligned, labelled as A. + * That might not map onto the shadow in a way that is page-aligned: + * + * start end + * v v + * |????????|????????|AAAAAAAA|AA....AA|AAAAAAAA|????????| < vmalloc + * -------- -------- -------- -------- -------- + * | | | | | + * | | | /-------/ | + * \-------\|/------/ |/---------------/ + * ||| || + * |??AAAAAA|AAAAAAAA|AA??????| < shadow + * (1) (2) (3) + * + * First we align the start upwards and the end downwards, so that the + * shadow of the region aligns with shadow page boundaries. In the + * example, this gives us the shadow page (2). This is the shadow entirely + * covered by this allocation. + * + * Then we have the tricky bits. We want to know if we can free the + * partially covered shadow pages - (1) and (3) in the example. For this, + * we are given the start and end of the free region that contains this + * allocation. Extending our previous example, we could have: + * + * free_region_start free_region_end + * | start end | + * v v v v + * |FFFFFFFF|FFFFFFFF|AAAAAAAA|AA....AA|AAAAAAAA|FFFFFFFF| < vmalloc + * -------- -------- -------- -------- -------- + * | | | | | + * | | | /-------/ | + * \-------\|/------/ |/---------------/ + * ||| || + * |FFAAAAAA|AAAAAAAA|AAF?????| < shadow + * (1) (2) (3) + * + * Once again, we align the start of the free region up, and the end of + * the free region down so that the shadow is page aligned. So we can free + * page (1) - we know no allocation currently uses anything in that page, + * because all of it is in the vmalloc free region. But we cannot free + * page (3), because we can't be sure that the rest of it is unused. + * + * We only consider pages that contain part of the original region for + * freeing: we don't try to free other pages from the free region or we'd + * end up trying to free huge chunks of virtual address space. + * + * Concurrency + * ----------- + * + * How do we know that we're not freeing a page that is simultaneously + * being used for a fresh allocation in kasan_populate_vmalloc(_pte)? + * + * We _can_ have kasan_release_vmalloc and kasan_populate_vmalloc running + * at the same time. While we run under free_vmap_area_lock, the population + * code does not. + * + * free_vmap_area_lock instead operates to ensure that the larger range + * [free_region_start, free_region_end) is safe: because __alloc_vmap_area and + * the per-cpu region-finding algorithm both run under free_vmap_area_lock, + * no space identified as free will become used while we are running. This + * means that so long as we are careful with alignment and only free shadow + * pages entirely covered by the free region, we will not run in to any + * trouble - any simultaneous allocations will be for disjoint regions. + */ +void kasan_release_vmalloc(unsigned long start, unsigned long end, + unsigned long free_region_start, + unsigned long free_region_end) +{ + void *shadow_start, *shadow_end; + unsigned long region_start, region_end; + + region_start = ALIGN(start, PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE); + region_end = ALIGN_DOWN(end, PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE); + + free_region_start = ALIGN(free_region_start, + PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE); + + if (start != region_start && + free_region_start < region_start) + region_start -= PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE; + + free_region_end = ALIGN_DOWN(free_region_end, + PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE); + + if (end != region_end && + free_region_end > region_end) + region_end += PAGE_SIZE * KASAN_SHADOW_SCALE_SIZE; + + shadow_start = kasan_mem_to_shadow((void *)region_start); + shadow_end = kasan_mem_to_shadow((void *)region_end); + + if (shadow_end > shadow_start) { + apply_to_page_range(&init_mm, (unsigned long)shadow_start, + (unsigned long)(shadow_end - shadow_start), + kasan_depopulate_vmalloc_pte, NULL); + flush_tlb_kernel_range((unsigned long)shadow_start, + (unsigned long)shadow_end); + } +} +#endif --- a/mm/kasan/generic_report.c~kasan-support-backing-vmalloc-space-with-real-shadow-memory +++ a/mm/kasan/generic_report.c @@ -86,6 +86,9 @@ static const char *get_shadow_bug_type(s case KASAN_ALLOCA_RIGHT: bug_type = "alloca-out-of-bounds"; break; + case KASAN_VMALLOC_INVALID: + bug_type = "vmalloc-out-of-bounds"; + break; } return bug_type; --- a/mm/kasan/kasan.h~kasan-support-backing-vmalloc-space-with-real-shadow-memory +++ a/mm/kasan/kasan.h @@ -25,6 +25,7 @@ #endif #define KASAN_GLOBAL_REDZONE 0xFA /* redzone for global variable */ +#define KASAN_VMALLOC_INVALID 0xF9 /* unallocated space in vmapped page */ /* * Stack redzone shadow values --- a/mm/vmalloc.c~kasan-support-backing-vmalloc-space-with-real-shadow-memory +++ a/mm/vmalloc.c @@ -683,7 +683,7 @@ insert_vmap_area_augment(struct vmap_are * free area is inserted. If VA has been merged, it is * freed. */ -static __always_inline void +static __always_inline struct vmap_area * merge_or_add_vmap_area(struct vmap_area *va, struct rb_root *root, struct list_head *head) { @@ -750,7 +750,10 @@ merge_or_add_vmap_area(struct vmap_area /* Free vmap_area object. */ kmem_cache_free(vmap_area_cachep, va); - return; + + /* Point to the new merged area. */ + va = sibling; + merged = true; } } @@ -759,6 +762,8 @@ insert: link_va(va, root, parent, link, head); augment_tree_propagate_from(va); } + + return va; } static __always_inline bool @@ -1196,8 +1201,7 @@ static void free_vmap_area(struct vmap_a * Insert/Merge it back to the free tree/list. */ spin_lock(&free_vmap_area_lock); - merge_or_add_vmap_area(va, - &free_vmap_area_root, &free_vmap_area_list); + merge_or_add_vmap_area(va, &free_vmap_area_root, &free_vmap_area_list); spin_unlock(&free_vmap_area_lock); } @@ -1294,14 +1298,20 @@ static bool __purge_vmap_area_lazy(unsig spin_lock(&free_vmap_area_lock); llist_for_each_entry_safe(va, n_va, valist, purge_list) { unsigned long nr = (va->va_end - va->va_start) >> PAGE_SHIFT; + unsigned long orig_start = va->va_start; + unsigned long orig_end = va->va_end; /* * Finally insert or merge lazily-freed area. It is * detached and there is no need to "unlink" it from * anything. */ - merge_or_add_vmap_area(va, - &free_vmap_area_root, &free_vmap_area_list); + va = merge_or_add_vmap_area(va, &free_vmap_area_root, + &free_vmap_area_list); + + if (is_vmalloc_or_module_addr((void *)orig_start)) + kasan_release_vmalloc(orig_start, orig_end, + va->va_start, va->va_end); atomic_long_sub(nr, &vmap_lazy_nr); @@ -2090,6 +2100,22 @@ static struct vm_struct *__get_vm_area_n setup_vmalloc_vm(area, va, flags, caller); + /* + * For KASAN, if we are in vmalloc space, we need to cover the shadow + * area with real memory. If we come here through VM_ALLOC, this is + * done by a higher level function that has access to the true size, + * which might not be a full page. + * + * We assume module space comes via VM_ALLOC path. + */ + if (is_vmalloc_addr(area->addr) && !(area->flags & VM_ALLOC)) { + if (kasan_populate_vmalloc(area->size, area)) { + unmap_vmap_area(va); + kfree(area); + return NULL; + } + } + return area; } @@ -2267,6 +2293,9 @@ static void __vunmap(const void *addr, i debug_check_no_locks_freed(area->addr, get_vm_area_size(area)); debug_check_no_obj_freed(area->addr, get_vm_area_size(area)); + if (area->flags & VM_KASAN) + kasan_poison_vmalloc(area->addr, area->size); + vm_remove_mappings(area, deallocate_pages); if (deallocate_pages) { @@ -2519,6 +2548,11 @@ void *__vmalloc_node_range(unsigned long if (!addr) return NULL; + if (is_vmalloc_or_module_addr(area->addr)) { + if (kasan_populate_vmalloc(real_size, area)) + return NULL; + } + /* * In this function, newly allocated vm_struct has VM_UNINITIALIZED * flag. It means that vm_struct is not fully initialized. @@ -3400,6 +3434,12 @@ retry: } spin_unlock(&vmap_area_lock); + /* populate the shadow space outside of the lock */ + for (area = 0; area < nr_vms; area++) { + /* assume success here */ + kasan_populate_vmalloc(sizes[area], vms[area]); + } + kfree(vas); return vms; @@ -3411,8 +3451,8 @@ recovery: * and when pcpu_get_vm_areas() is success. */ while (area--) { - merge_or_add_vmap_area(vas[area], - &free_vmap_area_root, &free_vmap_area_list); + merge_or_add_vmap_area(vas[area], &free_vmap_area_root, + &free_vmap_area_list); vas[area] = NULL; } From patchwork Sun Dec 1 01:54:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268365 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 12E3E921 for ; Sun, 1 Dec 2019 01:54:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D4707206E1 for ; Sun, 1 Dec 2019 01:54:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="01Jo+efR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D4707206E1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 35F346B0313; Sat, 30 Nov 2019 20:54:56 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2E8706B0318; Sat, 30 Nov 2019 20:54:56 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1B3326B0319; Sat, 30 Nov 2019 20:54:56 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0204.hostedemail.com [216.40.44.204]) by kanga.kvack.org (Postfix) with ESMTP id E4EC26B0313 for ; Sat, 30 Nov 2019 20:54:55 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 9B1522A87 for ; Sun, 1 Dec 2019 01:54:55 +0000 (UTC) X-FDA: 76214904150.15.sleet04_2afbd2379a32a X-Spam-Summary: 2,0,0,1fe31cdc7831a6a3,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:aryabinin@virtuozzo.com:christophe.leroy@c-s.fr:dja@axtens.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com::mark.rutland@arm.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:5007:6261:6653:6737:7576:7903:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12783:12986:13069:13311:13357:13846:14181:14384:14721:14849:21080:21451:21627:21939:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Cust om_rules X-HE-Tag: sleet04_2afbd2379a32a X-Filterd-Recvd-Size: 3051 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:55 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 51A3020880; Sun, 1 Dec 2019 01:54:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165294; bh=vgxK+npNQhmQ19/EAy9lp8EOiwns+9NoJTKRZ63mP70=; h=Date:From:To:Subject:From; b=01Jo+efR7X4IkBdJv6xxwmzaMEN0CcRLxuBBeTdCfRjSM9v9pLwW96zXafKFFXslP Rh6JCfmpOrvypoosH4UIins+NjtPexky4b+pgO6biAgwXRh8QuUH55tvhLSscqZIMD M3iG56FijDAMhp2RQuJk4GhOTJRBep6zecWWVP2Y= Date: Sat, 30 Nov 2019 17:54:53 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, aryabinin@virtuozzo.com, christophe.leroy@c-s.fr, dja@axtens.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, linux-mm@kvack.org, mark.rutland@arm.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 094/158] kasan: add test for vmalloc Message-ID: <20191201015453.iMLeakr_q%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Daniel Axtens Subject: kasan: add test for vmalloc Test kasan vmalloc support by adding a new test to the module. Link: http://lkml.kernel.org/r/20191031093909.9228-3-dja@axtens.net Signed-off-by: Daniel Axtens Reviewed-by: Andrey Ryabinin Cc: Alexander Potapenko Cc: Christophe Leroy Cc: Dmitry Vyukov Cc: Mark Rutland Cc: Vasily Gorbik Signed-off-by: Andrew Morton --- lib/test_kasan.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) --- a/lib/test_kasan.c~kasan-add-test-for-vmalloc +++ a/lib/test_kasan.c @@ -19,6 +19,7 @@ #include #include #include +#include #include @@ -748,6 +749,30 @@ static noinline void __init kmalloc_doub kzfree(ptr); } +#ifdef CONFIG_KASAN_VMALLOC +static noinline void __init vmalloc_oob(void) +{ + void *area; + + pr_info("vmalloc out-of-bounds\n"); + + /* + * We have to be careful not to hit the guard page. + * The MMU will catch that and crash us. + */ + area = vmalloc(3000); + if (!area) { + pr_err("Allocation failed\n"); + return; + } + + ((volatile char *)area)[3100]; + vfree(area); +} +#else +static void __init vmalloc_oob(void) {} +#endif + static int __init kmalloc_tests_init(void) { /* @@ -793,6 +818,7 @@ static int __init kmalloc_tests_init(voi kasan_strings(); kasan_bitops(); kmalloc_double_kzfree(); + vmalloc_oob(); kasan_restore_multi_shot(multishot); From patchwork Sun Dec 1 01:54:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268367 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F08A6112B for ; Sun, 1 Dec 2019 01:55:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B4B25206E1 for ; Sun, 1 Dec 2019 01:55:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="Nashcs7F" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B4B25206E1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 66C116B0318; Sat, 30 Nov 2019 20:54:59 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5F7A86B031A; Sat, 30 Nov 2019 20:54:59 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4E6196B031B; Sat, 30 Nov 2019 20:54:59 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0146.hostedemail.com [216.40.44.146]) by kanga.kvack.org (Postfix) with ESMTP id 33B4F6B0318 for ; Sat, 30 Nov 2019 20:54:59 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id DAC41180AD817 for ; Sun, 1 Dec 2019 01:54:58 +0000 (UTC) X-FDA: 76214904276.03.pie47_2b730646e4456 X-Spam-Summary: 2,0,0,33e21d77b93300e5,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:aryabinin@virtuozzo.com:christophe.leroy@c-s.fr:dja@axtens.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com::mark.rutland@arm.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2689:2859:2898:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:3873:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4470:5007:6261:6653:6737:7576:7875:8599:8784:9025:9545:10004:10913:11026:11232:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13846:14093:14181:14721:14849:14877:21063:21080:21451:21627:21939:30054:30070:30091,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,D NSBL:neu X-HE-Tag: pie47_2b730646e4456 X-Filterd-Recvd-Size: 3740 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:54:58 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 753B4208C3; Sun, 1 Dec 2019 01:54:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165297; bh=jgbbAX+heAdcowRF+vVgj0a8bmik7GM/njjLpxxA8wA=; h=Date:From:To:Subject:From; b=Nashcs7FlUqDFKm/dorbC4ZPuhKVK/lOPbQEq3uwmisvHyhMsCyknRBvzCXbcNvFP V4471ZbH7SMHwMmUzGqdS8O2vC7gwrb9iYyLWeDtgRbh68XgNk390nb2yTXN6/8J+x ab0WPfTZ7RdoTZl+VJNIwb0LNGDkRJ3Aov+E14zo= Date: Sat, 30 Nov 2019 17:54:57 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, aryabinin@virtuozzo.com, christophe.leroy@c-s.fr, dja@axtens.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, linux-mm@kvack.org, mark.rutland@arm.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 095/158] fork: support VMAP_STACK with KASAN_VMALLOC Message-ID: <20191201015457.mzKZyzliP%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Daniel Axtens Subject: fork: support VMAP_STACK with KASAN_VMALLOC Supporting VMAP_STACK with KASAN_VMALLOC is straightforward: - clear the shadow region of vmapped stacks when swapping them in - tweak Kconfig to allow VMAP_STACK to be turned on with KASAN Link: http://lkml.kernel.org/r/20191031093909.9228-4-dja@axtens.net Signed-off-by: Daniel Axtens Reviewed-by: Dmitry Vyukov Reviewed-by: Andrey Ryabinin Cc: Alexander Potapenko Cc: Christophe Leroy Cc: Mark Rutland Cc: Vasily Gorbik Signed-off-by: Andrew Morton --- arch/Kconfig | 9 +++++---- kernel/fork.c | 4 ++++ 2 files changed, 9 insertions(+), 4 deletions(-) --- a/arch/Kconfig~fork-support-vmap_stack-with-kasan_vmalloc +++ a/arch/Kconfig @@ -843,16 +843,17 @@ config HAVE_ARCH_VMAP_STACK config VMAP_STACK default y bool "Use a virtually-mapped stack" - depends on HAVE_ARCH_VMAP_STACK && !KASAN + depends on HAVE_ARCH_VMAP_STACK + depends on !KASAN || KASAN_VMALLOC ---help--- Enable this if you want the use virtually-mapped kernel stacks with guard pages. This causes kernel stack overflows to be caught immediately rather than causing difficult-to-diagnose corruption. - This is presently incompatible with KASAN because KASAN expects - the stack to map directly to the KASAN shadow map using a formula - that is incorrect if the stack is in vmalloc space. + To use this with KASAN, the architecture must support backing + virtual mappings with real shadow memory, and KASAN_VMALLOC must + be enabled. config ARCH_OPTIONAL_KERNEL_RWX def_bool n --- a/kernel/fork.c~fork-support-vmap_stack-with-kasan_vmalloc +++ a/kernel/fork.c @@ -93,6 +93,7 @@ #include #include #include +#include #include #include @@ -223,6 +224,9 @@ static unsigned long *alloc_thread_stack if (!s) continue; + /* Clear the KASAN shadow of the stack. */ + kasan_unpoison_shadow(s->addr, THREAD_SIZE); + /* Clear stale pointers from reused stack. */ memset(s->addr, 0, THREAD_SIZE); From patchwork Sun Dec 1 01:55:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268369 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9D18C112B for ; Sun, 1 Dec 2019 01:55:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 606AF208C3 for ; Sun, 1 Dec 2019 01:55:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="XPFPsN8K" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 606AF208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 841746B031A; Sat, 30 Nov 2019 20:55:05 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7CAC36B031C; Sat, 30 Nov 2019 20:55:05 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 694EC6B031D; Sat, 30 Nov 2019 20:55:05 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 4DDAB6B031A for ; Sat, 30 Nov 2019 20:55:05 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 000652C7C for ; Sun, 1 Dec 2019 01:55:04 +0000 (UTC) X-FDA: 76214904570.14.chess19_2c544c2340519 X-Spam-Summary: 2,0,0,5f841c57ab8cae2b,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:aryabinin@virtuozzo.com:christophe.leroy@c-s.fr:dja@axtens.net:dvyukov@google.com:glider@google.com:gor@linux.ibm.com::mark.rutland@arm.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1535:1543:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4605:5007:6119:6261:6653:6737:7576:8599:8784:9025:9545:10004:10913:11026:11233:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13141:13230:13846:14096:14181:14721:14849:21080:21451:21627:21939:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutr al,Custo X-HE-Tag: chess19_2c544c2340519 X-Filterd-Recvd-Size: 5053 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:04 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B5B0E20880; Sun, 1 Dec 2019 01:55:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165303; bh=vId6BvL2gsVrDejxmnEJ+wki9ykjb9Ss5DsSTaaaZWw=; h=Date:From:To:Subject:From; b=XPFPsN8KVRHZ9IUYdwezEteKi1+IR0GgM5Es1qsefQX1/7swQobz1UDYb5bQFCff0 LE6lTtwkaio9WrzbZYrswjfeQDsEvbDJ2Aegvli0KPcaHJLAMNNV55clUJsypxlru1 BW8C6LGRKZ5anx35QvpI/FPcQ7nG3rGhMa6dHJ70= Date: Sat, 30 Nov 2019 17:55:00 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, aryabinin@virtuozzo.com, christophe.leroy@c-s.fr, dja@axtens.net, dvyukov@google.com, glider@google.com, gor@linux.ibm.com, linux-mm@kvack.org, mark.rutland@arm.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 096/158] x86/kasan: support KASAN_VMALLOC Message-ID: <20191201015500.YucT296fu%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Daniel Axtens Subject: x86/kasan: support KASAN_VMALLOC In the case where KASAN directly allocates memory to back vmalloc space, don't map the early shadow page over it. We prepopulate pgds/p4ds for the range that would otherwise be empty. This is required to get it synced to hardware on boot, allowing the lower levels of the page tables to be filled dynamically. Link: http://lkml.kernel.org/r/20191031093909.9228-5-dja@axtens.net Signed-off-by: Daniel Axtens Acked-by: Dmitry Vyukov Reviewed-by: Andrey Ryabinin Cc: Alexander Potapenko Cc: Christophe Leroy Cc: Mark Rutland Cc: Vasily Gorbik Signed-off-by: Andrew Morton --- arch/x86/Kconfig | 1 arch/x86/mm/kasan_init_64.c | 61 ++++++++++++++++++++++++++++++++++ 2 files changed, 62 insertions(+) --- a/arch/x86/Kconfig~x86-kasan-support-kasan_vmalloc +++ a/arch/x86/Kconfig @@ -135,6 +135,7 @@ config X86 select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if X86_64 + select HAVE_ARCH_KASAN_VMALLOC if X86_64 select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS if MMU select HAVE_ARCH_MMAP_RND_COMPAT_BITS if MMU && COMPAT --- a/arch/x86/mm/kasan_init_64.c~x86-kasan-support-kasan_vmalloc +++ a/arch/x86/mm/kasan_init_64.c @@ -245,6 +245,49 @@ static void __init kasan_map_early_shado } while (pgd++, addr = next, addr != end); } +static void __init kasan_shallow_populate_p4ds(pgd_t *pgd, + unsigned long addr, + unsigned long end) +{ + p4d_t *p4d; + unsigned long next; + void *p; + + p4d = p4d_offset(pgd, addr); + do { + next = p4d_addr_end(addr, end); + + if (p4d_none(*p4d)) { + p = early_alloc(PAGE_SIZE, NUMA_NO_NODE, true); + p4d_populate(&init_mm, p4d, p); + } + } while (p4d++, addr = next, addr != end); +} + +static void __init kasan_shallow_populate_pgds(void *start, void *end) +{ + unsigned long addr, next; + pgd_t *pgd; + void *p; + + addr = (unsigned long)start; + pgd = pgd_offset_k(addr); + do { + next = pgd_addr_end(addr, (unsigned long)end); + + if (pgd_none(*pgd)) { + p = early_alloc(PAGE_SIZE, NUMA_NO_NODE, true); + pgd_populate(&init_mm, pgd, p); + } + + /* + * we need to populate p4ds to be synced when running in + * four level mode - see sync_global_pgds_l4() + */ + kasan_shallow_populate_p4ds(pgd, addr, next); + } while (pgd++, addr = next, addr != (unsigned long)end); +} + #ifdef CONFIG_KASAN_INLINE static int kasan_die_handler(struct notifier_block *self, unsigned long val, @@ -354,6 +397,24 @@ void __init kasan_init(void) kasan_populate_early_shadow( kasan_mem_to_shadow((void *)PAGE_OFFSET + MAXMEM), + kasan_mem_to_shadow((void *)VMALLOC_START)); + + /* + * If we're in full vmalloc mode, don't back vmalloc space with early + * shadow pages. Instead, prepopulate pgds/p4ds so they are synced to + * the global table and we can populate the lower levels on demand. + */ + if (IS_ENABLED(CONFIG_KASAN_VMALLOC)) + kasan_shallow_populate_pgds( + kasan_mem_to_shadow((void *)VMALLOC_START), + kasan_mem_to_shadow((void *)VMALLOC_END)); + else + kasan_populate_early_shadow( + kasan_mem_to_shadow((void *)VMALLOC_START), + kasan_mem_to_shadow((void *)VMALLOC_END)); + + kasan_populate_early_shadow( + kasan_mem_to_shadow((void *)VMALLOC_END + 1), shadow_cpu_entry_begin); kasan_populate_shadow((unsigned long)shadow_cpu_entry_begin, From patchwork Sun Dec 1 01:55:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268371 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3AE90921 for ; Sun, 1 Dec 2019 01:55:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E12502084E for ; Sun, 1 Dec 2019 01:55:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="ExVX+EEv" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E12502084E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ED0AE6B031C; Sat, 30 Nov 2019 20:55:10 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E828A6B031E; Sat, 30 Nov 2019 20:55:10 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D49946B031F; Sat, 30 Nov 2019 20:55:10 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0026.hostedemail.com [216.40.44.26]) by kanga.kvack.org (Postfix) with ESMTP id B56A26B031C for ; Sat, 30 Nov 2019 20:55:10 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 6522D824999B for ; Sun, 1 Dec 2019 01:55:10 +0000 (UTC) X-FDA: 76214904780.16.mint96_2d1cbefec5802 X-Spam-Summary: 2,0,0,c9e9ee1a9f13d78e,d41d8cd98f00b204,akpm@linux-foundation.org,:aarcange@redhat.com:akpm@linux-foundation.org:anshuman.khandual@arm.com:dan.j.williams@intel.com:david@redhat.com::mgorman@techsingularity.net:mhocko@suse.com:mike.kravetz@oracle.com:mm-commits@vger.kernel.org:osalvador@suse.de:pavel.tatashin@microsoft.com:rientjes@google.com:rppt@linux.ibm.com:torvalds@linux-foundation.org:vbabka@suse.cz:willy@infradead.org,RULES_HIT:1:2:41:69:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1605:1730:1747:1777:1792:1963:1981:2194:2196:2198:2199:2200:2201:2393:2525:2553:2559:2563:2682:2685:2693:2731:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4051:4250:4321:4385:4605:5007:6119:6261:6630:6653:6737:7576:7688:7875:7903:8957:9025:9391:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12114:12291:12295:12296:12297:12438:1 2517:125 X-HE-Tag: mint96_2d1cbefec5802 X-Filterd-Recvd-Size: 10147 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:09 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 52837206E1; Sun, 1 Dec 2019 01:55:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165309; bh=lgaF4lRPrwq/2ZIvQoTZucsr0H2Llw88rAMnLY4+xJk=; h=Date:From:To:Subject:From; b=ExVX+EEvR+UAPDFSceHOFHqccjSq+Xhr/Kr4BKXWP4YaOxC8vdC4d3KfDCjAI3GxJ uoGSqP4Yqy6quxQe6XQw1VGlMc+9Ic3VIO7OqudkSYlkqVv7ZYmkAWYTSiKQB3HPNU 9ZXvjPVnhaMIRYb/e9X2R98ETkEuHyUlIEcT5Lto= Date: Sat, 30 Nov 2019 17:55:06 -0800 From: akpm@linux-foundation.org To: aarcange@redhat.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, dan.j.williams@intel.com, david@redhat.com, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, osalvador@suse.de, pavel.tatashin@microsoft.com, rientjes@google.com, rppt@linux.ibm.com, torvalds@linux-foundation.org, vbabka@suse.cz, willy@infradead.org Subject: [patch 097/158] mm/page_alloc: add alloc_contig_pages() Message-ID: <20191201015506.7d9vdT2Lu%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Anshuman Khandual Subject: mm/page_alloc: add alloc_contig_pages() HugeTLB helper alloc_gigantic_page() implements fairly generic allocation method where it scans over various zones looking for a large contiguous pfn range before trying to allocate it with alloc_contig_range(). Other than deriving the requested order from 'struct hstate', there is nothing HugeTLB specific in there. This can be made available for general use to allocate contiguous memory which could not have been allocated through the buddy allocator. alloc_gigantic_page() has been split carving out actual allocation method which is then made available via new alloc_contig_pages() helper wrapped under CONFIG_CONTIG_ALLOC. All references to 'gigantic' have been replaced with more generic term 'contig'. Allocated pages here should be freed with free_contig_range() or by calling __free_page() on each allocated page. Link: http://lkml.kernel.org/r/1571300646-32240-1-git-send-email-anshuman.khandual@arm.com Signed-off-by: Anshuman Khandual Acked-by: David Hildenbrand Acked-by: Michal Hocko Cc: Mike Kravetz Cc: Vlastimil Babka Cc: Michal Hocko Cc: David Rientjes Cc: Andrea Arcangeli Cc: Oscar Salvador Cc: Mel Gorman Cc: Mike Rapoport Cc: Dan Williams Cc: Pavel Tatashin Cc: Matthew Wilcox Cc: David Hildenbrand Signed-off-by: Andrew Morton --- include/linux/gfp.h | 2 mm/hugetlb.c | 77 -------------------------------- mm/page_alloc.c | 101 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 105 insertions(+), 75 deletions(-) --- a/include/linux/gfp.h~mm-page_alloc-add-alloc_contig_pages +++ a/include/linux/gfp.h @@ -612,6 +612,8 @@ static inline bool pm_suspended_storage( /* The below functions must be run on a range from a single zone. */ extern int alloc_contig_range(unsigned long start, unsigned long end, unsigned migratetype, gfp_t gfp_mask); +extern struct page *alloc_contig_pages(unsigned long nr_pages, gfp_t gfp_mask, + int nid, nodemask_t *nodemask); #endif void free_contig_range(unsigned long pfn, unsigned int nr_pages); --- a/mm/hugetlb.c~mm-page_alloc-add-alloc_contig_pages +++ a/mm/hugetlb.c @@ -1069,85 +1069,12 @@ static void free_gigantic_page(struct pa } #ifdef CONFIG_CONTIG_ALLOC -static int __alloc_gigantic_page(unsigned long start_pfn, - unsigned long nr_pages, gfp_t gfp_mask) -{ - unsigned long end_pfn = start_pfn + nr_pages; - return alloc_contig_range(start_pfn, end_pfn, MIGRATE_MOVABLE, - gfp_mask); -} - -static bool pfn_range_valid_gigantic(struct zone *z, - unsigned long start_pfn, unsigned long nr_pages) -{ - unsigned long i, end_pfn = start_pfn + nr_pages; - struct page *page; - - for (i = start_pfn; i < end_pfn; i++) { - page = pfn_to_online_page(i); - if (!page) - return false; - - if (page_zone(page) != z) - return false; - - if (PageReserved(page)) - return false; - - if (page_count(page) > 0) - return false; - - if (PageHuge(page)) - return false; - } - - return true; -} - -static bool zone_spans_last_pfn(const struct zone *zone, - unsigned long start_pfn, unsigned long nr_pages) -{ - unsigned long last_pfn = start_pfn + nr_pages - 1; - return zone_spans_pfn(zone, last_pfn); -} - static struct page *alloc_gigantic_page(struct hstate *h, gfp_t gfp_mask, int nid, nodemask_t *nodemask) { - unsigned int order = huge_page_order(h); - unsigned long nr_pages = 1 << order; - unsigned long ret, pfn, flags; - struct zonelist *zonelist; - struct zone *zone; - struct zoneref *z; - - zonelist = node_zonelist(nid, gfp_mask); - for_each_zone_zonelist_nodemask(zone, z, zonelist, gfp_zone(gfp_mask), nodemask) { - spin_lock_irqsave(&zone->lock, flags); - - pfn = ALIGN(zone->zone_start_pfn, nr_pages); - while (zone_spans_last_pfn(zone, pfn, nr_pages)) { - if (pfn_range_valid_gigantic(zone, pfn, nr_pages)) { - /* - * We release the zone lock here because - * alloc_contig_range() will also lock the zone - * at some point. If there's an allocation - * spinning on this lock, it may win the race - * and cause alloc_contig_range() to fail... - */ - spin_unlock_irqrestore(&zone->lock, flags); - ret = __alloc_gigantic_page(pfn, nr_pages, gfp_mask); - if (!ret) - return pfn_to_page(pfn); - spin_lock_irqsave(&zone->lock, flags); - } - pfn += nr_pages; - } - - spin_unlock_irqrestore(&zone->lock, flags); - } + unsigned long nr_pages = 1UL << huge_page_order(h); - return NULL; + return alloc_contig_pages(nr_pages, gfp_mask, nid, nodemask); } static void prep_new_huge_page(struct hstate *h, struct page *page, int nid); --- a/mm/page_alloc.c~mm-page_alloc-add-alloc_contig_pages +++ a/mm/page_alloc.c @@ -8502,6 +8502,107 @@ done: pfn_max_align_up(end), migratetype); return ret; } + +static int __alloc_contig_pages(unsigned long start_pfn, + unsigned long nr_pages, gfp_t gfp_mask) +{ + unsigned long end_pfn = start_pfn + nr_pages; + + return alloc_contig_range(start_pfn, end_pfn, MIGRATE_MOVABLE, + gfp_mask); +} + +static bool pfn_range_valid_contig(struct zone *z, unsigned long start_pfn, + unsigned long nr_pages) +{ + unsigned long i, end_pfn = start_pfn + nr_pages; + struct page *page; + + for (i = start_pfn; i < end_pfn; i++) { + page = pfn_to_online_page(i); + if (!page) + return false; + + if (page_zone(page) != z) + return false; + + if (PageReserved(page)) + return false; + + if (page_count(page) > 0) + return false; + + if (PageHuge(page)) + return false; + } + return true; +} + +static bool zone_spans_last_pfn(const struct zone *zone, + unsigned long start_pfn, unsigned long nr_pages) +{ + unsigned long last_pfn = start_pfn + nr_pages - 1; + + return zone_spans_pfn(zone, last_pfn); +} + +/** + * alloc_contig_pages() -- tries to find and allocate contiguous range of pages + * @nr_pages: Number of contiguous pages to allocate + * @gfp_mask: GFP mask to limit search and used during compaction + * @nid: Target node + * @nodemask: Mask for other possible nodes + * + * This routine is a wrapper around alloc_contig_range(). It scans over zones + * on an applicable zonelist to find a contiguous pfn range which can then be + * tried for allocation with alloc_contig_range(). This routine is intended + * for allocation requests which can not be fulfilled with the buddy allocator. + * + * The allocated memory is always aligned to a page boundary. If nr_pages is a + * power of two then the alignment is guaranteed to be to the given nr_pages + * (e.g. 1GB request would be aligned to 1GB). + * + * Allocated pages can be freed with free_contig_range() or by manually calling + * __free_page() on each allocated page. + * + * Return: pointer to contiguous pages on success, or NULL if not successful. + */ +struct page *alloc_contig_pages(unsigned long nr_pages, gfp_t gfp_mask, + int nid, nodemask_t *nodemask) +{ + unsigned long ret, pfn, flags; + struct zonelist *zonelist; + struct zone *zone; + struct zoneref *z; + + zonelist = node_zonelist(nid, gfp_mask); + for_each_zone_zonelist_nodemask(zone, z, zonelist, + gfp_zone(gfp_mask), nodemask) { + spin_lock_irqsave(&zone->lock, flags); + + pfn = ALIGN(zone->zone_start_pfn, nr_pages); + while (zone_spans_last_pfn(zone, pfn, nr_pages)) { + if (pfn_range_valid_contig(zone, pfn, nr_pages)) { + /* + * We release the zone lock here because + * alloc_contig_range() will also lock the zone + * at some point. If there's an allocation + * spinning on this lock, it may win the race + * and cause alloc_contig_range() to fail... + */ + spin_unlock_irqrestore(&zone->lock, flags); + ret = __alloc_contig_pages(pfn, nr_pages, + gfp_mask); + if (!ret) + return pfn_to_page(pfn); + spin_lock_irqsave(&zone->lock, flags); + } + pfn += nr_pages; + } + spin_unlock_irqrestore(&zone->lock, flags); + } + return NULL; +} #endif /* CONFIG_CONTIG_ALLOC */ void free_contig_range(unsigned long pfn, unsigned int nr_pages) From patchwork Sun Dec 1 01:55:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268373 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9A059921 for ; Sun, 1 Dec 2019 01:55:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6649420880 for ; Sun, 1 Dec 2019 01:55:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="t9MIOlkO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6649420880 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5839C6B031E; Sat, 30 Nov 2019 20:55:14 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 50AD56B0320; Sat, 30 Nov 2019 20:55:14 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3FA4A6B0321; Sat, 30 Nov 2019 20:55:14 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0163.hostedemail.com [216.40.44.163]) by kanga.kvack.org (Postfix) with ESMTP id 24A766B031E for ; Sat, 30 Nov 2019 20:55:14 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id CAA3E248E for ; Sun, 1 Dec 2019 01:55:13 +0000 (UTC) X-FDA: 76214904906.01.patch93_2d9e0d7b02647 X-Spam-Summary: 2,0,0,899337d97eea5003,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:bp@alien8.de:cai@lca.pw::matt@codeblueprint.co.uk:mgorman@techsingularity.net:mhocko@suse.com:mm-commits@vger.kernel.org:tglx@linutronix.de:torvalds@linux-foundation.org:vbabka@suse.cz,RULES_HIT:41:69:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3167:3353:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:5007:6119:6261:6653:6737:7576:7875:8599:9025:9545:9592:10004:10913:11026:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12783:12986:13255:13846:14096:14181:14721:14849:21080:21222:21451:21627:21939:30012:30054:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neu tral,Cus X-HE-Tag: patch93_2d9e0d7b02647 X-Filterd-Recvd-Size: 3647 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:13 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 53B432084E; Sun, 1 Dec 2019 01:55:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165312; bh=hsDpJyt7+dgi5OalEtoJXJJK+NycPljmJOktHoCLpZY=; h=Date:From:To:Subject:From; b=t9MIOlkOL2cSQEK+vF3+IlYq9khSjDZwhkWS7CGE7VO/d7lONJL3AEy8oWnDCvneF Jke48GoanNrXKD7puGsoIEwlC/eGmiT4ygjqQk5doK+S5YmF8bpfrp0FJq0RvwBP/i nMC8RsymGXoy/PJFvCWRSOU0SvEf1Ul500KB6EzM= Date: Sat, 30 Nov 2019 17:55:11 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, bp@alien8.de, cai@lca.pw, linux-mm@kvack.org, matt@codeblueprint.co.uk, mgorman@techsingularity.net, mhocko@suse.com, mm-commits@vger.kernel.org, tglx@linutronix.de, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 098/158] mm, pcp: share common code between memory hotplug and percpu sysctl handler Message-ID: <20191201015511.bVmlGRd2K%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Subject: mm, pcp: share common code between memory hotplug and percpu sysctl handler Both the percpu_pagelist_fraction sysctl handler and memory hotplug have a common requirement of updating the pcpu page allocation batch and high values. Split the relevant helper to share common code. No functional change. Link: http://lkml.kernel.org/r/20191021094808.28824-3-mgorman@techsingularity.net Signed-off-by: Mel Gorman Acked-by: Michal Hocko Acked-by: Vlastimil Babka Cc: Borislav Petkov Cc: Matt Fleming Cc: Qian Cai Cc: Thomas Gleixner Signed-off-by: Andrew Morton --- mm/page_alloc.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) --- a/mm/page_alloc.c~mm-pcp-share-common-code-between-memory-hotplug-and-percpu-sysctl-handler +++ a/mm/page_alloc.c @@ -7988,6 +7988,15 @@ int lowmem_reserve_ratio_sysctl_handler( return 0; } +static void __zone_pcp_update(struct zone *zone) +{ + unsigned int cpu; + + for_each_possible_cpu(cpu) + pageset_set_high_and_batch(zone, + per_cpu_ptr(zone->pageset, cpu)); +} + /* * percpu_pagelist_fraction - changes the pcp->high for each zone on each * cpu. It is the fraction of total pages in each zone that a hot per cpu @@ -8019,13 +8028,8 @@ int percpu_pagelist_fraction_sysctl_hand if (percpu_pagelist_fraction == old_percpu_pagelist_fraction) goto out; - for_each_populated_zone(zone) { - unsigned int cpu; - - for_each_possible_cpu(cpu) - pageset_set_high_and_batch(zone, - per_cpu_ptr(zone->pageset, cpu)); - } + for_each_populated_zone(zone) + __zone_pcp_update(zone); out: mutex_unlock(&pcp_batch_high_lock); return ret; @@ -8624,11 +8628,8 @@ void free_contig_range(unsigned long pfn */ void __meminit zone_pcp_update(struct zone *zone) { - unsigned cpu; mutex_lock(&pcp_batch_high_lock); - for_each_possible_cpu(cpu) - pageset_set_high_and_batch(zone, - per_cpu_ptr(zone->pageset, cpu)); + __zone_pcp_update(zone); mutex_unlock(&pcp_batch_high_lock); } From patchwork Sun Dec 1 01:55:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268375 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CC68C112B for ; Sun, 1 Dec 2019 01:55:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 945272084D for ; Sun, 1 Dec 2019 01:55:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="F1CaiQB0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 945272084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7CC036B0320; Sat, 30 Nov 2019 20:55:17 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 72E496B0322; Sat, 30 Nov 2019 20:55:17 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5CFCF6B0323; Sat, 30 Nov 2019 20:55:17 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0136.hostedemail.com [216.40.44.136]) by kanga.kvack.org (Postfix) with ESMTP id 428806B0320 for ; Sat, 30 Nov 2019 20:55:17 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 007A1181AEF0B for ; Sun, 1 Dec 2019 01:55:17 +0000 (UTC) X-FDA: 76214905074.13.name59_2e16e0c2cd009 X-Spam-Summary: 2,0,0,3c0e9c31927142ae,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:bp@alien8.de:cai@lca.pw::matt@codeblueprint.co.uk:mgorman@techsingularity.net:mhocko@suse.com:mm-commits@vger.kernel.org:tglx@linutronix.de:torvalds@linux-foundation.org:vbabka@suse.cz,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3167:3352:3865:3867:3868:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4605:5007:6119:6120:6261:6653:6737:7576:7901:7903:8599:9025:9545:9592:10004:10913:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13069:13311:13357:13846:14181:14384:14721:14849:21080:21451:21627:21939:30054:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutr al,Custo X-HE-Tag: name59_2e16e0c2cd009 X-Filterd-Recvd-Size: 3011 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:16 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9D70720880; Sun, 1 Dec 2019 01:55:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165316; bh=o6Doqt9SB77VAHHIGbjGZuOhM7jNCwY7lT/qcOGQUVk=; h=Date:From:To:Subject:From; b=F1CaiQB0pZhZBXrZdtuSlzKhyIGzn9Q+FlzJFI+Dubqdtpsf1xMYZDN0vvAC0ch7S 9XaZIWCq/+1I0nx0aEso1ChTmRGhvyl4ZNPgVcqL0OWvAjGXqU/Al0xpGc3EUVlNcc 0QToGZp7jIS5T//X7hOAEeYQuaVCju5Oh9iPaBis= Date: Sat, 30 Nov 2019 17:55:15 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, bp@alien8.de, cai@lca.pw, linux-mm@kvack.org, matt@codeblueprint.co.uk, mgorman@techsingularity.net, mhocko@suse.com, mm-commits@vger.kernel.org, tglx@linutronix.de, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 099/158] mm, pcpu: make zone pcp updates and reset internal to the mm Message-ID: <20191201015515.zqptNJ_r3%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Subject: mm, pcpu: make zone pcp updates and reset internal to the mm Memory hotplug needs to be able to reset and reinit the pcpu allocator batch and high limits but this action is internal to the VM. Move the declaration to internal.h Link: http://lkml.kernel.org/r/20191021094808.28824-4-mgorman@techsingularity.net Signed-off-by: Mel Gorman Acked-by: Michal Hocko Acked-by: Vlastimil Babka Cc: Borislav Petkov Cc: Matt Fleming Cc: Qian Cai Cc: Thomas Gleixner Signed-off-by: Andrew Morton --- include/linux/mm.h | 3 --- mm/internal.h | 3 +++ 2 files changed, 3 insertions(+), 3 deletions(-) --- a/include/linux/mm.h~mm-pcpu-make-zone-pcp-updates-and-reset-internal-to-the-mm +++ a/include/linux/mm.h @@ -2207,9 +2207,6 @@ void warn_alloc(gfp_t gfp_mask, nodemask extern void setup_per_cpu_pageset(void); -extern void zone_pcp_update(struct zone *zone); -extern void zone_pcp_reset(struct zone *zone); - /* page_alloc.c */ extern int min_free_kbytes; extern int watermark_boost_factor; --- a/mm/internal.h~mm-pcpu-make-zone-pcp-updates-and-reset-internal-to-the-mm +++ a/mm/internal.h @@ -165,6 +165,9 @@ extern void post_alloc_hook(struct page gfp_t gfp_flags); extern int user_min_free_kbytes; +extern void zone_pcp_update(struct zone *zone); +extern void zone_pcp_reset(struct zone *zone); + #if defined CONFIG_COMPACTION || defined CONFIG_CMA /* From patchwork Sun Dec 1 01:55:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268377 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF0CB921 for ; Sun, 1 Dec 2019 01:55:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AC2AB208C3 for ; Sun, 1 Dec 2019 01:55:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="M8eBbRJP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AC2AB208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9A15C6B0322; Sat, 30 Nov 2019 20:55:20 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 906546B0324; Sat, 30 Nov 2019 20:55:20 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8199D6B0325; Sat, 30 Nov 2019 20:55:20 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0180.hostedemail.com [216.40.44.180]) by kanga.kvack.org (Postfix) with ESMTP id 664F66B0322 for ; Sat, 30 Nov 2019 20:55:20 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 27250824999B for ; Sun, 1 Dec 2019 01:55:20 +0000 (UTC) X-FDA: 76214905200.21.dolls87_2e8e854f2a835 X-Spam-Summary: 2,0,0,e0f143494c3060e8,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:dan.j.williams@intel.com:hannes@cmpxchg.org:haolee.swjtu@gmail.com::mgorman@techsingularity.net:mhocko@suse.com:mm-commits@vger.kernel.org:richard.weiyang@gmail.com:torvalds@linux-foundation.org:vbabka@suse.cz,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3868:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:6737:7514:7576:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12696:12737:12783:12986:13069:13311:13357:14181:14384:14721:14849:21080:21451:21627:21939:30054:30064,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0, LFtime:3 X-HE-Tag: dolls87_2e8e854f2a835 X-Filterd-Recvd-Size: 2570 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:19 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E043B2084D; Sun, 1 Dec 2019 01:55:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165319; bh=IEkODHg01/lb8Z7vfZ1+esLh+D2UKf/1xib5/4wEPYQ=; h=Date:From:To:Subject:From; b=M8eBbRJPraJ8AU5K0y7HKIms2/4golUyi8xCCJqlVzdPCCVjQzC7Es7PI8654qfIM CSPpSepZZAQP8+vOeeydwje7N5C2J6/dhsDf8qYe/TkEdBSTlzSABRjhq7fRB9oDYv x+S1v5Vg4o5ahUGbFrtaJtVSJyUXArEumLOn4KPA= Date: Sat, 30 Nov 2019 17:55:18 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, dan.j.williams@intel.com, hannes@cmpxchg.org, haolee.swjtu@gmail.com, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@suse.com, mm-commits@vger.kernel.org, richard.weiyang@gmail.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 100/158] include/linux/mmzone.h: fix comment for ISOLATE_UNMAPPED macro Message-ID: <20191201015518.vT38I98Zz%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Hao Lee Subject: include/linux/mmzone.h: fix comment for ISOLATE_UNMAPPED macro Both file-backed pages and anonymous pages can be unmapped. ISOLATE_UNMAPPED is not just for file-backed pages. Link: http://lkml.kernel.org/r/20191024151621.GA20400@haolee.github.io Signed-off-by: Hao Lee Reviewed-by: Andrew Morton Cc: Vlastimil Babka Cc: Dan Williams Cc: Michal Hocko Cc: Wei Yang Cc: Mel Gorman Cc: Johannes Weiner Signed-off-by: Andrew Morton --- include/linux/mmzone.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/include/linux/mmzone.h~mm-fix-comment-for-isolate_unmapped-macro +++ a/include/linux/mmzone.h @@ -308,7 +308,7 @@ struct lruvec { #endif }; -/* Isolate unmapped file */ +/* Isolate unmapped pages */ #define ISOLATE_UNMAPPED ((__force isolate_mode_t)0x2) /* Isolate for asynchronous migration */ #define ISOLATE_ASYNC_MIGRATE ((__force isolate_mode_t)0x4) From patchwork Sun Dec 1 01:55:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268379 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 32E0F921 for ; Sun, 1 Dec 2019 01:55:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F16A1208E4 for ; Sun, 1 Dec 2019 01:55:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="NNZFUUYw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F16A1208E4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0528D6B0324; Sat, 30 Nov 2019 20:55:24 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F1CCC6B0326; Sat, 30 Nov 2019 20:55:23 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0BEF6B0327; Sat, 30 Nov 2019 20:55:23 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0038.hostedemail.com [216.40.44.38]) by kanga.kvack.org (Postfix) with ESMTP id C58A06B0324 for ; Sat, 30 Nov 2019 20:55:23 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 7C5A12A87 for ; Sun, 1 Dec 2019 01:55:23 +0000 (UTC) X-FDA: 76214905326.26.doll55_2f093c3aba03c X-Spam-Summary: 2,0,0,a08d76c992e898b5,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:jqqlijiazi@gmail.com:lijiazi@xiaomi.com::mgorman@techsingularity.net:mm-commits@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1714:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2693:2859:2899:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3351:3865:3870:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:5007:6261:6653:7514:7576:9025:9545:10004:10913:11026:11257:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13069:13311:13357:13846:14181:14384:14721:14849:21080:21451:21627:21939:30054:30075,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:4,LUA_SUMMARY:none X-HE-Tag: doll55_2f093c3aba03c X-Filterd-Recvd-Size: 2673 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:23 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 22E07208C3; Sun, 1 Dec 2019 01:55:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165322; bh=ICyPJ3VVcywJgy42PXxV2zrqmZcAQVo1QDly+Avs8c4=; h=Date:From:To:Subject:From; b=NNZFUUYwgmD0kGg8FanUuvajQmbLfq1ZW4NIibY2mt6scCGIuwiw5CzoQxcyrDjvz SoXPvc1EC81pjV4nWGh4ele0vDWE8ppr064r05fj92ZDKhxSd34VyXNPX64DFTfMo5 ZmNgKl99eE33DV4S0gXS75hrlxjk8T5qfHfyrziY= Date: Sat, 30 Nov 2019 17:55:21 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, jqqlijiazi@gmail.com, lijiazi@xiaomi.com, linux-mm@kvack.org, mgorman@techsingularity.net, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 101/158] mm/page_alloc.c: print reserved_highatomic info Message-ID: <20191201015521.O5E5mCmVA%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: lijiazi Subject: mm/page_alloc.c: print reserved_highatomic info Print nr_reserved_highatomic in show_free_areas, because when alloc_harder is false, this value will be subtracted from the free_pages in __zone_watermark_ok. Printing this value can help analyze memory allocaction failure issues. Link: http://lkml.kernel.org/r/19515f3de2fb6abe66b52e03e4b676a21e82beda.1573634806.git.lijiazi@xiaomi.com Signed-off-by: lijiazi Reviewed-by: Andrew Morton Cc: Mel Gorman Signed-off-by: Andrew Morton --- mm/page_alloc.c | 2 ++ 1 file changed, 2 insertions(+) --- a/mm/page_alloc.c~mm-page_alloc-print-reserved_highatomic-info +++ a/mm/page_alloc.c @@ -5354,6 +5354,7 @@ void show_free_areas(unsigned int filter " min:%lukB" " low:%lukB" " high:%lukB" + " reserved_highatomic:%luKB" " active_anon:%lukB" " inactive_anon:%lukB" " active_file:%lukB" @@ -5375,6 +5376,7 @@ void show_free_areas(unsigned int filter K(min_wmark_pages(zone)), K(low_wmark_pages(zone)), K(high_wmark_pages(zone)), + K(zone->nr_reserved_highatomic), K(zone_page_state(zone, NR_ZONE_ACTIVE_ANON)), K(zone_page_state(zone, NR_ZONE_INACTIVE_ANON)), K(zone_page_state(zone, NR_ZONE_ACTIVE_FILE)), From patchwork Sun Dec 1 01:55:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268381 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 82E59921 for ; Sun, 1 Dec 2019 01:55:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5056A208C3 for ; Sun, 1 Dec 2019 01:55:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="gxHvrEoT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5056A208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 498896B0326; Sat, 30 Nov 2019 20:55:27 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 421806B0328; Sat, 30 Nov 2019 20:55:27 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E9286B0329; Sat, 30 Nov 2019 20:55:27 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0133.hostedemail.com [216.40.44.133]) by kanga.kvack.org (Postfix) with ESMTP id 123936B0326 for ; Sat, 30 Nov 2019 20:55:27 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id C0AB92C7C for ; Sun, 1 Dec 2019 01:55:26 +0000 (UTC) X-FDA: 76214905452.21.pain72_2f81fc2463b4c X-Spam-Summary: 2,0,0,6026af9875612be1,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:aryabinin@virtuozzo.com:hannes@cmpxchg.org:jhubbard@nvidia.com::mgorman@techsingularity.net:mhocko@kernel.org:mm-commits@vger.kernel.org:riel@surriel.com:torvalds@linux-foundation.org:vbabka@suse.cz:william.kucharski@oracle.com,RULES_HIT:41:69:355:379:800:960:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1535:1543:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3867:3868:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4605:5007:6119:6261:6653:6737:7576:7903:7974:7996:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13846:14096:14181:14721:14849:21080:21325:21451:21611:21627:21939:30054:30064,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,Do mainCach X-HE-Tag: pain72_2f81fc2463b4c X-Filterd-Recvd-Size: 5209 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf26.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:26 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 53457208E4; Sun, 1 Dec 2019 01:55:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165325; bh=XGALu3sZP0/tYYPJ3ERo8kH7ajG0VBmkhcN1xJPi5wQ=; h=Date:From:To:Subject:From; b=gxHvrEoTwhQ3VbZJ61RU+hI4rmleahg3NjxcTolYiFJeJVeUwD9ucLW/Tf5SlChx8 9LvqFqZLJX1xqTUXxnOm+S4NQbSHTdhNS351S7RCYi/MfbD6L58VBI6dM5qLLHun6r i045LBFC8/ef27VHngDYUfSyfCjOjzD1PbGeMRlY= Date: Sat, 30 Nov 2019 17:55:24 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, aryabinin@virtuozzo.com, hannes@cmpxchg.org, jhubbard@nvidia.com, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mm-commits@vger.kernel.org, riel@surriel.com, torvalds@linux-foundation.org, vbabka@suse.cz, william.kucharski@oracle.com Subject: [patch 102/158] mm/vmscan: remove unused lru_pages argument Message-ID: <20191201015524.fXucDKo4z%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Ryabinin Subject: mm/vmscan: remove unused lru_pages argument Since 9092c71bb724 ("mm: use sc->priority for slab shrink targets") the argument 'unsigned long *lru_pages' passed around with no purpose. Remove it. Link: http://lkml.kernel.org/r/20190228083329.31892-4-aryabinin@virtuozzo.com Signed-off-by: Andrey Ryabinin Acked-by: Johannes Weiner Acked-by: Vlastimil Babka Acked-by: Mel Gorman Cc: Michal Hocko Cc: Rik van Riel Cc: William Kucharski Cc: John Hubbard Signed-off-by: Andrew Morton --- mm/vmscan.c | 17 +++++------------ 1 file changed, 5 insertions(+), 12 deletions(-) --- a/mm/vmscan.c~mm-vmscan-remove-unused-lru_pages-argument +++ a/mm/vmscan.c @@ -2302,8 +2302,7 @@ enum scan_balance { * nr[2] = file inactive pages to scan; nr[3] = file active pages to scan */ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg, - struct scan_control *sc, unsigned long *nr, - unsigned long *lru_pages) + struct scan_control *sc, unsigned long *nr) { int swappiness = mem_cgroup_swappiness(memcg); struct zone_reclaim_stat *reclaim_stat = &lruvec->reclaim_stat; @@ -2454,7 +2453,6 @@ static void get_scan_count(struct lruvec fraction[1] = fp; denominator = ap + fp + 1; out: - *lru_pages = 0; for_each_evictable_lru(lru) { int file = is_file_lru(lru); unsigned long lruvec_size; @@ -2549,7 +2547,6 @@ out: BUG(); } - *lru_pages += lruvec_size; nr[lru] = scan; } } @@ -2558,7 +2555,7 @@ out: * This is a basic per-node page freer. Used by both kswapd and direct reclaim. */ static void shrink_node_memcg(struct pglist_data *pgdat, struct mem_cgroup *memcg, - struct scan_control *sc, unsigned long *lru_pages) + struct scan_control *sc) { struct lruvec *lruvec = mem_cgroup_lruvec(pgdat, memcg); unsigned long nr[NR_LRU_LISTS]; @@ -2570,7 +2567,7 @@ static void shrink_node_memcg(struct pgl struct blk_plug plug; bool scan_adjusted; - get_scan_count(lruvec, memcg, sc, nr, lru_pages); + get_scan_count(lruvec, memcg, sc, nr); /* Record the original scan target for proportional adjustments later */ memcpy(targets, nr, sizeof(nr)); @@ -2758,7 +2755,6 @@ static bool shrink_node(pg_data_t *pgdat do { struct mem_cgroup *root = sc->target_mem_cgroup; - unsigned long node_lru_pages = 0; struct mem_cgroup *memcg; memset(&sc->nr, 0, sizeof(sc->nr)); @@ -2768,7 +2764,6 @@ static bool shrink_node(pg_data_t *pgdat memcg = mem_cgroup_iter(root, NULL, NULL); do { - unsigned long lru_pages; unsigned long reclaimed; unsigned long scanned; @@ -2805,8 +2800,7 @@ static bool shrink_node(pg_data_t *pgdat reclaimed = sc->nr_reclaimed; scanned = sc->nr_scanned; - shrink_node_memcg(pgdat, memcg, sc, &lru_pages); - node_lru_pages += lru_pages; + shrink_node_memcg(pgdat, memcg, sc); shrink_slab(sc->gfp_mask, pgdat->node_id, memcg, sc->priority); @@ -3317,7 +3311,6 @@ unsigned long mem_cgroup_shrink_node(str .reclaim_idx = MAX_NR_ZONES - 1, .may_swap = !noswap, }; - unsigned long lru_pages; WARN_ON_ONCE(!current->reclaim_state); @@ -3334,7 +3327,7 @@ unsigned long mem_cgroup_shrink_node(str * will pick up pages from other mem cgroup's as well. We hack * the priority and make it zero. */ - shrink_node_memcg(pgdat, memcg, &sc, &lru_pages); + shrink_node_memcg(pgdat, memcg, &sc); trace_mm_vmscan_memcg_softlimit_reclaim_end(sc.nr_reclaimed); From patchwork Sun Dec 1 01:55:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268383 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CE490112B for ; Sun, 1 Dec 2019 01:55:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9216F208E4 for ; Sun, 1 Dec 2019 01:55:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="NKnlCYB2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9216F208E4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 841496B0328; Sat, 30 Nov 2019 20:55:30 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7CB5D6B032A; Sat, 30 Nov 2019 20:55:30 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6E1156B032B; Sat, 30 Nov 2019 20:55:30 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0028.hostedemail.com [216.40.44.28]) by kanga.kvack.org (Postfix) with ESMTP id 521086B0328 for ; Sat, 30 Nov 2019 20:55:30 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 06EF3181AEF0B for ; Sun, 1 Dec 2019 01:55:30 +0000 (UTC) X-FDA: 76214905620.18.walk28_2ffb35ad5475f X-Spam-Summary: 2,0,0,c2df013af3926a54,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hannes@cmpxchg.org::mgorman@techsingularity.net:mhocko@suse.com:mm-commits@vger.kernel.org:rientjes@google.com:torvalds@linux-foundation.org:yang.shi@linux.alibaba.com,RULES_HIT:41:355:379:800:960:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:5007:6261:6653:7576:7875:7903:9025:9391:9545:10004:10913:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13255:14096:14181:14721:14849:21060:21080:21433:21451:21627:21819:21939:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: walk28_2ffb35ad5475f X-Filterd-Recvd-Size: 3501 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:29 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A9391208C3; Sun, 1 Dec 2019 01:55:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165329; bh=hR3yZZQVSJnLXIJ7lYCg4CSCAmuZ/+qQy2gDpXPEf3c=; h=Date:From:To:Subject:From; b=NKnlCYB2rQQIZG94z8YM1NjUadKBJVj2wIICvQKD/wVbfSzthh9eSI1vNeKFaLhwQ X+pQvyTN+ypYuu1A5w53+p3tcxfxJx6jWrjFQXkihPnMvGEnZrZId1MqXc/rmoOtgt z/psLYCH3feZPCJ+vWQbbwtW7NIKO9QHHEifxZPY= Date: Sat, 30 Nov 2019 17:55:28 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hannes@cmpxchg.org, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@suse.com, mm-commits@vger.kernel.org, rientjes@google.com, torvalds@linux-foundation.org, yang.shi@linux.alibaba.com Subject: [patch 103/158] mm/vmscan.c: remove unused scan_control parameter from pageout() Message-ID: <20191201015528.9S6Ztf0rS%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Shi Subject: mm/vmscan.c: remove unused scan_control parameter from pageout() Since lumpy reclaim was removed in v3.5 scan_control is not used by may_write_to_{queue|inode} and pageout() anymore, remove the unused parameter. Link: http://lkml.kernel.org/r/1570124498-19300-1-git-send-email-yang.shi@linux.alibaba.com Signed-off-by: Yang Shi Acked-by: David Rientjes Acked-by: Michal Hocko Cc: Mel Gorman Cc: Johannes Weiner Signed-off-by: Andrew Morton --- mm/vmscan.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) --- a/mm/vmscan.c~mm-vmscan-remove-unused-scan_control-parameter-from-pageout +++ a/mm/vmscan.c @@ -775,7 +775,7 @@ static inline int is_page_cache_freeable return page_count(page) - page_has_private(page) == 1 + page_cache_pins; } -static int may_write_to_inode(struct inode *inode, struct scan_control *sc) +static int may_write_to_inode(struct inode *inode) { if (current->flags & PF_SWAPWRITE) return 1; @@ -823,8 +823,7 @@ typedef enum { * pageout is called by shrink_page_list() for each dirty page. * Calls ->writepage(). */ -static pageout_t pageout(struct page *page, struct address_space *mapping, - struct scan_control *sc) +static pageout_t pageout(struct page *page, struct address_space *mapping) { /* * If the page is dirty, only perform writeback if that write @@ -860,7 +859,7 @@ static pageout_t pageout(struct page *pa } if (mapping->a_ops->writepage == NULL) return PAGE_ACTIVATE; - if (!may_write_to_inode(mapping->host, sc)) + if (!may_write_to_inode(mapping->host)) return PAGE_KEEP; if (clear_page_dirty_for_io(page)) { @@ -1394,7 +1393,7 @@ static unsigned long shrink_page_list(st * starts and then write it out here. */ try_to_unmap_flush_dirty(); - switch (pageout(page, mapping, sc)) { + switch (pageout(page, mapping)) { case PAGE_KEEP: goto keep_locked; case PAGE_ACTIVATE: From patchwork Sun Dec 1 01:55:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268385 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E3AE1112B for ; Sun, 1 Dec 2019 01:55:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A76C9215A5 for ; Sun, 1 Dec 2019 01:55:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="KQreSFZ4" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A76C9215A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A64F76B032A; Sat, 30 Nov 2019 20:55:33 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9ED6C6B032C; Sat, 30 Nov 2019 20:55:33 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B64D6B032D; Sat, 30 Nov 2019 20:55:33 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0192.hostedemail.com [216.40.44.192]) by kanga.kvack.org (Postfix) with ESMTP id 6EE846B032A for ; Sat, 30 Nov 2019 20:55:33 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 2CD8C181AEF0B for ; Sun, 1 Dec 2019 01:55:33 +0000 (UTC) X-FDA: 76214905746.15.able82_3070b590dd01d X-Spam-Summary: 30,2,0,120bc1e6cbc442ca,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:guro@fb.com:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:shakeelb@google.com:torvalds@linux-foundation.org,RULES_HIT:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2393:2525:2559:2565:2682:2685:2692:2736:2859:2898:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3369:3865:3866:3867:3868:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4605:5007:6119:6261:6653:7576:7903:8599:8603:9025:9545:9592:10010:10913:11026:11233:11256:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:13846:14093:14181:14721:14849:21080:21451:21627:21740:21939:21966:30054:30064:30070:30080,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:3,LUA _SUMMARY X-HE-Tag: able82_3070b590dd01d X-Filterd-Recvd-Size: 3796 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:32 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D6BA7208E4; Sun, 1 Dec 2019 01:55:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165332; bh=1C6drgu0QhevsUrKyb+77FKMLplPHuvqNY60pnrokcw=; h=Date:From:To:Subject:From; b=KQreSFZ45OjFa2p3I1CG6aGIj9U6jRIvSytU9+5fBQcS9MSWEkT5mWj8G/ysv+FmW ZVCN3GLjdvM29I764HmO/9FGQql2KFriEKU/5VY6wiYRXt+SSvmOubZw1wFBjqhRau vK8WHPSTruSNDrzlSgh9X87jyEHWXeeg69kR5q2U= Date: Sat, 30 Nov 2019 17:55:31 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 104/158] mm: vmscan: simplify lruvec_lru_size() Message-ID: <20191201015531.6FvHrYN3P%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: mm: vmscan: simplify lruvec_lru_size() Patch series "mm: vmscan: cgroup-related cleanups". Here are 8 patches that clean up the reclaim code's interaction with cgroups a bit. They're not supposed to change any behavior, just make the implementation easier to understand and work with. This patch (of 8): This function currently takes the node or lruvec size and subtracts the zones that are excluded by the classzone index of the allocation. It uses four different types of counters to do this. Just add up the eligible zones. [cai@lca.pw: fix an undefined behavior for zone id] Link: http://lkml.kernel.org/r/20191108204407.1435-1-cai@lca.pw [akpm@linux-foundation.org: deal with the MAX_NR_ZONES special case. per Qian Cai] Link: http://lkml.kernel.org/r/64E60F6F-7582-427B-8DD5-EF97B1656F5A@lca.pw Link: http://lkml.kernel.org/r/20191022144803.302233-2-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Reviewed-by: Roman Gushchin Reviewed-by: Shakeel Butt Acked-by: Michal Hocko Signed-off-by: Andrew Morton --- mm/vmscan.c | 21 +++++---------------- 1 file changed, 5 insertions(+), 16 deletions(-) --- a/mm/vmscan.c~mm-vmscan-simplify-lruvec_lru_size +++ a/mm/vmscan.c @@ -351,32 +351,21 @@ unsigned long zone_reclaimable_pages(str */ unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru, int zone_idx) { - unsigned long lru_size = 0; + unsigned long size = 0; int zid; - if (!mem_cgroup_disabled()) { - for (zid = 0; zid < MAX_NR_ZONES; zid++) - lru_size += mem_cgroup_get_zone_lru_size(lruvec, lru, zid); - } else - lru_size = node_page_state(lruvec_pgdat(lruvec), NR_LRU_BASE + lru); - - for (zid = zone_idx + 1; zid < MAX_NR_ZONES; zid++) { + for (zid = 0; zid <= zone_idx && zid < MAX_NR_ZONES; zid++) { struct zone *zone = &lruvec_pgdat(lruvec)->node_zones[zid]; - unsigned long size; if (!managed_zone(zone)) continue; if (!mem_cgroup_disabled()) - size = mem_cgroup_get_zone_lru_size(lruvec, lru, zid); + size += mem_cgroup_get_zone_lru_size(lruvec, lru, zid); else - size = zone_page_state(&lruvec_pgdat(lruvec)->node_zones[zid], - NR_ZONE_LRU_BASE + lru); - lru_size -= min(size, lru_size); + size += zone_page_state(zone, NR_ZONE_LRU_BASE + lru); } - - return lru_size; - + return size; } /* From patchwork Sun Dec 1 01:55:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268387 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 64DDA921 for ; Sun, 1 Dec 2019 01:55:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 15E342231B for ; Sun, 1 Dec 2019 01:55:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="m4MhIy2X" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 15E342231B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ED1606B032C; Sat, 30 Nov 2019 20:55:36 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E5A4C6B032E; Sat, 30 Nov 2019 20:55:36 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CFB646B032F; Sat, 30 Nov 2019 20:55:36 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0197.hostedemail.com [216.40.44.197]) by kanga.kvack.org (Postfix) with ESMTP id B107D6B032C for ; Sat, 30 Nov 2019 20:55:36 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 5E2C4824999B for ; Sun, 1 Dec 2019 01:55:36 +0000 (UTC) X-FDA: 76214905872.23.girls48_30e33fe9eec2e X-Spam-Summary: 2,0,0,8d31c6cb3039362d,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:guro@fb.com:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:shakeelb@google.com:torvalds@linux-foundation.org,RULES_HIT:1:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1605:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2636:2682:2685:2731:2859:2902:2918:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4605:5007:6261:6653:7576:7809:7882:7903:8599:8660:8957:9010:9025:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13148:13149:13161:13200:13229:13230:13846:14096:21080:21324:21325:21450:21451:21627:21740:21939:21966:21972:30012:30029:30054:30064,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL: neutral, X-HE-Tag: girls48_30e33fe9eec2e X-Filterd-Recvd-Size: 11465 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:35 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id F10A5215A5; Sun, 1 Dec 2019 01:55:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165335; bh=nnrtyLtdV6V22bOxOtVBu0u+/q1C9WJHaDhcscXSO8U=; h=Date:From:To:Subject:From; b=m4MhIy2XOvK/cf7RSkeauPxjXUHAwTVkp2J5ROKDvVUQDobNCXWmfdbiB0P7ONJY5 /OdTdoIq1gpN+ZxkcxaEQU4IWuPKaJGe23eIL1xIZ6cG5fauy9SRoHF5pqLau3wUa5 P2mGbQGOjBx1g5So1fbx9QLLVRKzibeUb7ihoM5Y= Date: Sat, 30 Nov 2019 17:55:34 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 105/158] mm: clean up and clarify lruvec lookup procedure Message-ID: <20191201015534.GRTpgpTmy%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: mm: clean up and clarify lruvec lookup procedure There is a per-memcg lruvec and a NUMA node lruvec. Which one is being used is somewhat confusing right now, and it's easy to make mistakes - especially when it comes to global reclaim. How it works: when memory cgroups are enabled, we always use the root_mem_cgroup's per-node lruvecs. When memory cgroups are not compiled in or disabled at runtime, we use pgdat->lruvec. Document that in a comment. Due to the way the reclaim code is generalized, all lookups use the mem_cgroup_lruvec() helper function, and nobody should have to find the right lruvec manually right now. But to avoid future mistakes, rename the pgdat->lruvec member to pgdat->__lruvec and delete the convenience wrapper that suggests it's a commonly accessed member. While in this area, swap the mem_cgroup_lruvec() argument order. The name suggests a memcg operation, yet it takes a pgdat first and a memcg second. I have to double take every time I call this. Fix that. Link: http://lkml.kernel.org/r/20191022144803.302233-3-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Acked-by: Michal Hocko Reviewed-by: Shakeel Butt Cc: Roman Gushchin Signed-off-by: Andrew Morton --- include/linux/memcontrol.h | 24 ++++++++++++------------ include/linux/mmzone.h | 15 ++++++++------- mm/memcontrol.c | 10 +++++----- mm/page_alloc.c | 2 +- mm/slab.h | 4 ++-- mm/vmscan.c | 6 +++--- mm/workingset.c | 8 ++++---- 7 files changed, 35 insertions(+), 34 deletions(-) --- a/include/linux/memcontrol.h~mm-clean-up-and-clarify-lruvec-lookup-procedure +++ a/include/linux/memcontrol.h @@ -385,21 +385,21 @@ mem_cgroup_nodeinfo(struct mem_cgroup *m } /** - * mem_cgroup_lruvec - get the lru list vector for a node or a memcg zone - * @node: node of the wanted lruvec + * mem_cgroup_lruvec - get the lru list vector for a memcg & node * @memcg: memcg of the wanted lruvec * - * Returns the lru list vector holding pages for a given @node or a given - * @memcg. This can be the node lruvec, if the memory controller is disabled. + * Returns the lru list vector holding pages for a given @memcg & + * @node combination. This can be the node lruvec, if the memory + * controller is disabled. */ -static inline struct lruvec *mem_cgroup_lruvec(struct pglist_data *pgdat, - struct mem_cgroup *memcg) +static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg, + struct pglist_data *pgdat) { struct mem_cgroup_per_node *mz; struct lruvec *lruvec; if (mem_cgroup_disabled()) { - lruvec = node_lruvec(pgdat); + lruvec = &pgdat->__lruvec; goto out; } @@ -718,7 +718,7 @@ static inline void __mod_lruvec_page_sta return; } - lruvec = mem_cgroup_lruvec(pgdat, page->mem_cgroup); + lruvec = mem_cgroup_lruvec(page->mem_cgroup, pgdat); __mod_lruvec_state(lruvec, idx, val); } @@ -889,16 +889,16 @@ static inline void mem_cgroup_migrate(st { } -static inline struct lruvec *mem_cgroup_lruvec(struct pglist_data *pgdat, - struct mem_cgroup *memcg) +static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg, + struct pglist_data *pgdat) { - return node_lruvec(pgdat); + return &pgdat->__lruvec; } static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct pglist_data *pgdat) { - return &pgdat->lruvec; + return &pgdat->__lruvec; } static inline bool mm_match_cgroup(struct mm_struct *mm, --- a/include/linux/mmzone.h~mm-clean-up-and-clarify-lruvec-lookup-procedure +++ a/include/linux/mmzone.h @@ -777,7 +777,13 @@ typedef struct pglist_data { #endif /* Fields commonly accessed by the page reclaim scanner */ - struct lruvec lruvec; + + /* + * NOTE: THIS IS UNUSED IF MEMCG IS ENABLED. + * + * Use mem_cgroup_lruvec() to look up lruvecs. + */ + struct lruvec __lruvec; unsigned long flags; @@ -800,11 +806,6 @@ typedef struct pglist_data { #define node_start_pfn(nid) (NODE_DATA(nid)->node_start_pfn) #define node_end_pfn(nid) pgdat_end_pfn(NODE_DATA(nid)) -static inline struct lruvec *node_lruvec(struct pglist_data *pgdat) -{ - return &pgdat->lruvec; -} - static inline unsigned long pgdat_end_pfn(pg_data_t *pgdat) { return pgdat->node_start_pfn + pgdat->node_spanned_pages; @@ -842,7 +843,7 @@ static inline struct pglist_data *lruvec #ifdef CONFIG_MEMCG return lruvec->pgdat; #else - return container_of(lruvec, struct pglist_data, lruvec); + return container_of(lruvec, struct pglist_data, __lruvec); #endif } --- a/mm/memcontrol.c~mm-clean-up-and-clarify-lruvec-lookup-procedure +++ a/mm/memcontrol.c @@ -777,7 +777,7 @@ void __mod_lruvec_slab_state(void *p, en if (!memcg || memcg == root_mem_cgroup) { __mod_node_page_state(pgdat, idx, val); } else { - lruvec = mem_cgroup_lruvec(pgdat, memcg); + lruvec = mem_cgroup_lruvec(memcg, pgdat); __mod_lruvec_state(lruvec, idx, val); } rcu_read_unlock(); @@ -1221,7 +1221,7 @@ struct lruvec *mem_cgroup_page_lruvec(st struct lruvec *lruvec; if (mem_cgroup_disabled()) { - lruvec = &pgdat->lruvec; + lruvec = &pgdat->__lruvec; goto out; } @@ -3634,7 +3634,7 @@ static int mem_cgroup_move_charge_write( static unsigned long mem_cgroup_node_nr_lru_pages(struct mem_cgroup *memcg, int nid, unsigned int lru_mask) { - struct lruvec *lruvec = mem_cgroup_lruvec(NODE_DATA(nid), memcg); + struct lruvec *lruvec = mem_cgroup_lruvec(memcg, NODE_DATA(nid)); unsigned long nr = 0; enum lru_list lru; @@ -5338,8 +5338,8 @@ static int mem_cgroup_move_account(struc anon = PageAnon(page); pgdat = page_pgdat(page); - from_vec = mem_cgroup_lruvec(pgdat, from); - to_vec = mem_cgroup_lruvec(pgdat, to); + from_vec = mem_cgroup_lruvec(from, pgdat); + to_vec = mem_cgroup_lruvec(to, pgdat); spin_lock_irqsave(&from->move_lock, flags); --- a/mm/page_alloc.c~mm-clean-up-and-clarify-lruvec-lookup-procedure +++ a/mm/page_alloc.c @@ -6713,7 +6713,7 @@ static void __meminit pgdat_init_interna pgdat_page_ext_init(pgdat); spin_lock_init(&pgdat->lru_lock); - lruvec_init(node_lruvec(pgdat)); + lruvec_init(&pgdat->__lruvec); } static void __meminit zone_init_internals(struct zone *zone, enum zone_type idx, int nid, --- a/mm/slab.h~mm-clean-up-and-clarify-lruvec-lookup-procedure +++ a/mm/slab.h @@ -369,7 +369,7 @@ static __always_inline int memcg_charge_ if (ret) goto out; - lruvec = mem_cgroup_lruvec(page_pgdat(page), memcg); + lruvec = mem_cgroup_lruvec(memcg, page_pgdat(page)); mod_lruvec_state(lruvec, cache_vmstat_idx(s), 1 << order); /* transer try_charge() page references to kmem_cache */ @@ -393,7 +393,7 @@ static __always_inline void memcg_unchar rcu_read_lock(); memcg = READ_ONCE(s->memcg_params.memcg); if (likely(!mem_cgroup_is_root(memcg))) { - lruvec = mem_cgroup_lruvec(page_pgdat(page), memcg); + lruvec = mem_cgroup_lruvec(memcg, page_pgdat(page)); mod_lruvec_state(lruvec, cache_vmstat_idx(s), -(1 << order)); memcg_kmem_uncharge_memcg(page, order, memcg); } else { --- a/mm/vmscan.c~mm-clean-up-and-clarify-lruvec-lookup-procedure +++ a/mm/vmscan.c @@ -2545,7 +2545,7 @@ out: static void shrink_node_memcg(struct pglist_data *pgdat, struct mem_cgroup *memcg, struct scan_control *sc) { - struct lruvec *lruvec = mem_cgroup_lruvec(pgdat, memcg); + struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); unsigned long nr[NR_LRU_LISTS]; unsigned long targets[NR_LRU_LISTS]; unsigned long nr_to_scan; @@ -3023,7 +3023,7 @@ static void snapshot_refaults(struct mem unsigned long refaults; struct lruvec *lruvec; - lruvec = mem_cgroup_lruvec(pgdat, memcg); + lruvec = mem_cgroup_lruvec(memcg, pgdat); refaults = lruvec_page_state_local(lruvec, WORKINGSET_ACTIVATE); lruvec->refaults = refaults; } while ((memcg = mem_cgroup_iter(root_memcg, memcg, NULL))); @@ -3379,7 +3379,7 @@ static void age_active_anon(struct pglis memcg = mem_cgroup_iter(NULL, NULL, NULL); do { - struct lruvec *lruvec = mem_cgroup_lruvec(pgdat, memcg); + struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); if (inactive_list_is_low(lruvec, false, sc, true)) shrink_active_list(SWAP_CLUSTER_MAX, lruvec, --- a/mm/workingset.c~mm-clean-up-and-clarify-lruvec-lookup-procedure +++ a/mm/workingset.c @@ -233,7 +233,7 @@ void *workingset_eviction(struct page *p VM_BUG_ON_PAGE(page_count(page), page); VM_BUG_ON_PAGE(!PageLocked(page), page); - lruvec = mem_cgroup_lruvec(pgdat, memcg); + lruvec = mem_cgroup_lruvec(memcg, pgdat); eviction = atomic_long_inc_return(&lruvec->inactive_age); return pack_shadow(memcgid, pgdat, eviction, PageWorkingset(page)); } @@ -280,7 +280,7 @@ void workingset_refault(struct page *pag memcg = mem_cgroup_from_id(memcgid); if (!mem_cgroup_disabled() && !memcg) goto out; - lruvec = mem_cgroup_lruvec(pgdat, memcg); + lruvec = mem_cgroup_lruvec(memcg, pgdat); refault = atomic_long_read(&lruvec->inactive_age); active_file = lruvec_lru_size(lruvec, LRU_ACTIVE_FILE, MAX_NR_ZONES); @@ -345,7 +345,7 @@ void workingset_activation(struct page * memcg = page_memcg_rcu(page); if (!mem_cgroup_disabled() && !memcg) goto out; - lruvec = mem_cgroup_lruvec(page_pgdat(page), memcg); + lruvec = mem_cgroup_lruvec(memcg, page_pgdat(page)); atomic_long_inc(&lruvec->inactive_age); out: rcu_read_unlock(); @@ -426,7 +426,7 @@ static unsigned long count_shadow_nodes( struct lruvec *lruvec; int i; - lruvec = mem_cgroup_lruvec(NODE_DATA(sc->nid), sc->memcg); + lruvec = mem_cgroup_lruvec(sc->memcg, NODE_DATA(sc->nid)); for (pages = 0, i = 0; i < NR_LRU_LISTS; i++) pages += lruvec_page_state_local(lruvec, NR_LRU_BASE + i); From patchwork Sun Dec 1 01:55:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268389 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2DA79921 for ; Sun, 1 Dec 2019 01:55:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EEAD3215F1 for ; Sun, 1 Dec 2019 01:55:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="zQAkTauz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EEAD3215F1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D72CE6B032E; Sat, 30 Nov 2019 20:55:39 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CFC5B6B0330; Sat, 30 Nov 2019 20:55:39 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC42E6B0331; Sat, 30 Nov 2019 20:55:39 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0167.hostedemail.com [216.40.44.167]) by kanga.kvack.org (Postfix) with ESMTP id 9D91F6B032E for ; Sat, 30 Nov 2019 20:55:39 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 5D6A4180AD817 for ; Sun, 1 Dec 2019 01:55:39 +0000 (UTC) X-FDA: 76214905998.07.end17_3156f1b320500 X-Spam-Summary: 2,0,0,afcce249e7704240,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:guro@fb.com:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:shakeelb@google.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6119:6261:6653:7576:7903:8599:9010:9025:9545:9592:10004:10241:10913:11026:11473:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:13255:13846:14096:14181:14721:14819:14849:21080:21212:21451:21627:21939:30054:30064:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:26,LUA_SUMMARY:none X-HE-Tag: end17_3156f1b320500 X-Filterd-Recvd-Size: 3575 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:38 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0AF9F215E5; Sun, 1 Dec 2019 01:55:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165338; bh=xgrCcg30swxsYcKGelq275jijxR7bheU9GWA5wG14Vc=; h=Date:From:To:Subject:From; b=zQAkTauzPmLhGQnTM8o9GcHdeYeYj7PAZlxVRwvvyMcX2sFlOYKrQ+phxKa+CISn2 Xr4Ixe3wA7Ace8LzEC1qmyNwxKe8Ju2ustv8uH26ogAN1yUQw3h4ctc1p3HgFBKmhp Bu1fAFqndnnBMjH1IXOtcQ7wHDKhZYL4ZiLxVMV4= Date: Sat, 30 Nov 2019 17:55:37 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 106/158] mm: vmscan: move inactive_list_is_low() swap check to the caller Message-ID: <20191201015537.V_kjbI37_%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: mm: vmscan: move inactive_list_is_low() swap check to the caller inactive_list_is_low() should be about one thing: checking the ratio between inactive and active list. Kitchensink checks like the one for swap space makes the function hard to use and modify its callsites. Luckly, most callers already have an understanding of the swap situation, so it's easy to clean up. get_scan_count() has its own, memcg-aware swap check, and doesn't even get to the inactive_list_is_low() check on the anon list when there is no swap space available. shrink_list() is called on the results of get_scan_count(), so that check is redundant too. age_active_anon() has its own totalswap_pages check right before it checks the list proportions. The shrink_node_memcg() site is the only one that doesn't do its own swap check. Add it there. Then delete the swap check from inactive_list_is_low(). Link: http://lkml.kernel.org/r/20191022144803.302233-4-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Reviewed-by: Roman Gushchin Reviewed-by: Shakeel Butt Acked-by: Michal Hocko Signed-off-by: Andrew Morton --- mm/vmscan.c | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) --- a/mm/vmscan.c~mm-vmscan-move-inactive_list_is_low-swap-check-to-the-caller +++ a/mm/vmscan.c @@ -2226,13 +2226,6 @@ static bool inactive_list_is_low(struct unsigned long refaults; unsigned long gb; - /* - * If we don't have swap space, anonymous page deactivation - * is pointless. - */ - if (!file && !total_swap_pages) - return false; - inactive = lruvec_lru_size(lruvec, inactive_lru, sc->reclaim_idx); active = lruvec_lru_size(lruvec, active_lru, sc->reclaim_idx); @@ -2653,7 +2646,7 @@ static void shrink_node_memcg(struct pgl * Even if we did not try to evict anon pages at all, we want to * rebalance the anon lru active/inactive ratio. */ - if (inactive_list_is_low(lruvec, false, sc, true)) + if (total_swap_pages && inactive_list_is_low(lruvec, false, sc, true)) shrink_active_list(SWAP_CLUSTER_MAX, lruvec, sc, LRU_ACTIVE_ANON); } From patchwork Sun Dec 1 01:55:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268391 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 47A16112B for ; Sun, 1 Dec 2019 01:55:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 07FAD2245C for ; Sun, 1 Dec 2019 01:55:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="KBw6AmoC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 07FAD2245C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D4AA86B0330; Sat, 30 Nov 2019 20:55:42 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CD48D6B0332; Sat, 30 Nov 2019 20:55:42 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B9C426B0333; Sat, 30 Nov 2019 20:55:42 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0099.hostedemail.com [216.40.44.99]) by kanga.kvack.org (Postfix) with ESMTP id A190F6B0330 for ; Sat, 30 Nov 2019 20:55:42 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 59B2D40F1 for ; Sun, 1 Dec 2019 01:55:42 +0000 (UTC) X-FDA: 76214906124.17.wound04_31c4c9f59b132 X-Spam-Summary: 2,0,0,afe97512460d6b24,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:guro@fb.com:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:2:41:69:355:379:800:960:966:967:973:982:988:989:1260:1263:1345:1381:1431:1437:1535:1605:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2693:2859:2898:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4049:4118:4250:4321:4385:5007:6119:6261:6653:7576:7875:8599:8603:8957:9025:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13161:13229:13846:13972:14096:21080:21324:21451:21611:21627:21796:21939:30029:30036:30041:30054:30064:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFt ime:23,L X-HE-Tag: wound04_31c4c9f59b132 X-Filterd-Recvd-Size: 7305 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:41 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 109902231B; Sun, 1 Dec 2019 01:55:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165341; bh=gdiOj1wpH2wQAl9O3YzBOZ9dYBjMAinnZrXYepoooxo=; h=Date:From:To:Subject:From; b=KBw6AmoCTo4QQYiBRBswHLqRd7aX3FKhSWDxTrNC1wlW29vLwezcWnZvtghrFD1So vLwAo2sI3+M4aJMqZ/GQIgyx0KZHuUQDzMAeqj6w2LUT66QYxR2S64JL/371XGSrTa ceU0uqXI/khPKvw/XslZNzouGAPd+y6stdAwZ4dU= Date: Sat, 30 Nov 2019 17:55:40 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 107/158] mm: vmscan: naming fixes: global_reclaim() and sane_reclaim() Message-ID: <20191201015540.IivK3zfAf%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: mm: vmscan: naming fixes: global_reclaim() and sane_reclaim() Seven years after introducing the global_reclaim() function, I still have to double take when reading a callsite. I don't know how others do it, this is a terrible name. Invert the meaning and rename it to cgroup_reclaim(). [ After all, "global reclaim" is just regular reclaim invoked from the page allocator. It's reclaim on behalf of a cgroup limit that is a special case of reclaim, and should be explicit - not the reverse. ] sane_reclaim() isn't very descriptive either: it tests whether we can use the regular writeback throttling - available during regular page reclaim or cgroup2 limit reclaim - or need to use the broken wait_on_page_writeback() method. Use "writeback_throttling_sane()". Link: http://lkml.kernel.org/r/20191022144803.302233-5-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Reviewed-by: Roman Gushchin Acked-by: Michal Hocko Signed-off-by: Andrew Morton --- mm/vmscan.c | 38 ++++++++++++++++++-------------------- 1 file changed, 18 insertions(+), 20 deletions(-) --- a/mm/vmscan.c~mm-vmscan-naming-fixes-global_reclaim-and-sane_reclaim +++ a/mm/vmscan.c @@ -239,13 +239,13 @@ static void unregister_memcg_shrinker(st up_write(&shrinker_rwsem); } -static bool global_reclaim(struct scan_control *sc) +static bool cgroup_reclaim(struct scan_control *sc) { - return !sc->target_mem_cgroup; + return sc->target_mem_cgroup; } /** - * sane_reclaim - is the usual dirty throttling mechanism operational? + * writeback_throttling_sane - is the usual dirty throttling mechanism available? * @sc: scan_control in question * * The normal page dirty throttling mechanism in balance_dirty_pages() is @@ -257,11 +257,9 @@ static bool global_reclaim(struct scan_c * This function tests whether the vmscan currently in progress can assume * that the normal dirty throttling mechanism is operational. */ -static bool sane_reclaim(struct scan_control *sc) +static bool writeback_throttling_sane(struct scan_control *sc) { - struct mem_cgroup *memcg = sc->target_mem_cgroup; - - if (!memcg) + if (!cgroup_reclaim(sc)) return true; #ifdef CONFIG_CGROUP_WRITEBACK if (cgroup_subsys_on_dfl(memory_cgrp_subsys)) @@ -302,12 +300,12 @@ static void unregister_memcg_shrinker(st { } -static bool global_reclaim(struct scan_control *sc) +static bool cgroup_reclaim(struct scan_control *sc) { - return true; + return false; } -static bool sane_reclaim(struct scan_control *sc) +static bool writeback_throttling_sane(struct scan_control *sc) { return true; } @@ -1227,7 +1225,7 @@ static unsigned long shrink_page_list(st goto activate_locked; /* Case 2 above */ - } else if (sane_reclaim(sc) || + } else if (writeback_throttling_sane(sc) || !PageReclaim(page) || !may_enter_fs) { /* * This is slightly racy - end_page_writeback() @@ -1821,7 +1819,7 @@ static int too_many_isolated(struct pgli if (current_is_kswapd()) return 0; - if (!sane_reclaim(sc)) + if (!writeback_throttling_sane(sc)) return 0; if (file) { @@ -1971,7 +1969,7 @@ shrink_inactive_list(unsigned long nr_to reclaim_stat->recent_scanned[file] += nr_taken; item = current_is_kswapd() ? PGSCAN_KSWAPD : PGSCAN_DIRECT; - if (global_reclaim(sc)) + if (!cgroup_reclaim(sc)) __count_vm_events(item, nr_scanned); __count_memcg_events(lruvec_memcg(lruvec), item, nr_scanned); spin_unlock_irq(&pgdat->lru_lock); @@ -1985,7 +1983,7 @@ shrink_inactive_list(unsigned long nr_to spin_lock_irq(&pgdat->lru_lock); item = current_is_kswapd() ? PGSTEAL_KSWAPD : PGSTEAL_DIRECT; - if (global_reclaim(sc)) + if (!cgroup_reclaim(sc)) __count_vm_events(item, nr_reclaimed); __count_memcg_events(lruvec_memcg(lruvec), item, nr_reclaimed); reclaim_stat->recent_rotated[0] += stat.nr_activate[0]; @@ -2309,7 +2307,7 @@ static void get_scan_count(struct lruvec * using the memory controller's swap limit feature would be * too expensive. */ - if (!global_reclaim(sc) && !swappiness) { + if (cgroup_reclaim(sc) && !swappiness) { scan_balance = SCAN_FILE; goto out; } @@ -2333,7 +2331,7 @@ static void get_scan_count(struct lruvec * thrashing file LRU becomes infinitely more attractive than * anon pages. Try to detect this based on file LRU size. */ - if (global_reclaim(sc)) { + if (!cgroup_reclaim(sc)) { unsigned long pgdatfile; unsigned long pgdatfree; int z; @@ -2564,7 +2562,7 @@ static void shrink_node_memcg(struct pgl * abort proportional reclaim if either the file or anon lru has already * dropped to zero at the first pass. */ - scan_adjusted = (global_reclaim(sc) && !current_is_kswapd() && + scan_adjusted = (!cgroup_reclaim(sc) && !current_is_kswapd() && sc->priority == DEF_PRIORITY); blk_start_plug(&plug); @@ -2853,7 +2851,7 @@ static bool shrink_node(pg_data_t *pgdat * Legacy memcg will stall in page writeback so avoid forcibly * stalling in wait_iff_congested(). */ - if (!global_reclaim(sc) && sane_reclaim(sc) && + if (cgroup_reclaim(sc) && writeback_throttling_sane(sc) && sc->nr.dirty && sc->nr.dirty == sc->nr.congested) set_memcg_congestion(pgdat, root, true); @@ -2948,7 +2946,7 @@ static void shrink_zones(struct zonelist * Take care memory controller reclaiming has small influence * to global LRU. */ - if (global_reclaim(sc)) { + if (!cgroup_reclaim(sc)) { if (!cpuset_zone_allowed(zone, GFP_KERNEL | __GFP_HARDWALL)) continue; @@ -3048,7 +3046,7 @@ static unsigned long do_try_to_free_page retry: delayacct_freepages_start(); - if (global_reclaim(sc)) + if (!cgroup_reclaim(sc)) __count_zid_vm_events(ALLOCSTALL, sc->reclaim_idx, 1); do { From patchwork Sun Dec 1 01:55:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268393 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7959E921 for ; Sun, 1 Dec 2019 01:55:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2C6BE22464 for ; Sun, 1 Dec 2019 01:55:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="AVtUfTqm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2C6BE22464 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E82FF6B0332; Sat, 30 Nov 2019 20:55:45 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E0D806B0334; Sat, 30 Nov 2019 20:55:45 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CAE686B0335; Sat, 30 Nov 2019 20:55:45 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0194.hostedemail.com [216.40.44.194]) by kanga.kvack.org (Postfix) with ESMTP id AAA096B0332 for ; Sat, 30 Nov 2019 20:55:45 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 5819A2C7C for ; Sun, 1 Dec 2019 01:55:45 +0000 (UTC) X-FDA: 76214906250.17.hook81_3233c15ccb460 X-Spam-Summary: 2,0,0,f178ad9f6621dea7,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:guro@fb.com:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:shakeelb@google.com:torvalds@linux-foundation.org,RULES_HIT:1:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1605:1730:1747:1777:1792:2198:2199:2393:2525:2553:2559:2563:2636:2682:2685:2693:2859:2897:2898:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6261:6653:7576:7875:7903:8599:9025:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12291:12295:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13161:13229:13846:14096:21080:21324:21433:21451:21627:21939:30012:30034:30041:30054:30064:30070:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0, LFtime:4 X-HE-Tag: hook81_3233c15ccb460 X-Filterd-Recvd-Size: 11543 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:44 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id F201C206E1; Sun, 1 Dec 2019 01:55:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165344; bh=CaEIeLMV54Eysk4jj3o85Ep+uFgk632oe8BzkQxNSIM=; h=Date:From:To:Subject:From; b=AVtUfTqmn1A7ojlTNhm1vhi2nmVWtxhU87YVzl65D2dFG9Wye7BSXAjaVZkv2Z1Fn JlEalyWkxFkguuegYyQ56mM+VBaqZXbAZk+5xQ/Vo6+pSnUCcDu+fJrjaEtCkWTN7C kceUey5/UHUrd6JTcmbJjV4SNpGqMMbuQQF5MGMU= Date: Sat, 30 Nov 2019 17:55:43 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 108/158] mm: vmscan: replace shrink_node() loop with a retry jump Message-ID: <20191201015543.MkXRt-zxY%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: mm: vmscan: replace shrink_node() loop with a retry jump Most of the function body is inside a loop, which imposes an additional indentation and scoping level that makes the code a bit hard to follow and modify. The looping only happens in case of reclaim-compaction, which isn't the common case. So rather than adding yet another function level to the reclaim path and have every reclaim invocation go through a level that only exists for one specific cornercase, use a retry goto. Link: http://lkml.kernel.org/r/20191022144803.302233-6-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Reviewed-by: Roman Gushchin Reviewed-by: Shakeel Butt Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/vmscan.c | 237 ++++++++++++++++++++++++-------------------------- 1 file changed, 118 insertions(+), 119 deletions(-) --- a/mm/vmscan.c~mm-vmscan-replace-shrink_node-loop-with-a-retry-jump +++ a/mm/vmscan.c @@ -2729,144 +2729,143 @@ static bool pgdat_memcg_congested(pg_dat static bool shrink_node(pg_data_t *pgdat, struct scan_control *sc) { struct reclaim_state *reclaim_state = current->reclaim_state; + struct mem_cgroup *root = sc->target_mem_cgroup; unsigned long nr_reclaimed, nr_scanned; bool reclaimable = false; + struct mem_cgroup *memcg; +again: + memset(&sc->nr, 0, sizeof(sc->nr)); - do { - struct mem_cgroup *root = sc->target_mem_cgroup; - struct mem_cgroup *memcg; - - memset(&sc->nr, 0, sizeof(sc->nr)); - - nr_reclaimed = sc->nr_reclaimed; - nr_scanned = sc->nr_scanned; - - memcg = mem_cgroup_iter(root, NULL, NULL); - do { - unsigned long reclaimed; - unsigned long scanned; - - switch (mem_cgroup_protected(root, memcg)) { - case MEMCG_PROT_MIN: - /* - * Hard protection. - * If there is no reclaimable memory, OOM. - */ - continue; - case MEMCG_PROT_LOW: - /* - * Soft protection. - * Respect the protection only as long as - * there is an unprotected supply - * of reclaimable memory from other cgroups. - */ - if (!sc->memcg_low_reclaim) { - sc->memcg_low_skipped = 1; - continue; - } - memcg_memory_event(memcg, MEMCG_LOW); - break; - case MEMCG_PROT_NONE: - /* - * All protection thresholds breached. We may - * still choose to vary the scan pressure - * applied based on by how much the cgroup in - * question has exceeded its protection - * thresholds (see get_scan_count). - */ - break; - } + nr_reclaimed = sc->nr_reclaimed; + nr_scanned = sc->nr_scanned; - reclaimed = sc->nr_reclaimed; - scanned = sc->nr_scanned; - shrink_node_memcg(pgdat, memcg, sc); - - shrink_slab(sc->gfp_mask, pgdat->node_id, memcg, - sc->priority); - - /* Record the group's reclaim efficiency */ - vmpressure(sc->gfp_mask, memcg, false, - sc->nr_scanned - scanned, - sc->nr_reclaimed - reclaimed); - - } while ((memcg = mem_cgroup_iter(root, memcg, NULL))); - - if (reclaim_state) { - sc->nr_reclaimed += reclaim_state->reclaimed_slab; - reclaim_state->reclaimed_slab = 0; - } - - /* Record the subtree's reclaim efficiency */ - vmpressure(sc->gfp_mask, sc->target_mem_cgroup, true, - sc->nr_scanned - nr_scanned, - sc->nr_reclaimed - nr_reclaimed); - - if (sc->nr_reclaimed - nr_reclaimed) - reclaimable = true; + memcg = mem_cgroup_iter(root, NULL, NULL); + do { + unsigned long reclaimed; + unsigned long scanned; - if (current_is_kswapd()) { + switch (mem_cgroup_protected(root, memcg)) { + case MEMCG_PROT_MIN: /* - * If reclaim is isolating dirty pages under writeback, - * it implies that the long-lived page allocation rate - * is exceeding the page laundering rate. Either the - * global limits are not being effective at throttling - * processes due to the page distribution throughout - * zones or there is heavy usage of a slow backing - * device. The only option is to throttle from reclaim - * context which is not ideal as there is no guarantee - * the dirtying process is throttled in the same way - * balance_dirty_pages() manages. - * - * Once a node is flagged PGDAT_WRITEBACK, kswapd will - * count the number of pages under pages flagged for - * immediate reclaim and stall if any are encountered - * in the nr_immediate check below. + * Hard protection. + * If there is no reclaimable memory, OOM. */ - if (sc->nr.writeback && sc->nr.writeback == sc->nr.taken) - set_bit(PGDAT_WRITEBACK, &pgdat->flags); - + continue; + case MEMCG_PROT_LOW: /* - * Tag a node as congested if all the dirty pages - * scanned were backed by a congested BDI and - * wait_iff_congested will stall. + * Soft protection. + * Respect the protection only as long as + * there is an unprotected supply + * of reclaimable memory from other cgroups. */ - if (sc->nr.dirty && sc->nr.dirty == sc->nr.congested) - set_bit(PGDAT_CONGESTED, &pgdat->flags); - - /* Allow kswapd to start writing pages during reclaim.*/ - if (sc->nr.unqueued_dirty == sc->nr.file_taken) - set_bit(PGDAT_DIRTY, &pgdat->flags); - + if (!sc->memcg_low_reclaim) { + sc->memcg_low_skipped = 1; + continue; + } + memcg_memory_event(memcg, MEMCG_LOW); + break; + case MEMCG_PROT_NONE: /* - * If kswapd scans pages marked marked for immediate - * reclaim and under writeback (nr_immediate), it - * implies that pages are cycling through the LRU - * faster than they are written so also forcibly stall. + * All protection thresholds breached. We may + * still choose to vary the scan pressure + * applied based on by how much the cgroup in + * question has exceeded its protection + * thresholds (see get_scan_count). */ - if (sc->nr.immediate) - congestion_wait(BLK_RW_ASYNC, HZ/10); + break; } + reclaimed = sc->nr_reclaimed; + scanned = sc->nr_scanned; + shrink_node_memcg(pgdat, memcg, sc); + + shrink_slab(sc->gfp_mask, pgdat->node_id, memcg, + sc->priority); + + /* Record the group's reclaim efficiency */ + vmpressure(sc->gfp_mask, memcg, false, + sc->nr_scanned - scanned, + sc->nr_reclaimed - reclaimed); + + } while ((memcg = mem_cgroup_iter(root, memcg, NULL))); + + if (reclaim_state) { + sc->nr_reclaimed += reclaim_state->reclaimed_slab; + reclaim_state->reclaimed_slab = 0; + } + + /* Record the subtree's reclaim efficiency */ + vmpressure(sc->gfp_mask, sc->target_mem_cgroup, true, + sc->nr_scanned - nr_scanned, + sc->nr_reclaimed - nr_reclaimed); + + if (sc->nr_reclaimed - nr_reclaimed) + reclaimable = true; + + if (current_is_kswapd()) { + /* + * If reclaim is isolating dirty pages under writeback, + * it implies that the long-lived page allocation rate + * is exceeding the page laundering rate. Either the + * global limits are not being effective at throttling + * processes due to the page distribution throughout + * zones or there is heavy usage of a slow backing + * device. The only option is to throttle from reclaim + * context which is not ideal as there is no guarantee + * the dirtying process is throttled in the same way + * balance_dirty_pages() manages. + * + * Once a node is flagged PGDAT_WRITEBACK, kswapd will + * count the number of pages under pages flagged for + * immediate reclaim and stall if any are encountered + * in the nr_immediate check below. + */ + if (sc->nr.writeback && sc->nr.writeback == sc->nr.taken) + set_bit(PGDAT_WRITEBACK, &pgdat->flags); + /* - * Legacy memcg will stall in page writeback so avoid forcibly - * stalling in wait_iff_congested(). + * Tag a node as congested if all the dirty pages + * scanned were backed by a congested BDI and + * wait_iff_congested will stall. */ - if (cgroup_reclaim(sc) && writeback_throttling_sane(sc) && - sc->nr.dirty && sc->nr.dirty == sc->nr.congested) - set_memcg_congestion(pgdat, root, true); + if (sc->nr.dirty && sc->nr.dirty == sc->nr.congested) + set_bit(PGDAT_CONGESTED, &pgdat->flags); + + /* Allow kswapd to start writing pages during reclaim.*/ + if (sc->nr.unqueued_dirty == sc->nr.file_taken) + set_bit(PGDAT_DIRTY, &pgdat->flags); /* - * Stall direct reclaim for IO completions if underlying BDIs - * and node is congested. Allow kswapd to continue until it - * starts encountering unqueued dirty pages or cycling through - * the LRU too quickly. + * If kswapd scans pages marked marked for immediate + * reclaim and under writeback (nr_immediate), it + * implies that pages are cycling through the LRU + * faster than they are written so also forcibly stall. */ - if (!sc->hibernation_mode && !current_is_kswapd() && - current_may_throttle() && pgdat_memcg_congested(pgdat, root)) - wait_iff_congested(BLK_RW_ASYNC, HZ/10); + if (sc->nr.immediate) + congestion_wait(BLK_RW_ASYNC, HZ/10); + } - } while (should_continue_reclaim(pgdat, sc->nr_reclaimed - nr_reclaimed, - sc)); + /* + * Legacy memcg will stall in page writeback so avoid forcibly + * stalling in wait_iff_congested(). + */ + if (cgroup_reclaim(sc) && writeback_throttling_sane(sc) && + sc->nr.dirty && sc->nr.dirty == sc->nr.congested) + set_memcg_congestion(pgdat, root, true); + + /* + * Stall direct reclaim for IO completions if underlying BDIs + * and node is congested. Allow kswapd to continue until it + * starts encountering unqueued dirty pages or cycling through + * the LRU too quickly. + */ + if (!sc->hibernation_mode && !current_is_kswapd() && + current_may_throttle() && pgdat_memcg_congested(pgdat, root)) + wait_iff_congested(BLK_RW_ASYNC, HZ/10); + + if (should_continue_reclaim(pgdat, sc->nr_reclaimed - nr_reclaimed, + sc)) + goto again; /* * Kswapd gives up on balancing particular nodes after too From patchwork Sun Dec 1 01:55:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268395 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 42ABD921 for ; Sun, 1 Dec 2019 01:55:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0664A222D9 for ; Sun, 1 Dec 2019 01:55:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="A0Pn4kru" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0664A222D9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D23176B0334; Sat, 30 Nov 2019 20:55:48 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CAFF16B0336; Sat, 30 Nov 2019 20:55:48 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B74226B0337; Sat, 30 Nov 2019 20:55:48 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0048.hostedemail.com [216.40.44.48]) by kanga.kvack.org (Postfix) with ESMTP id 9E75D6B0334 for ; Sat, 30 Nov 2019 20:55:48 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 534DF824999B for ; Sun, 1 Dec 2019 01:55:48 +0000 (UTC) X-FDA: 76214906376.07.plane89_32a559b932330 X-Spam-Summary: 2,0,0,3a16f6f00b5030be,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:guro@fb.com:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:shakeelb@google.com:torvalds@linux-foundation.org,RULES_HIT:41:69:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1543:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2693:2859:2901:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4605:5007:6119:6261:6653:7576:7903:7974:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:13846:14096:14181:14721:14849:21080:21325:21451:21627:21796:21939:30036:30054:30064,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMM ARY:none X-HE-Tag: plane89_32a559b932330 X-Filterd-Recvd-Size: 4755 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:47 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 14D6F215F1; Sun, 1 Dec 2019 01:55:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165347; bh=E7C1UG6ISMVq+aAihHhTmjFUqyh2tZOpOe0ZoVlMGaY=; h=Date:From:To:Subject:From; b=A0Pn4kruMHuH0XySrBhWSIEhcmfYK+lI06/qTte4WMt1Rjsqx8GqJ9x7mAR2/lk5I BiBfbRUnFgwx41XoH+Q8B+g4hPcR55175woXV/5ygh6Q/HgCeOOhGCcFno39nJsXOl adgDanxvghFVSLNqonuBNcvm1ph5FKLt2qFwWZVM= Date: Sat, 30 Nov 2019 17:55:46 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 109/158] mm: vmscan: turn shrink_node_memcg() into shrink_lruvec() Message-ID: <20191201015546.IskJtHlaT%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: mm: vmscan: turn shrink_node_memcg() into shrink_lruvec() An lruvec holds LRU pages owned by a certain NUMA node and cgroup. Instead of awkwardly passing around a combination of a pgdat and a memcg pointer, pass down the lruvec as soon as we can look it up. Nested callers that need to access node or cgroup properties can look them them up if necessary, but there are only a few cases. Link: http://lkml.kernel.org/r/20191022144803.302233-7-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Reviewed-by: Roman Gushchin Reviewed-by: Shakeel Butt Acked-by: Michal Hocko Signed-off-by: Andrew Morton --- mm/vmscan.c | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) --- a/mm/vmscan.c~mm-vmscan-turn-shrink_node_memcg-into-shrink_lruvec +++ a/mm/vmscan.c @@ -2280,9 +2280,10 @@ enum scan_balance { * nr[0] = anon inactive pages to scan; nr[1] = anon active pages to scan * nr[2] = file inactive pages to scan; nr[3] = file active pages to scan */ -static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg, - struct scan_control *sc, unsigned long *nr) +static void get_scan_count(struct lruvec *lruvec, struct scan_control *sc, + unsigned long *nr) { + struct mem_cgroup *memcg = lruvec_memcg(lruvec); int swappiness = mem_cgroup_swappiness(memcg); struct zone_reclaim_stat *reclaim_stat = &lruvec->reclaim_stat; u64 fraction[2]; @@ -2530,13 +2531,8 @@ out: } } -/* - * This is a basic per-node page freer. Used by both kswapd and direct reclaim. - */ -static void shrink_node_memcg(struct pglist_data *pgdat, struct mem_cgroup *memcg, - struct scan_control *sc) +static void shrink_lruvec(struct lruvec *lruvec, struct scan_control *sc) { - struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); unsigned long nr[NR_LRU_LISTS]; unsigned long targets[NR_LRU_LISTS]; unsigned long nr_to_scan; @@ -2546,7 +2542,7 @@ static void shrink_node_memcg(struct pgl struct blk_plug plug; bool scan_adjusted; - get_scan_count(lruvec, memcg, sc, nr); + get_scan_count(lruvec, sc, nr); /* Record the original scan target for proportional adjustments later */ memcpy(targets, nr, sizeof(nr)); @@ -2741,6 +2737,7 @@ again: memcg = mem_cgroup_iter(root, NULL, NULL); do { + struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); unsigned long reclaimed; unsigned long scanned; @@ -2777,7 +2774,8 @@ again: reclaimed = sc->nr_reclaimed; scanned = sc->nr_scanned; - shrink_node_memcg(pgdat, memcg, sc); + + shrink_lruvec(lruvec, sc); shrink_slab(sc->gfp_mask, pgdat->node_id, memcg, sc->priority); @@ -3281,6 +3279,7 @@ unsigned long mem_cgroup_shrink_node(str pg_data_t *pgdat, unsigned long *nr_scanned) { + struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); struct scan_control sc = { .nr_to_reclaim = SWAP_CLUSTER_MAX, .target_mem_cgroup = memcg, @@ -3305,7 +3304,7 @@ unsigned long mem_cgroup_shrink_node(str * will pick up pages from other mem cgroup's as well. We hack * the priority and make it zero. */ - shrink_node_memcg(pgdat, memcg, &sc); + shrink_lruvec(lruvec, &sc); trace_mm_vmscan_memcg_softlimit_reclaim_end(sc.nr_reclaimed); From patchwork Sun Dec 1 01:55:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268397 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 73EA9921 for ; Sun, 1 Dec 2019 01:55:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 40C9D20880 for ; Sun, 1 Dec 2019 01:55:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="fZJ6eOA3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 40C9D20880 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1D8906B0336; Sat, 30 Nov 2019 20:55:52 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 160AF6B0339; Sat, 30 Nov 2019 20:55:52 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 051D96B033A; Sat, 30 Nov 2019 20:55:52 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0022.hostedemail.com [216.40.44.22]) by kanga.kvack.org (Postfix) with ESMTP id DFBD66B0336 for ; Sat, 30 Nov 2019 20:55:51 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 936102C7C for ; Sun, 1 Dec 2019 01:55:51 +0000 (UTC) X-FDA: 76214906502.27.pull45_331c752e6a94e X-Spam-Summary: 2,0,0,0f0417dd78d33200,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:guro@fb.com:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:shakeelb@google.com:torvalds@linux-foundation.org,RULES_HIT:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1535:1544:1711:1730:1747:1777:1792:1981:2194:2199:2393:2525:2559:2564:2682:2685:2859:2895:2897:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3867:3868:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6261:6653:7576:7875:8599:9025:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12696:12737:12783:12986:13161:13229:13846:14096:14181:14721:14849:21080:21324:21451:21627:21939:30054:30064,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: pull45_331c752e6a94e X-Filterd-Recvd-Size: 5414 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:51 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2CEB02084E; Sun, 1 Dec 2019 01:55:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165350; bh=u/yLwlI6UAZFdf/9z4V3yhhk7/PelwES2CBDyxsbXG8=; h=Date:From:To:Subject:From; b=fZJ6eOA3jLs//holNpXOoeVaowkppu8I1HGbtoAALrPr452Zd7uKDLBziLa5//+Vu Mznr1cBnbWlFmUx8nwoZ+Dh0tdkIqni5NWFRZ4cvKN7LvWd5LfdirUzrBNcwJGjRiL sF1hYnuGJSlRDCXETuwiLBvN0pjLmmZgxM+/U/6c= Date: Sat, 30 Nov 2019 17:55:49 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 110/158] mm: vmscan: split shrink_node() into node part and memcgs part Message-ID: <20191201015549.W0vWmPbON%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: mm: vmscan: split shrink_node() into node part and memcgs part This function is getting long and unwieldy, split out the memcg bits. The updated shrink_node() handles the generic (node) reclaim aspects: - global vmpressure notifications - writeback and congestion throttling - reclaim/compaction management - kswapd giving up on unreclaimable nodes It then calls a new shrink_node_memcgs() which handles cgroup specifics: - the cgroup tree traversal - memory.low considerations - per-cgroup slab shrinking callbacks - per-cgroup vmpressure notifications [hannes@cmpxchg.org: rename "root" to "target_memcg", per Roman] Link: http://lkml.kernel.org/r/20191025143640.GA386981@cmpxchg.org Link: http://lkml.kernel.org/r/20191022144803.302233-8-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Reviewed-by: Roman Gushchin Reviewed-by: Shakeel Butt Acked-by: Michal Hocko Signed-off-by: Andrew Morton --- mm/vmscan.c | 41 +++++++++++++++++++++++++---------------- 1 file changed, 25 insertions(+), 16 deletions(-) --- a/mm/vmscan.c~mm-vmscan-split-shrink_node-into-node-part-and-memcgs-part +++ a/mm/vmscan.c @@ -2722,26 +2722,18 @@ static bool pgdat_memcg_congested(pg_dat (memcg && memcg_congested(pgdat, memcg)); } -static bool shrink_node(pg_data_t *pgdat, struct scan_control *sc) +static void shrink_node_memcgs(pg_data_t *pgdat, struct scan_control *sc) { - struct reclaim_state *reclaim_state = current->reclaim_state; - struct mem_cgroup *root = sc->target_mem_cgroup; - unsigned long nr_reclaimed, nr_scanned; - bool reclaimable = false; + struct mem_cgroup *target_memcg = sc->target_mem_cgroup; struct mem_cgroup *memcg; -again: - memset(&sc->nr, 0, sizeof(sc->nr)); - nr_reclaimed = sc->nr_reclaimed; - nr_scanned = sc->nr_scanned; - - memcg = mem_cgroup_iter(root, NULL, NULL); + memcg = mem_cgroup_iter(target_memcg, NULL, NULL); do { struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); unsigned long reclaimed; unsigned long scanned; - switch (mem_cgroup_protected(root, memcg)) { + switch (mem_cgroup_protected(target_memcg, memcg)) { case MEMCG_PROT_MIN: /* * Hard protection. @@ -2785,7 +2777,23 @@ again: sc->nr_scanned - scanned, sc->nr_reclaimed - reclaimed); - } while ((memcg = mem_cgroup_iter(root, memcg, NULL))); + } while ((memcg = mem_cgroup_iter(target_memcg, memcg, NULL))); +} + +static bool shrink_node(pg_data_t *pgdat, struct scan_control *sc) +{ + struct reclaim_state *reclaim_state = current->reclaim_state; + struct mem_cgroup *target_memcg = sc->target_mem_cgroup; + unsigned long nr_reclaimed, nr_scanned; + bool reclaimable = false; + +again: + memset(&sc->nr, 0, sizeof(sc->nr)); + + nr_reclaimed = sc->nr_reclaimed; + nr_scanned = sc->nr_scanned; + + shrink_node_memcgs(pgdat, sc); if (reclaim_state) { sc->nr_reclaimed += reclaim_state->reclaimed_slab; @@ -2793,7 +2801,7 @@ again: } /* Record the subtree's reclaim efficiency */ - vmpressure(sc->gfp_mask, sc->target_mem_cgroup, true, + vmpressure(sc->gfp_mask, target_memcg, true, sc->nr_scanned - nr_scanned, sc->nr_reclaimed - nr_reclaimed); @@ -2849,7 +2857,7 @@ again: */ if (cgroup_reclaim(sc) && writeback_throttling_sane(sc) && sc->nr.dirty && sc->nr.dirty == sc->nr.congested) - set_memcg_congestion(pgdat, root, true); + set_memcg_congestion(pgdat, target_memcg, true); /* * Stall direct reclaim for IO completions if underlying BDIs @@ -2858,7 +2866,8 @@ again: * the LRU too quickly. */ if (!sc->hibernation_mode && !current_is_kswapd() && - current_may_throttle() && pgdat_memcg_congested(pgdat, root)) + current_may_throttle() && + pgdat_memcg_congested(pgdat, target_memcg)) wait_iff_congested(BLK_RW_ASYNC, HZ/10); if (should_continue_reclaim(pgdat, sc->nr_reclaimed - nr_reclaimed, From patchwork Sun Dec 1 01:55:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268399 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9405214B7 for ; Sun, 1 Dec 2019 01:55:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 54474205ED for ; Sun, 1 Dec 2019 01:55:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="CTtNdWco" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 54474205ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 29C976B0339; Sat, 30 Nov 2019 20:55:55 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 24C466B033B; Sat, 30 Nov 2019 20:55:55 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1144E6B033D; Sat, 30 Nov 2019 20:55:55 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0239.hostedemail.com [216.40.44.239]) by kanga.kvack.org (Postfix) with ESMTP id ECC0B6B0339 for ; Sat, 30 Nov 2019 20:55:54 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id A1A4B824999B for ; Sun, 1 Dec 2019 01:55:54 +0000 (UTC) X-FDA: 76214906628.17.fog59_338c0b8fd1a0e X-Spam-Summary: 2,0,0,5385deb21e05bc5a,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:guro@fb.com:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:shakeelb@google.com:torvalds@linux-foundation.org,RULES_HIT:2:41:69:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1535:1605:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2563:2682:2685:2693:2859:2898:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4050:4120:4250:4321:4385:4605:5007:6261:6653:7576:7903:8599:9025:9545:9592:10004:10913:11026:11232:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13161:13229:13255:13846:14096:21080:21324:21325:21451:21627:21939:30045:30054:30064:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFti me:24,LU X-HE-Tag: fog59_338c0b8fd1a0e X-Filterd-Recvd-Size: 9292 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:54 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 4BF23222D9; Sun, 1 Dec 2019 01:55:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165353; bh=AwYl/oX9nLdNmk8Gmxq3+ooH/1WbeveuuzutGQVGdJM=; h=Date:From:To:Subject:From; b=CTtNdWcoAfZqjBy/IbFJk1gyUWiK2oEnSpiJlz1eR4REIoN+RjIZKxmk6xTiZcMH5 wcawcnYbGbUWx9pct0E8wgwy1+qnGWLlhaXGAYcQCIHzO/hDqRKzNEg9o12E3S0Na4 QdyISEykLj+MUYxC1/iQgs5cvJN/qbt7KYYZ725U= Date: Sat, 30 Nov 2019 17:55:52 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 111/158] mm: vmscan: harmonize writeback congestion tracking for nodes & memcgs Message-ID: <20191201015552.IZAis3nOs%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: mm: vmscan: harmonize writeback congestion tracking for nodes & memcgs The current writeback congestion tracking has separate flags for kswapd reclaim (node level) and cgroup limit reclaim (memcg-node level). This is unnecessarily complicated: the lruvec is an existing abstraction layer for that node-memcg intersection. Introduce lruvec->flags and LRUVEC_CONGESTED. Then track that at the reclaim root level, which is either the NUMA node for global reclaim, or the cgroup-node intersection for cgroup reclaim. Link: http://lkml.kernel.org/r/20191022144803.302233-9-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Reviewed-by: Roman Gushchin Reviewed-by: Shakeel Butt Cc: Michal Hocko Signed-off-by: Andrew Morton --- include/linux/memcontrol.h | 6 +- include/linux/mmzone.h | 11 +++- mm/vmscan.c | 84 ++++++++++------------------------- 3 files changed, 37 insertions(+), 64 deletions(-) --- a/include/linux/memcontrol.h~mm-vmscan-harmonize-writeback-congestion-tracking-for-nodes-memcgs +++ a/include/linux/memcontrol.h @@ -132,9 +132,6 @@ struct mem_cgroup_per_node { unsigned long usage_in_excess;/* Set to the value by which */ /* the soft limit is exceeded*/ bool on_tree; - bool congested; /* memcg has many dirty pages */ - /* backed by a congested BDI */ - struct mem_cgroup *memcg; /* Back pointer, we cannot */ /* use container_of */ }; @@ -403,6 +400,9 @@ static inline struct lruvec *mem_cgroup_ goto out; } + if (!memcg) + memcg = root_mem_cgroup; + mz = mem_cgroup_nodeinfo(memcg, pgdat->node_id); lruvec = &mz->lruvec; out: --- a/include/linux/mmzone.h~mm-vmscan-harmonize-writeback-congestion-tracking-for-nodes-memcgs +++ a/include/linux/mmzone.h @@ -296,6 +296,12 @@ struct zone_reclaim_stat { unsigned long recent_scanned[2]; }; +enum lruvec_flags { + LRUVEC_CONGESTED, /* lruvec has many dirty pages + * backed by a congested BDI + */ +}; + struct lruvec { struct list_head lists[NR_LRU_LISTS]; struct zone_reclaim_stat reclaim_stat; @@ -303,6 +309,8 @@ struct lruvec { atomic_long_t inactive_age; /* Refaults at the time of last reclaim cycle */ unsigned long refaults; + /* Various lruvec state flags (enum lruvec_flags) */ + unsigned long flags; #ifdef CONFIG_MEMCG struct pglist_data *pgdat; #endif @@ -572,9 +580,6 @@ struct zone { } ____cacheline_internodealigned_in_smp; enum pgdat_flags { - PGDAT_CONGESTED, /* pgdat has many dirty pages backed by - * a congested BDI - */ PGDAT_DIRTY, /* reclaim scanning has recently found * many dirty file pages at the tail * of the LRU. --- a/mm/vmscan.c~mm-vmscan-harmonize-writeback-congestion-tracking-for-nodes-memcgs +++ a/mm/vmscan.c @@ -267,29 +267,6 @@ static bool writeback_throttling_sane(st #endif return false; } - -static void set_memcg_congestion(pg_data_t *pgdat, - struct mem_cgroup *memcg, - bool congested) -{ - struct mem_cgroup_per_node *mn; - - if (!memcg) - return; - - mn = mem_cgroup_nodeinfo(memcg, pgdat->node_id); - WRITE_ONCE(mn->congested, congested); -} - -static bool memcg_congested(pg_data_t *pgdat, - struct mem_cgroup *memcg) -{ - struct mem_cgroup_per_node *mn; - - mn = mem_cgroup_nodeinfo(memcg, pgdat->node_id); - return READ_ONCE(mn->congested); - -} #else static int prealloc_memcg_shrinker(struct shrinker *shrinker) { @@ -309,18 +286,6 @@ static bool writeback_throttling_sane(st { return true; } - -static inline void set_memcg_congestion(struct pglist_data *pgdat, - struct mem_cgroup *memcg, bool congested) -{ -} - -static inline bool memcg_congested(struct pglist_data *pgdat, - struct mem_cgroup *memcg) -{ - return false; - -} #endif /* @@ -2716,12 +2681,6 @@ static inline bool should_continue_recla return inactive_lru_pages > pages_for_compaction; } -static bool pgdat_memcg_congested(pg_data_t *pgdat, struct mem_cgroup *memcg) -{ - return test_bit(PGDAT_CONGESTED, &pgdat->flags) || - (memcg && memcg_congested(pgdat, memcg)); -} - static void shrink_node_memcgs(pg_data_t *pgdat, struct scan_control *sc) { struct mem_cgroup *target_memcg = sc->target_mem_cgroup; @@ -2783,10 +2742,12 @@ static void shrink_node_memcgs(pg_data_t static bool shrink_node(pg_data_t *pgdat, struct scan_control *sc) { struct reclaim_state *reclaim_state = current->reclaim_state; - struct mem_cgroup *target_memcg = sc->target_mem_cgroup; unsigned long nr_reclaimed, nr_scanned; + struct lruvec *target_lruvec; bool reclaimable = false; + target_lruvec = mem_cgroup_lruvec(sc->target_mem_cgroup, pgdat); + again: memset(&sc->nr, 0, sizeof(sc->nr)); @@ -2801,7 +2762,7 @@ again: } /* Record the subtree's reclaim efficiency */ - vmpressure(sc->gfp_mask, target_memcg, true, + vmpressure(sc->gfp_mask, sc->target_mem_cgroup, true, sc->nr_scanned - nr_scanned, sc->nr_reclaimed - nr_reclaimed); @@ -2829,14 +2790,6 @@ again: if (sc->nr.writeback && sc->nr.writeback == sc->nr.taken) set_bit(PGDAT_WRITEBACK, &pgdat->flags); - /* - * Tag a node as congested if all the dirty pages - * scanned were backed by a congested BDI and - * wait_iff_congested will stall. - */ - if (sc->nr.dirty && sc->nr.dirty == sc->nr.congested) - set_bit(PGDAT_CONGESTED, &pgdat->flags); - /* Allow kswapd to start writing pages during reclaim.*/ if (sc->nr.unqueued_dirty == sc->nr.file_taken) set_bit(PGDAT_DIRTY, &pgdat->flags); @@ -2852,12 +2805,17 @@ again: } /* + * Tag a node/memcg as congested if all the dirty pages + * scanned were backed by a congested BDI and + * wait_iff_congested will stall. + * * Legacy memcg will stall in page writeback so avoid forcibly * stalling in wait_iff_congested(). */ - if (cgroup_reclaim(sc) && writeback_throttling_sane(sc) && + if ((current_is_kswapd() || + (cgroup_reclaim(sc) && writeback_throttling_sane(sc))) && sc->nr.dirty && sc->nr.dirty == sc->nr.congested) - set_memcg_congestion(pgdat, target_memcg, true); + set_bit(LRUVEC_CONGESTED, &target_lruvec->flags); /* * Stall direct reclaim for IO completions if underlying BDIs @@ -2865,9 +2823,9 @@ again: * starts encountering unqueued dirty pages or cycling through * the LRU too quickly. */ - if (!sc->hibernation_mode && !current_is_kswapd() && - current_may_throttle() && - pgdat_memcg_congested(pgdat, target_memcg)) + if (!current_is_kswapd() && current_may_throttle() && + !sc->hibernation_mode && + test_bit(LRUVEC_CONGESTED, &target_lruvec->flags)) wait_iff_congested(BLK_RW_ASYNC, HZ/10); if (should_continue_reclaim(pgdat, sc->nr_reclaimed - nr_reclaimed, @@ -3081,8 +3039,16 @@ retry: if (zone->zone_pgdat == last_pgdat) continue; last_pgdat = zone->zone_pgdat; + snapshot_refaults(sc->target_mem_cgroup, zone->zone_pgdat); - set_memcg_congestion(last_pgdat, sc->target_mem_cgroup, false); + + if (cgroup_reclaim(sc)) { + struct lruvec *lruvec; + + lruvec = mem_cgroup_lruvec(sc->target_mem_cgroup, + zone->zone_pgdat); + clear_bit(LRUVEC_CONGESTED, &lruvec->flags); + } } delayacct_freepages_end(); @@ -3450,7 +3416,9 @@ static bool pgdat_balanced(pg_data_t *pg /* Clear pgdat state for congested, dirty or under writeback. */ static void clear_pgdat_congested(pg_data_t *pgdat) { - clear_bit(PGDAT_CONGESTED, &pgdat->flags); + struct lruvec *lruvec = mem_cgroup_lruvec(NULL, pgdat); + + clear_bit(LRUVEC_CONGESTED, &lruvec->flags); clear_bit(PGDAT_DIRTY, &pgdat->flags); clear_bit(PGDAT_WRITEBACK, &pgdat->flags); } From patchwork Sun Dec 1 01:55:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268401 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EA4BD14B7 for ; Sun, 1 Dec 2019 01:55:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A9BB62084D for ; Sun, 1 Dec 2019 01:55:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="wGihkg5t" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A9BB62084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6B9C96B033B; Sat, 30 Nov 2019 20:55:58 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 643526B033E; Sat, 30 Nov 2019 20:55:58 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 532706B033F; Sat, 30 Nov 2019 20:55:58 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0021.hostedemail.com [216.40.44.21]) by kanga.kvack.org (Postfix) with ESMTP id 36A766B033B for ; Sat, 30 Nov 2019 20:55:58 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id ED3C1824999B for ; Sun, 1 Dec 2019 01:55:57 +0000 (UTC) X-FDA: 76214906754.16.camp35_340a77ba77f63 X-Spam-Summary: 2,0,0,ff67abacdc28feb1,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:aryabinin@virtuozzo.com:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:riel@surriel.com:shakeelb@google.com:surenb@google.com:torvalds@linux-foundation.org,RULES_HIT:2:41:69:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1535:1605:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2563:2682:2685:2689:2692:2693:2731:2859:2897:2898:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4051:4120:4250:4321:4385:4470:4605:5007:6119:6261:6653:7576:7875:7903:7904:8599:8603:8957:9010:9025:9036:9545:9592:10004:10241:10913:11026:11473:11658:11914:12043:12048:12291:12295:12296:12297:12438:12517:12519:12555:12663:12679:12683:12783:12986:13161:13172:13180:13229:13846:13870:21080:21450:21451:21627:21740:21790:21795:21939:30012:30051:30 054:3007 X-HE-Tag: camp35_340a77ba77f63 X-Filterd-Recvd-Size: 9467 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:55:57 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9BF12205ED; Sun, 1 Dec 2019 01:55:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165357; bh=JUkc753ioEqpAX9oyBs9UZGZ8T9Y4gB0yoVtI26WD00=; h=Date:From:To:Subject:From; b=wGihkg5t9o5Y/WVI7J9yluDjN/otFm6tbkyiaKedUezmsxdwNiQRUhQzrjQnAMamS HgsGbhHD+Wu0F+8sK3PCwQEdkCKYcO2CepJvVqDobmm/nF6ivb67zEa5Ot19nxky9f ewDMGHeV662mCp/M/sFWYAfBnCqE0HfkKGp8bYHI= Date: Sat, 30 Nov 2019 17:55:56 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, aryabinin@virtuozzo.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, riel@surriel.com, shakeelb@google.com, surenb@google.com, torvalds@linux-foundation.org Subject: [patch 112/158] mm: vmscan: move file exhaustion detection to the node level Message-ID: <20191201015556.HzCjHml6V%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: mm: vmscan: move file exhaustion detection to the node level Patch series "mm: fix page aging across multiple cgroups". When applications are put into unconfigured cgroups for memory accounting purposes, the cgrouping itself should not change the behavior of the page reclaim code. We expect the VM to reclaim the coldest pages in the system. But right now the VM can reclaim hot pages in one cgroup while there is eligible cold cache in others. This is because one part of the reclaim algorithm isn't truly cgroup hierarchy aware: the inactive/active list balancing. That is the part that is supposed to protect hot cache data from one-off streaming IO. The recursive cgroup reclaim scheme will scan and rotate the physical LRU lists of each eligible cgroup at the same rate in a round-robin fashion, thereby establishing a relative order among the pages of all those cgroups. However, the inactive/active balancing decisions are made locally within each cgroup, so when a cgroup is running low on cold pages, its hot pages will get reclaimed - even when sibling cgroups have plenty of cold cache eligible in the same reclaim run. For example: [root@ham ~]# head -n1 /proc/meminfo MemTotal: 1016336 kB [root@ham ~]# ./reclaimtest2.sh Establishing 50M active files in cgroup A... Hot pages cached: 12800/12800 workingset-a Linearly scanning through 18G of file data in cgroup B: real 0m4.269s user 0m0.051s sys 0m4.182s Hot pages cached: 134/12800 workingset-a The streaming IO in B, which doesn't benefit from caching at all, pushes out most of the workingset in A. Solution This series fixes the problem by elevating inactive/active balancing decisions to the toplevel of the reclaim run. This is either a cgroup that hit its limit, or straight-up global reclaim if there is physical memory pressure. From there, it takes a recursive view of the cgroup subtree to decide whether page deactivation is necessary. In the test above, the VM will then recognize that cgroup B has plenty of eligible cold cache, and that the hot pages in A can be spared: [root@ham ~]# ./reclaimtest2.sh Establishing 50M active files in cgroup A... Hot pages cached: 12800/12800 workingset-a Linearly scanning through 18G of file data in cgroup B: real 0m4.244s user 0m0.064s sys 0m4.177s Hot pages cached: 12800/12800 workingset-a Implementation Whether active pages can be deactivated or not is influenced by two factors: the inactive list dropping below a minimum size relative to the active list, and the occurence of refaults. This patch series first moves refault detection to the reclaim root, then enforces the minimum inactive size based on a recursive view of the cgroup tree's LRUs. History Note that this actually never worked correctly in Linux cgroups. In the past it worked for global reclaim and leaf limit reclaim only (we used to have two physical LRU linkages per page), but it never worked for intermediate limit reclaim over multiple leaf cgroups. We're noticing this now because 1) we're putting everything into cgroups for accounting, not just the things we want to control and 2) we're moving away from leaf limits that invoke reclaim on individual cgroups, toward large tree reclaim, triggered by high-level limits, or physical memory pressure that is influenced by local protections such as memory.low and memory.min instead. This patch (of 3): When file pages are lower than the watermark on a node, we try to force scan anonymous pages to counter-act the balancing algorithms preference for new file pages when they are likely thrashing. This is a node-level decision, but it's currently made each time we look at an lruvec. This is unnecessarily expensive and also a layering violation that makes the code harder to understand. Clean this up by making the check once per node and setting a flag in the scan_control. Link: http://lkml.kernel.org/r/20191107205334.158354-2-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Reviewed-by: Shakeel Butt Reviewed-by: Suren Baghdasaryan Cc: Andrey Ryabinin Cc: Michal Hocko Cc: Rik van Riel Signed-off-by: Andrew Morton --- mm/vmscan.c | 82 ++++++++++++++++++++++++++------------------------ 1 file changed, 43 insertions(+), 39 deletions(-) --- a/mm/vmscan.c~mm-vmscan-move-file-exhaustion-detection-to-the-node-level +++ a/mm/vmscan.c @@ -101,6 +101,9 @@ struct scan_control { /* One of the zones is ready for compaction */ unsigned int compaction_ready:1; + /* The file pages on the current node are dangerously low */ + unsigned int file_is_tiny:1; + /* Allocation order */ s8 order; @@ -2289,45 +2292,16 @@ static void get_scan_count(struct lruvec } /* - * Prevent the reclaimer from falling into the cache trap: as - * cache pages start out inactive, every cache fault will tip - * the scan balance towards the file LRU. And as the file LRU - * shrinks, so does the window for rotation from references. - * This means we have a runaway feedback loop where a tiny - * thrashing file LRU becomes infinitely more attractive than - * anon pages. Try to detect this based on file LRU size. - */ - if (!cgroup_reclaim(sc)) { - unsigned long pgdatfile; - unsigned long pgdatfree; - int z; - unsigned long total_high_wmark = 0; - - pgdatfree = sum_zone_node_page_state(pgdat->node_id, NR_FREE_PAGES); - pgdatfile = node_page_state(pgdat, NR_ACTIVE_FILE) + - node_page_state(pgdat, NR_INACTIVE_FILE); - - for (z = 0; z < MAX_NR_ZONES; z++) { - struct zone *zone = &pgdat->node_zones[z]; - if (!managed_zone(zone)) - continue; - - total_high_wmark += high_wmark_pages(zone); - } - - if (unlikely(pgdatfile + pgdatfree <= total_high_wmark)) { - /* - * Force SCAN_ANON if there are enough inactive - * anonymous pages on the LRU in eligible zones. - * Otherwise, the small LRU gets thrashed. - */ - if (!inactive_list_is_low(lruvec, false, sc, false) && - lruvec_lru_size(lruvec, LRU_INACTIVE_ANON, sc->reclaim_idx) - >> sc->priority) { - scan_balance = SCAN_ANON; - goto out; - } - } + * If the system is almost out of file pages, force-scan anon. + * But only if there are enough inactive anonymous pages on + * the LRU. Otherwise, the small LRU gets thrashed. + */ + if (sc->file_is_tiny && + !inactive_list_is_low(lruvec, false, sc, false) && + lruvec_lru_size(lruvec, LRU_INACTIVE_ANON, + sc->reclaim_idx) >> sc->priority) { + scan_balance = SCAN_ANON; + goto out; } /* @@ -2754,6 +2728,36 @@ again: nr_reclaimed = sc->nr_reclaimed; nr_scanned = sc->nr_scanned; + /* + * Prevent the reclaimer from falling into the cache trap: as + * cache pages start out inactive, every cache fault will tip + * the scan balance towards the file LRU. And as the file LRU + * shrinks, so does the window for rotation from references. + * This means we have a runaway feedback loop where a tiny + * thrashing file LRU becomes infinitely more attractive than + * anon pages. Try to detect this based on file LRU size. + */ + if (!cgroup_reclaim(sc)) { + unsigned long file; + unsigned long free; + int z; + unsigned long total_high_wmark = 0; + + free = sum_zone_node_page_state(pgdat->node_id, NR_FREE_PAGES); + file = node_page_state(pgdat, NR_ACTIVE_FILE) + + node_page_state(pgdat, NR_INACTIVE_FILE); + + for (z = 0; z < MAX_NR_ZONES; z++) { + struct zone *zone = &pgdat->node_zones[z]; + if (!managed_zone(zone)) + continue; + + total_high_wmark += high_wmark_pages(zone); + } + + sc->file_is_tiny = file + free <= total_high_wmark; + } + shrink_node_memcgs(pgdat, sc); if (reclaim_state) { From patchwork Sun Dec 1 01:55:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268403 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 773CB921 for ; Sun, 1 Dec 2019 01:56:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 28CB92084D for ; Sun, 1 Dec 2019 01:56:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="pUAS8l/E" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 28CB92084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DBEFC6B033E; Sat, 30 Nov 2019 20:56:01 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D47866B0340; Sat, 30 Nov 2019 20:56:01 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C0EC86B0341; Sat, 30 Nov 2019 20:56:01 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0070.hostedemail.com [216.40.44.70]) by kanga.kvack.org (Postfix) with ESMTP id A17A26B033E for ; Sat, 30 Nov 2019 20:56:01 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 54F232C0C for ; Sun, 1 Dec 2019 01:56:01 +0000 (UTC) X-FDA: 76214906922.17.sleep71_34847e8376f15 X-Spam-Summary: 2,0,0,4909f5beb1722799,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:aryabinin@virtuozzo.com:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:riel@surriel.com:shakeelb@google.com:surenb@google.com:torvalds@linux-foundation.org,RULES_HIT:1:41:355:379:800:960:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1605:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2564:2637:2682:2685:2693:2731:2859:2892:2897:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3308:3608:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4605:5007:6261:6653:7576:7875:8599:8603:8957:9010:9025:9036:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12696:12737:12783:12986:13161:13215:13229:13846:14096:21080:21324:21325:21433:21451:21611:21627:21740:21939:21966:21972:30004:30054:30075,0,RBL:error,CacheIP :none,Ba X-HE-Tag: sleep71_34847e8376f15 X-Filterd-Recvd-Size: 12952 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf50.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:00 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id CBD9121774; Sun, 1 Dec 2019 01:55:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165360; bh=nzlJCFn2Dx0dYQLiKaWl6qdKPfprNDO4KCbOY+W+xbg=; h=Date:From:To:Subject:From; b=pUAS8l/E73+z0oIVF83qm+BfH2tl1mbJRPjA51gkMR4Hs+QYnlheO2pGaw7oEMLV4 9/FdN5tz/aKLr/G805K02+/3xln7ddpc00n9c+Hek8QU5Pu/OKo8ZQmBN0VKuLtGg4 xNMKroxEqTcuxUgwPQevO8eFySqRrkdHbFMuA0/M= Date: Sat, 30 Nov 2019 17:55:59 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, aryabinin@virtuozzo.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, riel@surriel.com, shakeelb@google.com, surenb@google.com, torvalds@linux-foundation.org Subject: [patch 113/158] mm: vmscan: detect file thrashing at the reclaim root Message-ID: <20191201015559.rp-zSepME%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: mm: vmscan: detect file thrashing at the reclaim root We use refault information to determine whether the cache workingset is stable or transitioning, and dynamically adjust the inactive:active file LRU ratio so as to maximize protection from one-off cache during stable periods, and minimize IO during transitions. With cgroups and their nested LRU lists, we currently don't do this correctly. While recursive cgroup reclaim establishes a relative LRU order among the pages of all involved cgroups, refaults only affect the local LRU order in the cgroup in which they are occuring. As a result, cache transitions can take longer in a cgrouped system as the active pages of sibling cgroups aren't challenged when they should be. [ Right now, this is somewhat theoretical, because the siblings, under continued regular reclaim pressure, should eventually run out of inactive pages - and since inactive:active *size* balancing is also done on a cgroup-local level, we will challenge the active pages eventually in most cases. But the next patch will move that relative size enforcement to the reclaim root as well, and then this patch here will be necessary to propagate refault pressure to siblings. ] This patch moves refault detection to the root of reclaim. Instead of remembering the cgroup owner of an evicted page, remember the cgroup that caused the reclaim to happen. When refaults later occur, they'll correctly influence the cross-cgroup LRU order that reclaim follows. I.e. if global reclaim kicked out pages in some subgroup A/B/C, the refault of those pages will challenge the global LRU order, and not just the local order down inside C. [hannes@cmpxchg.org: use page_memcg() instead of another lookup] Link: http://lkml.kernel.org/r/20191115160722.GA309754@cmpxchg.org Link: http://lkml.kernel.org/r/20191107205334.158354-3-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Reviewed-by: Suren Baghdasaryan Cc: Andrey Ryabinin Cc: Michal Hocko Cc: Rik van Riel Cc: Shakeel Butt Signed-off-by: Andrew Morton --- include/linux/memcontrol.h | 5 ++ include/linux/swap.h | 2 - mm/vmscan.c | 32 +++++++--------- mm/workingset.c | 67 +++++++++++++++++++++++++++-------- 4 files changed, 73 insertions(+), 33 deletions(-) --- a/include/linux/memcontrol.h~mm-vmscan-detect-file-thrashing-at-the-reclaim-root +++ a/include/linux/memcontrol.h @@ -901,6 +901,11 @@ static inline struct lruvec *mem_cgroup_ return &pgdat->__lruvec; } +static inline struct mem_cgroup *parent_mem_cgroup(struct mem_cgroup *memcg) +{ + return NULL; +} + static inline bool mm_match_cgroup(struct mm_struct *mm, struct mem_cgroup *memcg) { --- a/include/linux/swap.h~mm-vmscan-detect-file-thrashing-at-the-reclaim-root +++ a/include/linux/swap.h @@ -307,7 +307,7 @@ struct vma_swap_readahead { }; /* linux/mm/workingset.c */ -void *workingset_eviction(struct page *page); +void *workingset_eviction(struct page *page, struct mem_cgroup *target_memcg); void workingset_refault(struct page *page, void *shadow); void workingset_activation(struct page *page); --- a/mm/vmscan.c~mm-vmscan-detect-file-thrashing-at-the-reclaim-root +++ a/mm/vmscan.c @@ -853,7 +853,7 @@ static pageout_t pageout(struct page *pa * gets returned with a refcount of 0. */ static int __remove_mapping(struct address_space *mapping, struct page *page, - bool reclaimed) + bool reclaimed, struct mem_cgroup *target_memcg) { unsigned long flags; int refcount; @@ -925,7 +925,7 @@ static int __remove_mapping(struct addre */ if (reclaimed && page_is_file_cache(page) && !mapping_exiting(mapping) && !dax_mapping(mapping)) - shadow = workingset_eviction(page); + shadow = workingset_eviction(page, target_memcg); __delete_from_page_cache(page, shadow); xa_unlock_irqrestore(&mapping->i_pages, flags); @@ -948,7 +948,7 @@ cannot_free: */ int remove_mapping(struct address_space *mapping, struct page *page) { - if (__remove_mapping(mapping, page, false)) { + if (__remove_mapping(mapping, page, false, NULL)) { /* * Unfreezing the refcount with 1 rather than 2 effectively * drops the pagecache ref for us without requiring another @@ -1426,7 +1426,8 @@ static unsigned long shrink_page_list(st count_vm_event(PGLAZYFREED); count_memcg_page_event(page, PGLAZYFREED); - } else if (!mapping || !__remove_mapping(mapping, page, true)) + } else if (!mapping || !__remove_mapping(mapping, page, true, + sc->target_mem_cgroup)) goto keep_locked; unlock_page(page); @@ -2189,6 +2190,7 @@ static bool inactive_list_is_low(struct enum lru_list inactive_lru = file * LRU_FILE; unsigned long inactive, active; unsigned long inactive_ratio; + struct lruvec *target_lruvec; unsigned long refaults; unsigned long gb; @@ -2200,8 +2202,9 @@ static bool inactive_list_is_low(struct * is being established. Disable active list protection to get * rid of the stale workingset quickly. */ - refaults = lruvec_page_state_local(lruvec, WORKINGSET_ACTIVATE); - if (file && lruvec->refaults != refaults) { + target_lruvec = mem_cgroup_lruvec(sc->target_mem_cgroup, pgdat); + refaults = lruvec_page_state(target_lruvec, WORKINGSET_ACTIVATE); + if (file && target_lruvec->refaults != refaults) { inactive_ratio = 0; } else { gb = (inactive + active) >> (30 - PAGE_SHIFT); @@ -2973,19 +2976,14 @@ static void shrink_zones(struct zonelist sc->gfp_mask = orig_mask; } -static void snapshot_refaults(struct mem_cgroup *root_memcg, pg_data_t *pgdat) +static void snapshot_refaults(struct mem_cgroup *target_memcg, pg_data_t *pgdat) { - struct mem_cgroup *memcg; - - memcg = mem_cgroup_iter(root_memcg, NULL, NULL); - do { - unsigned long refaults; - struct lruvec *lruvec; + struct lruvec *target_lruvec; + unsigned long refaults; - lruvec = mem_cgroup_lruvec(memcg, pgdat); - refaults = lruvec_page_state_local(lruvec, WORKINGSET_ACTIVATE); - lruvec->refaults = refaults; - } while ((memcg = mem_cgroup_iter(root_memcg, memcg, NULL))); + target_lruvec = mem_cgroup_lruvec(target_memcg, pgdat); + refaults = lruvec_page_state(target_lruvec, WORKINGSET_ACTIVATE); + target_lruvec->refaults = refaults; } /* --- a/mm/workingset.c~mm-vmscan-detect-file-thrashing-at-the-reclaim-root +++ a/mm/workingset.c @@ -213,28 +213,53 @@ static void unpack_shadow(void *shadow, *workingsetp = workingset; } +static void advance_inactive_age(struct mem_cgroup *memcg, pg_data_t *pgdat) +{ + /* + * Reclaiming a cgroup means reclaiming all its children in a + * round-robin fashion. That means that each cgroup has an LRU + * order that is composed of the LRU orders of its child + * cgroups; and every page has an LRU position not just in the + * cgroup that owns it, but in all of that group's ancestors. + * + * So when the physical inactive list of a leaf cgroup ages, + * the virtual inactive lists of all its parents, including + * the root cgroup's, age as well. + */ + do { + struct lruvec *lruvec; + + lruvec = mem_cgroup_lruvec(memcg, pgdat); + atomic_long_inc(&lruvec->inactive_age); + } while (memcg && (memcg = parent_mem_cgroup(memcg))); +} + /** * workingset_eviction - note the eviction of a page from memory + * @target_memcg: the cgroup that is causing the reclaim * @page: the page being evicted * * Returns a shadow entry to be stored in @page->mapping->i_pages in place * of the evicted @page so that a later refault can be detected. */ -void *workingset_eviction(struct page *page) +void *workingset_eviction(struct page *page, struct mem_cgroup *target_memcg) { struct pglist_data *pgdat = page_pgdat(page); - struct mem_cgroup *memcg = page_memcg(page); - int memcgid = mem_cgroup_id(memcg); unsigned long eviction; struct lruvec *lruvec; + int memcgid; /* Page is fully exclusive and pins page->mem_cgroup */ VM_BUG_ON_PAGE(PageLRU(page), page); VM_BUG_ON_PAGE(page_count(page), page); VM_BUG_ON_PAGE(!PageLocked(page), page); - lruvec = mem_cgroup_lruvec(memcg, pgdat); - eviction = atomic_long_inc_return(&lruvec->inactive_age); + advance_inactive_age(page_memcg(page), pgdat); + + lruvec = mem_cgroup_lruvec(target_memcg, pgdat); + /* XXX: target_memcg can be NULL, go through lruvec */ + memcgid = mem_cgroup_id(lruvec_memcg(lruvec)); + eviction = atomic_long_read(&lruvec->inactive_age); return pack_shadow(memcgid, pgdat, eviction, PageWorkingset(page)); } @@ -244,10 +269,13 @@ void *workingset_eviction(struct page *p * @shadow: shadow entry of the evicted page * * Calculates and evaluates the refault distance of the previously - * evicted page in the context of the node it was allocated in. + * evicted page in the context of the node and the memcg whose memory + * pressure caused the eviction. */ void workingset_refault(struct page *page, void *shadow) { + struct mem_cgroup *eviction_memcg; + struct lruvec *eviction_lruvec; unsigned long refault_distance; struct pglist_data *pgdat; unsigned long active_file; @@ -277,12 +305,12 @@ void workingset_refault(struct page *pag * would be better if the root_mem_cgroup existed in all * configurations instead. */ - memcg = mem_cgroup_from_id(memcgid); - if (!mem_cgroup_disabled() && !memcg) + eviction_memcg = mem_cgroup_from_id(memcgid); + if (!mem_cgroup_disabled() && !eviction_memcg) goto out; - lruvec = mem_cgroup_lruvec(memcg, pgdat); - refault = atomic_long_read(&lruvec->inactive_age); - active_file = lruvec_lru_size(lruvec, LRU_ACTIVE_FILE, MAX_NR_ZONES); + eviction_lruvec = mem_cgroup_lruvec(eviction_memcg, pgdat); + refault = atomic_long_read(&eviction_lruvec->inactive_age); + active_file = lruvec_page_state(eviction_lruvec, NR_ACTIVE_FILE); /* * Calculate the refault distance @@ -302,6 +330,17 @@ void workingset_refault(struct page *pag */ refault_distance = (refault - eviction) & EVICTION_MASK; + /* + * The activation decision for this page is made at the level + * where the eviction occurred, as that is where the LRU order + * during page reclaim is being determined. + * + * However, the cgroup that will own the page is the one that + * is actually experiencing the refault event. + */ + memcg = page_memcg(page); + lruvec = mem_cgroup_lruvec(memcg, pgdat); + inc_lruvec_state(lruvec, WORKINGSET_REFAULT); /* @@ -313,7 +352,7 @@ void workingset_refault(struct page *pag goto out; SetPageActive(page); - atomic_long_inc(&lruvec->inactive_age); + advance_inactive_age(memcg, pgdat); inc_lruvec_state(lruvec, WORKINGSET_ACTIVATE); /* Page was active prior to eviction */ @@ -332,7 +371,6 @@ out: void workingset_activation(struct page *page) { struct mem_cgroup *memcg; - struct lruvec *lruvec; rcu_read_lock(); /* @@ -345,8 +383,7 @@ void workingset_activation(struct page * memcg = page_memcg_rcu(page); if (!mem_cgroup_disabled() && !memcg) goto out; - lruvec = mem_cgroup_lruvec(memcg, page_pgdat(page)); - atomic_long_inc(&lruvec->inactive_age); + advance_inactive_age(memcg, page_pgdat(page)); out: rcu_read_unlock(); } From patchwork Sun Dec 1 01:56:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268405 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D5402921 for ; Sun, 1 Dec 2019 01:56:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 870932084D for ; Sun, 1 Dec 2019 01:56:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="xcudiszf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 870932084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 24F356B0340; Sat, 30 Nov 2019 20:56:05 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1D72D6B0342; Sat, 30 Nov 2019 20:56:05 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 053EC6B0344; Sat, 30 Nov 2019 20:56:04 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0155.hostedemail.com [216.40.44.155]) by kanga.kvack.org (Postfix) with ESMTP id D82AB6B0340 for ; Sat, 30 Nov 2019 20:56:04 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 84A39181AEF0B for ; Sun, 1 Dec 2019 01:56:04 +0000 (UTC) X-FDA: 76214907048.16.drug32_34fa4c20b4711 X-Spam-Summary: 2,0,0,ebbf57d8939c0c68,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:aryabinin@virtuozzo.com:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:riel@surriel.com:shakeelb@google.com:surenb@google.com:torvalds@linux-foundation.org,RULES_HIT:1:41:69:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1605:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2564:2639:2682:2685:2690:2693:2731:2859:2892:2897:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4605:5007:6119:6261:6630:6653:7576:7875:7903:7904:8599:8603:9010:9025:9036:9391:9545:9592:10004:10128:10241:10913:11026:11233:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13215:13229:13846:14096:21080:21324:21433:21450:21451:21627:21740:21819:21939:30054:30075,0,RBL:error,CacheIP:non e,Bayesi X-HE-Tag: drug32_34fa4c20b4711 X-Filterd-Recvd-Size: 14803 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:03 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0ADB920873; Sun, 1 Dec 2019 01:56:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165363; bh=jnPU1yDW+fManwq5dnaLyxV8FFPgy9wqc9FO2+v80pU=; h=Date:From:To:Subject:From; b=xcudiszfe4fa+z8ltgZLRKBGvCr9rnlf8IG+a5687hWy1EYm4vgHL4OesbGp5ghdu WS8bpQzUeVph8zm8+qnqRCcP+OnlHRq55pkHM/0zSFaWcZmLBsQ4OqoMn1fOuOGE+O nd/2YaAKTaYSUEs/AscfO54udmf/nI0NcnicFyaA= Date: Sat, 30 Nov 2019 17:56:02 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, aryabinin@virtuozzo.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, riel@surriel.com, shakeelb@google.com, surenb@google.com, torvalds@linux-foundation.org Subject: [patch 114/158] mm: vmscan: enforce inactive:active ratio at the reclaim root Message-ID: <20191201015602.zhA4Xk0JC%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: mm: vmscan: enforce inactive:active ratio at the reclaim root We split the LRU lists into inactive and an active parts to maximize workingset protection while allowing just enough inactive cache space to faciltate readahead and writeback for one-off file accesses (e.g. a linear scan through a file, or logging); or just enough inactive anon to maintain recent reference information when reclaim needs to swap. With cgroups and their nested LRU lists, we currently don't do this correctly. While recursive cgroup reclaim establishes a relative LRU order among the pages of all involved cgroups, inactive:active size decisions are done on a per-cgroup level. As a result, we'll reclaim a cgroup's workingset when it doesn't have cold pages, even when one of its siblings has plenty of it that should be reclaimed first. For example: workload A has 50M worth of hot cache but doesn't do any one-off file accesses; meanwhile, parallel workload B scans files and rarely accesses the same page twice. If these workloads were to run in an uncgrouped system, A would be protected from the high rate of cache faults from B. But if they were put in parallel cgroups for memory accounting purposes, B's fast cache fault rate would push out the hot cache pages of A. This is unexpected and undesirable - the "scan resistance" of the page cache is broken. This patch moves inactive:active size balancing decisions to the root of reclaim - the same level where the LRU order is established. It does this by looking at the recursive size of the inactive and the active file sets of the cgroup subtree at the beginning of the reclaim cycle, and then making a decision - scan or skip active pages - that applies throughout the entire run and to every cgroup involved. With that in place, in the test above, the VM will recognize that there are plenty of inactive pages in the combined cache set of workloads A and B and prefer the one-off cache in B over the hot pages in A. The scan resistance of the cache is restored. [cai@lca.pw: fix some -Wenum-conversion warnings] Link: http://lkml.kernel.org/r/1573848697-29262-1-git-send-email-cai@lca.pw Link: http://lkml.kernel.org/r/20191107205334.158354-4-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Reviewed-by: Suren Baghdasaryan Reviewed-by: Shakeel Butt Cc: Andrey Ryabinin Cc: Rik van Riel Cc: Michal Hocko Signed-off-by: Andrew Morton --- include/linux/mmzone.h | 4 mm/vmscan.c | 185 ++++++++++++++++++++++++--------------- 2 files changed, 118 insertions(+), 71 deletions(-) --- a/include/linux/mmzone.h~mm-vmscan-enforce-inactive-active-ratio-at-the-reclaim-root +++ a/include/linux/mmzone.h @@ -273,12 +273,12 @@ enum lru_list { #define for_each_evictable_lru(lru) for (lru = 0; lru <= LRU_ACTIVE_FILE; lru++) -static inline int is_file_lru(enum lru_list lru) +static inline bool is_file_lru(enum lru_list lru) { return (lru == LRU_INACTIVE_FILE || lru == LRU_ACTIVE_FILE); } -static inline int is_active_lru(enum lru_list lru) +static inline bool is_active_lru(enum lru_list lru) { return (lru == LRU_ACTIVE_ANON || lru == LRU_ACTIVE_FILE); } --- a/mm/vmscan.c~mm-vmscan-enforce-inactive-active-ratio-at-the-reclaim-root +++ a/mm/vmscan.c @@ -79,6 +79,13 @@ struct scan_control { */ struct mem_cgroup *target_mem_cgroup; + /* Can active pages be deactivated as part of reclaim? */ +#define DEACTIVATE_ANON 1 +#define DEACTIVATE_FILE 2 + unsigned int may_deactivate:2; + unsigned int force_deactivate:1; + unsigned int skipped_deactivate:1; + /* Writepage batching in laptop mode; RECLAIM_WRITE */ unsigned int may_writepage:1; @@ -101,6 +108,9 @@ struct scan_control { /* One of the zones is ready for compaction */ unsigned int compaction_ready:1; + /* There is easily reclaimable cold cache in the current node */ + unsigned int cache_trim_mode:1; + /* The file pages on the current node are dangerously low */ unsigned int file_is_tiny:1; @@ -2154,6 +2164,20 @@ unsigned long reclaim_pages(struct list_ return nr_reclaimed; } +static unsigned long shrink_list(enum lru_list lru, unsigned long nr_to_scan, + struct lruvec *lruvec, struct scan_control *sc) +{ + if (is_active_lru(lru)) { + if (sc->may_deactivate & (1 << is_file_lru(lru))) + shrink_active_list(nr_to_scan, lruvec, sc, lru); + else + sc->skipped_deactivate = 1; + return 0; + } + + return shrink_inactive_list(nr_to_scan, lruvec, sc, lru); +} + /* * The inactive anon list should be small enough that the VM never has * to do too much work. @@ -2182,59 +2206,25 @@ unsigned long reclaim_pages(struct list_ * 1TB 101 10GB * 10TB 320 32GB */ -static bool inactive_list_is_low(struct lruvec *lruvec, bool file, - struct scan_control *sc, bool trace) +static bool inactive_is_low(struct lruvec *lruvec, enum lru_list inactive_lru) { - enum lru_list active_lru = file * LRU_FILE + LRU_ACTIVE; - struct pglist_data *pgdat = lruvec_pgdat(lruvec); - enum lru_list inactive_lru = file * LRU_FILE; + enum lru_list active_lru = inactive_lru + LRU_ACTIVE; unsigned long inactive, active; unsigned long inactive_ratio; - struct lruvec *target_lruvec; - unsigned long refaults; unsigned long gb; - inactive = lruvec_lru_size(lruvec, inactive_lru, sc->reclaim_idx); - active = lruvec_lru_size(lruvec, active_lru, sc->reclaim_idx); + inactive = lruvec_page_state(lruvec, NR_LRU_BASE + inactive_lru); + active = lruvec_page_state(lruvec, NR_LRU_BASE + active_lru); - /* - * When refaults are being observed, it means a new workingset - * is being established. Disable active list protection to get - * rid of the stale workingset quickly. - */ - target_lruvec = mem_cgroup_lruvec(sc->target_mem_cgroup, pgdat); - refaults = lruvec_page_state(target_lruvec, WORKINGSET_ACTIVATE); - if (file && target_lruvec->refaults != refaults) { - inactive_ratio = 0; - } else { - gb = (inactive + active) >> (30 - PAGE_SHIFT); - if (gb) - inactive_ratio = int_sqrt(10 * gb); - else - inactive_ratio = 1; - } - - if (trace) - trace_mm_vmscan_inactive_list_is_low(pgdat->node_id, sc->reclaim_idx, - lruvec_lru_size(lruvec, inactive_lru, MAX_NR_ZONES), inactive, - lruvec_lru_size(lruvec, active_lru, MAX_NR_ZONES), active, - inactive_ratio, file); + gb = (inactive + active) >> (30 - PAGE_SHIFT); + if (gb) + inactive_ratio = int_sqrt(10 * gb); + else + inactive_ratio = 1; return inactive * inactive_ratio < active; } -static unsigned long shrink_list(enum lru_list lru, unsigned long nr_to_scan, - struct lruvec *lruvec, struct scan_control *sc) -{ - if (is_active_lru(lru)) { - if (inactive_list_is_low(lruvec, is_file_lru(lru), sc, true)) - shrink_active_list(nr_to_scan, lruvec, sc, lru); - return 0; - } - - return shrink_inactive_list(nr_to_scan, lruvec, sc, lru); -} - enum scan_balance { SCAN_EQUAL, SCAN_FRACT, @@ -2296,28 +2286,17 @@ static void get_scan_count(struct lruvec /* * If the system is almost out of file pages, force-scan anon. - * But only if there are enough inactive anonymous pages on - * the LRU. Otherwise, the small LRU gets thrashed. */ - if (sc->file_is_tiny && - !inactive_list_is_low(lruvec, false, sc, false) && - lruvec_lru_size(lruvec, LRU_INACTIVE_ANON, - sc->reclaim_idx) >> sc->priority) { + if (sc->file_is_tiny) { scan_balance = SCAN_ANON; goto out; } /* - * If there is enough inactive page cache, i.e. if the size of the - * inactive list is greater than that of the active list *and* the - * inactive list actually has some pages to scan on this priority, we - * do not reclaim anything from the anonymous working set right now. - * Without the second condition we could end up never scanning an - * lruvec even if it has plenty of old anonymous pages unless the - * system is under heavy pressure. + * If there is enough inactive page cache, we do not reclaim + * anything from the anonymous working right now. */ - if (!inactive_list_is_low(lruvec, true, sc, false) && - lruvec_lru_size(lruvec, LRU_INACTIVE_FILE, sc->reclaim_idx) >> sc->priority) { + if (sc->cache_trim_mode) { scan_balance = SCAN_FILE; goto out; } @@ -2582,7 +2561,7 @@ static void shrink_lruvec(struct lruvec * Even if we did not try to evict anon pages at all, we want to * rebalance the anon lru active/inactive ratio. */ - if (total_swap_pages && inactive_list_is_low(lruvec, false, sc, true)) + if (total_swap_pages && inactive_is_low(lruvec, LRU_INACTIVE_ANON)) shrink_active_list(SWAP_CLUSTER_MAX, lruvec, sc, LRU_ACTIVE_ANON); } @@ -2722,6 +2701,7 @@ static bool shrink_node(pg_data_t *pgdat unsigned long nr_reclaimed, nr_scanned; struct lruvec *target_lruvec; bool reclaimable = false; + unsigned long file; target_lruvec = mem_cgroup_lruvec(sc->target_mem_cgroup, pgdat); @@ -2732,6 +2712,44 @@ again: nr_scanned = sc->nr_scanned; /* + * Target desirable inactive:active list ratios for the anon + * and file LRU lists. + */ + if (!sc->force_deactivate) { + unsigned long refaults; + + if (inactive_is_low(target_lruvec, LRU_INACTIVE_ANON)) + sc->may_deactivate |= DEACTIVATE_ANON; + else + sc->may_deactivate &= ~DEACTIVATE_ANON; + + /* + * When refaults are being observed, it means a new + * workingset is being established. Deactivate to get + * rid of any stale active pages quickly. + */ + refaults = lruvec_page_state(target_lruvec, + WORKINGSET_ACTIVATE); + if (refaults != target_lruvec->refaults || + inactive_is_low(target_lruvec, LRU_INACTIVE_FILE)) + sc->may_deactivate |= DEACTIVATE_FILE; + else + sc->may_deactivate &= ~DEACTIVATE_FILE; + } else + sc->may_deactivate = DEACTIVATE_ANON | DEACTIVATE_FILE; + + /* + * If we have plenty of inactive file pages that aren't + * thrashing, try to reclaim those first before touching + * anonymous pages. + */ + file = lruvec_page_state(target_lruvec, NR_INACTIVE_FILE); + if (file >> sc->priority && !(sc->may_deactivate & DEACTIVATE_FILE)) + sc->cache_trim_mode = 1; + else + sc->cache_trim_mode = 0; + + /* * Prevent the reclaimer from falling into the cache trap: as * cache pages start out inactive, every cache fault will tip * the scan balance towards the file LRU. And as the file LRU @@ -2741,10 +2759,9 @@ again: * anon pages. Try to detect this based on file LRU size. */ if (!cgroup_reclaim(sc)) { - unsigned long file; - unsigned long free; - int z; unsigned long total_high_wmark = 0; + unsigned long free, anon; + int z; free = sum_zone_node_page_state(pgdat->node_id, NR_FREE_PAGES); file = node_page_state(pgdat, NR_ACTIVE_FILE) + @@ -2758,7 +2775,17 @@ again: total_high_wmark += high_wmark_pages(zone); } - sc->file_is_tiny = file + free <= total_high_wmark; + /* + * Consider anon: if that's low too, this isn't a + * runaway file reclaim problem, but rather just + * extreme pressure. Reclaim as per usual then. + */ + anon = node_page_state(pgdat, NR_INACTIVE_ANON); + + sc->file_is_tiny = + file + free <= total_high_wmark && + !(sc->may_deactivate & DEACTIVATE_ANON) && + anon >> sc->priority; } shrink_node_memcgs(pgdat, sc); @@ -3062,9 +3089,27 @@ retry: if (sc->compaction_ready) return 1; + /* + * We make inactive:active ratio decisions based on the node's + * composition of memory, but a restrictive reclaim_idx or a + * memory.low cgroup setting can exempt large amounts of + * memory from reclaim. Neither of which are very common, so + * instead of doing costly eligibility calculations of the + * entire cgroup subtree up front, we assume the estimates are + * good, and retry with forcible deactivation if that fails. + */ + if (sc->skipped_deactivate) { + sc->priority = initial_priority; + sc->force_deactivate = 1; + sc->skipped_deactivate = 0; + goto retry; + } + /* Untapped cgroup reserves? Don't OOM, retry. */ if (sc->memcg_low_skipped) { sc->priority = initial_priority; + sc->force_deactivate = 0; + sc->skipped_deactivate = 0; sc->memcg_low_reclaim = 1; sc->memcg_low_skipped = 0; goto retry; @@ -3339,18 +3384,20 @@ static void age_active_anon(struct pglis struct scan_control *sc) { struct mem_cgroup *memcg; + struct lruvec *lruvec; if (!total_swap_pages) return; + lruvec = mem_cgroup_lruvec(NULL, pgdat); + if (!inactive_is_low(lruvec, LRU_INACTIVE_ANON)) + return; + memcg = mem_cgroup_iter(NULL, NULL, NULL); do { - struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); - - if (inactive_list_is_low(lruvec, false, sc, true)) - shrink_active_list(SWAP_CLUSTER_MAX, lruvec, - sc, LRU_ACTIVE_ANON); - + lruvec = mem_cgroup_lruvec(memcg, pgdat); + shrink_active_list(SWAP_CLUSTER_MAX, lruvec, + sc, LRU_ACTIVE_ANON); memcg = mem_cgroup_iter(NULL, memcg, NULL); } while (memcg); } From patchwork Sun Dec 1 01:56:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268407 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DE48314B7 for ; Sun, 1 Dec 2019 01:56:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AB6712084D for ; Sun, 1 Dec 2019 01:56:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="E3ojCv2k" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AB6712084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E86DF6B0342; Sat, 30 Nov 2019 20:56:07 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DBE176B0345; Sat, 30 Nov 2019 20:56:07 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C85C36B0346; Sat, 30 Nov 2019 20:56:07 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0142.hostedemail.com [216.40.44.142]) by kanga.kvack.org (Postfix) with ESMTP id ABF046B0342 for ; Sat, 30 Nov 2019 20:56:07 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 5D27F40DD for ; Sun, 1 Dec 2019 01:56:07 +0000 (UTC) X-FDA: 76214907174.27.rat37_356bce3fd0d40 X-Spam-Summary: 2,0,0,1e4c5d514138d080,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org::mm-commits@vger.kernel.org:torvalds@linux-foundation.org:xianting_tian@126.com,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2553:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3151:3352:3865:3866:3867:3868:3870:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6261:6653:7576:9010:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12297:12517:12519:12555:12679:12783:12986:13069:13311:13357:13846:14181:14384:14721:14777:14849:21080:21433:21451:21600:21627:21819:21939:30034:30054:30090,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: rat37_356bce3fd0d40 X-Filterd-Recvd-Size: 2276 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:06 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 19670208C3; Sun, 1 Dec 2019 01:56:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165366; bh=+U4am0HUfNyMCKPW0VWpBNeRXtBK7FnZLCo9UMg6j1o=; h=Date:From:To:Subject:From; b=E3ojCv2k3facpE0hv8jddq3F6vTaejZuuTS8VmCi0SwEY49wteIH9N5wkhr6Fnijq StRg39HqQr2qasLgoDw5Olw8+9qtJ/p+KtfS2vpe0XO9tFdIrSTCaVfseW3K23rPXY rxmPDcgr5QBmDMI0jgDbopp1z8iMaUZxYM+IzhGA= Date: Sat, 30 Nov 2019 17:56:05 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, xianting_tian@126.com Subject: [patch 115/158] mm/vmscan.c: fix typo in comment Message-ID: <20191201015605.mQo1-fuY_%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Xianting Tian Subject: mm/vmscan.c: fix typo in comment Fix the typo "resheduled" -> "rescheduled" in comment Link: http://lkml.kernel.org/r/1573486327-9591-1-git-send-email-xianting_tian@126.com Signed-off-by: Xianting Tian Reviewed-by: Andrew Morton Signed-off-by: Andrew Morton --- mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/vmscan.c~mm-vmscanc-fix-typo-in-comment +++ a/mm/vmscan.c @@ -1785,7 +1785,7 @@ int isolate_lru_page(struct page *page) /* * A direct reclaimer may isolate SWAP_CLUSTER_MAX pages from the LRU list and - * then get resheduled. When there are massive number of tasks doing page + * then get rescheduled. When there are massive number of tasks doing page * allocation, such sleeping direct reclaimers may keep piling up on each CPU, * the LRU list will go small and be scanned faster than necessary, leading to * unnecessary swapping, thrashing and OOM. From patchwork Sun Dec 1 01:56:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268409 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BEA8914B7 for ; Sun, 1 Dec 2019 01:56:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8C71922B48 for ; Sun, 1 Dec 2019 01:56:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="OMxLvpmY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8C71922B48 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E0CC96B0345; Sat, 30 Nov 2019 20:56:10 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D70606B0347; Sat, 30 Nov 2019 20:56:10 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C873F6B0348; Sat, 30 Nov 2019 20:56:10 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0245.hostedemail.com [216.40.44.245]) by kanga.kvack.org (Postfix) with ESMTP id A73516B0345 for ; Sat, 30 Nov 2019 20:56:10 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 5E10F180AD817 for ; Sun, 1 Dec 2019 01:56:10 +0000 (UTC) X-FDA: 76214907300.03.waste45_35dd139899d41 X-Spam-Summary: 2,0,0,1e1d4ebdb541d754,d41d8cd98f00b204,akpm@linux-foundation.org,:adobriyan@gmail.com:akpm@linux-foundation.org:chris@chrisdown.name:david@redhat.com:hannes@cmpxchg.org::mhocko@suse.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org:vbabka@suse.cz,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2198:2199:2393:2525:2553:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6261:6653:7514:7576:8599:9025:9545:10004:10913:11026:11658:11914:12043:12048:12050:12296:12297:12438:12517:12519:12555:12679:12783:12986:13069:13311:13357:13846:14181:14384:14721:14849:21080:21451:21611:21627:21939:30045:30054:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFt ime:3,LU X-HE-Tag: waste45_35dd139899d41 X-Filterd-Recvd-Size: 3104 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:09 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 142E2208E4; Sun, 1 Dec 2019 01:56:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165369; bh=InJUVethnyqo/0NLBqjrnd9yPOnRHCtiBP9Q9ZITlVI=; h=Date:From:To:Subject:From; b=OMxLvpmYWQz5KasDL0R+yHiQxdau91ibUu4WL7A6B3PeYBbX0ZDhZ6VKUw9IZSrPG E4nLr9lxc1C47ol9zMNUxaXxoALgFCESUAehDwc1BTJXQ/Xnz6U9xIN7mSGEgdCRsp 32nBAlBsszU3W3tHkSDnU8y/RERzwwH3crDLEIRA= Date: Sat, 30 Nov 2019 17:56:08 -0800 From: akpm@linux-foundation.org To: adobriyan@gmail.com, akpm@linux-foundation.org, chris@chrisdown.name, david@redhat.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 116/158] kernel: sysctl: make drop_caches write-only Message-ID: <20191201015608.f_Wv7r1zM%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: kernel: sysctl: make drop_caches write-only Currently, the drop_caches proc file and sysctl read back the last value written, suggesting this is somehow a stateful setting instead of a one-time command. Make it write-only, like e.g. compact_memory. While mitigating a VM problem at scale in our fleet, there was confusion about whether writing to this file will permanently switch the kernel into a non-caching mode. This influences the decision making in a tense situation, where tens of people are trying to fix tens of thousands of affected machines: Do we need a rollback strategy? What are the performance implications of operating in a non-caching state for several days? It also caused confusion when the kernel team said we may need to write the file several times to make sure it's effective ("But it already reads back 3?"). Link: http://lkml.kernel.org/r/20191031221602.9375-1-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Acked-by: Chris Down Acked-by: Vlastimil Babka Acked-by: David Hildenbrand Acked-by: Michal Hocko Acked-by: Alexey Dobriyan Signed-off-by: Andrew Morton --- kernel/sysctl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/kernel/sysctl.c~kernel-sysctl-make-drop_caches-write-only +++ a/kernel/sysctl.c @@ -1466,7 +1466,7 @@ static struct ctl_table vm_table[] = { .procname = "drop_caches", .data = &sysctl_drop_caches, .maxlen = sizeof(int), - .mode = 0644, + .mode = 0200, .proc_handler = drop_caches_sysctl_handler, .extra1 = SYSCTL_ONE, .extra2 = &four, From patchwork Sun Dec 1 01:56:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268411 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 81643921 for ; Sun, 1 Dec 2019 01:56:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 33D092082E for ; Sun, 1 Dec 2019 01:56:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="WzH4ceMg" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 33D092082E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 50C096B0347; Sat, 30 Nov 2019 20:56:14 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 497656B0349; Sat, 30 Nov 2019 20:56:14 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 33A266B034A; Sat, 30 Nov 2019 20:56:14 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0158.hostedemail.com [216.40.44.158]) by kanga.kvack.org (Postfix) with ESMTP id 109E46B0347 for ; Sat, 30 Nov 2019 20:56:14 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id B072540DD for ; Sun, 1 Dec 2019 01:56:13 +0000 (UTC) X-FDA: 76214907426.05.ants14_3650744cf1515 X-Spam-Summary: 2,0,0,df419a0962ea0430,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:ddstreet@ieee.org:henrywolfeburns@gmail.com::mm-commits@vger.kernel.org:shakeelb@google.com:torvalds@linux-foundation.org:vitaly.vul@sony.com:vitaly.wool@konsulko.com,RULES_HIT:41:69:327:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1605:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2525:2553:2559:2567:2682:2685:2859:2897:2898:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3867:3868:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4605:5007:6117:6119:6261:6653:7514:7558:7576:7875:7903:8599:8603:8957:9025:9036:9121:9391:9545:9592:10004:10913:11026:11232:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13146:13161:13229:13230:13846:14096:14799:21080:21094:21323:21324:21433:21451:21627:21740:21819:21939:30054:30070:30090,0,RBL:error,C acheIP:n X-HE-Tag: ants14_3650744cf1515 X-Filterd-Recvd-Size: 22816 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:13 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 3FDAF217AB; Sun, 1 Dec 2019 01:56:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165372; bh=LWYT7VXpewV0T705aJq8LRLoWJcGA9xmWkoUPPZcl9c=; h=Date:From:To:Subject:From; b=WzH4ceMgxX5zmfjYZ8EbZcsEyRvayjrWSAir5cmv3OvoIPH/bN/WZFZig4p8aNMgh 9aQErXH/yS6lJ6tnPMRMOeyFxhrbb60SEukLOEvSFUGb4Z5lpTR5ytbj+oreeVubYz b6lENWG5baFZBkiM/DEDt2gcIT5/B2I7iFkW0/3Y= Date: Sat, 30 Nov 2019 17:56:11 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, ddstreet@ieee.org, henrywolfeburns@gmail.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org, vitaly.vul@sony.com, vitaly.wool@konsulko.com Subject: [patch 117/158] mm/z3fold.c: add inter-page compaction Message-ID: <20191201015611.fw2I5fblp%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vitaly Wool Subject: mm/z3fold.c: add inter-page compaction For each page scheduled for compaction (e. g. by z3fold_free()), try to apply inter-page compaction before running the traditional/ existing intra-page compaction. That means, if the page has only one buddy, we treat that buddy as a new object that we aim to place into an existing z3fold page. If such a page is found, that object is transferred and the old page is freed completely. The transferred object is named "foreign" and treated slightly differently thereafter. Namely, we increase "foreign handle" counter for the new page. Pages with non-zero "foreign handle" count become unmovable. This patch implements "foreign handle" detection when a handle is freed to decrement the foreign handle counter accordingly, so a page may as well become movable again as the time goes by. As a result, we almost always have exactly 3 objects per page and significantly better average compression ratio. [cai@lca.pw: fix -Wunused-but-set-variable warnings] Link: http://lkml.kernel.org/r/1570542062-29144-1-git-send-email-cai@lca.pw [vitalywool@gmail.com: avoid subtle race when freeing slots] Link: http://lkml.kernel.org/r/20191127152118.6314b99074b0626d4c5a8835@gmail.com [vitalywool@gmail.com: compact objects more accurately] Link: http://lkml.kernel.org/r/20191127152216.6ad33745a21ba71c53606acb@gmail.com [vitalywool@gmail.com: protect handle reads] Link: http://lkml.kernel.org/r/20191127152345.8059852f60947686674d726d@gmail.com Link: http://lkml.kernel.org/r/20191006041457.24113-1-vitalywool@gmail.com Signed-off-by: Vitaly Wool Cc: Dan Streetman Cc: Henry Burns Cc: Shakeel Butt Signed-off-by: Andrew Morton --- mm/z3fold.c | 375 ++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 303 insertions(+), 72 deletions(-) --- a/mm/z3fold.c~z3fold-add-inter-page-compaction +++ a/mm/z3fold.c @@ -41,6 +41,7 @@ #include #include #include +#include #include #include @@ -90,6 +91,7 @@ struct z3fold_buddy_slots { */ unsigned long slot[BUDDY_MASK + 1]; unsigned long pool; /* back link + flags */ + rwlock_t lock; }; #define HANDLE_FLAG_MASK (0x03) @@ -124,6 +126,7 @@ struct z3fold_header { unsigned short start_middle; unsigned short first_num:2; unsigned short mapped_count:2; + unsigned short foreign_handles:2; }; /** @@ -178,6 +181,19 @@ enum z3fold_page_flags { PAGE_CLAIMED, /* by either reclaim or free */ }; +/* + * handle flags, go under HANDLE_FLAG_MASK + */ +enum z3fold_handle_flags { + HANDLES_ORPHANED = 0, +}; + +/* + * Forward declarations + */ +static struct z3fold_header *__z3fold_alloc(struct z3fold_pool *, size_t, bool); +static void compact_page_work(struct work_struct *w); + /***************** * Helpers *****************/ @@ -191,8 +207,6 @@ static int size_to_chunks(size_t size) #define for_each_unbuddied_list(_iter, _begin) \ for ((_iter) = (_begin); (_iter) < NCHUNKS; (_iter)++) -static void compact_page_work(struct work_struct *w); - static inline struct z3fold_buddy_slots *alloc_slots(struct z3fold_pool *pool, gfp_t gfp) { @@ -204,6 +218,7 @@ static inline struct z3fold_buddy_slots if (slots) { memset(slots->slot, 0, sizeof(slots->slot)); slots->pool = (unsigned long)pool; + rwlock_init(&slots->lock); } return slots; @@ -219,25 +234,110 @@ static inline struct z3fold_buddy_slots return (struct z3fold_buddy_slots *)(handle & ~(SLOTS_ALIGN - 1)); } +/* Lock a z3fold page */ +static inline void z3fold_page_lock(struct z3fold_header *zhdr) +{ + spin_lock(&zhdr->page_lock); +} + +/* Try to lock a z3fold page */ +static inline int z3fold_page_trylock(struct z3fold_header *zhdr) +{ + return spin_trylock(&zhdr->page_lock); +} + +/* Unlock a z3fold page */ +static inline void z3fold_page_unlock(struct z3fold_header *zhdr) +{ + spin_unlock(&zhdr->page_lock); +} + + +static inline struct z3fold_header *__get_z3fold_header(unsigned long handle, + bool lock) +{ + struct z3fold_buddy_slots *slots; + struct z3fold_header *zhdr; + int locked = 0; + + if (!(handle & (1 << PAGE_HEADLESS))) { + slots = handle_to_slots(handle); + do { + unsigned long addr; + + read_lock(&slots->lock); + addr = *(unsigned long *)handle; + zhdr = (struct z3fold_header *)(addr & PAGE_MASK); + if (lock) + locked = z3fold_page_trylock(zhdr); + read_unlock(&slots->lock); + if (locked) + break; + cpu_relax(); + } while (lock); + } else { + zhdr = (struct z3fold_header *)(handle & PAGE_MASK); + } + + return zhdr; +} + +/* Returns the z3fold page where a given handle is stored */ +static inline struct z3fold_header *handle_to_z3fold_header(unsigned long h) +{ + return __get_z3fold_header(h, false); +} + +/* return locked z3fold page if it's not headless */ +static inline struct z3fold_header *get_z3fold_header(unsigned long h) +{ + return __get_z3fold_header(h, true); +} + +static inline void put_z3fold_header(struct z3fold_header *zhdr) +{ + struct page *page = virt_to_page(zhdr); + + if (!test_bit(PAGE_HEADLESS, &page->private)) + z3fold_page_unlock(zhdr); +} + static inline void free_handle(unsigned long handle) { struct z3fold_buddy_slots *slots; + struct z3fold_header *zhdr; int i; bool is_free; if (handle & (1 << PAGE_HEADLESS)) return; - WARN_ON(*(unsigned long *)handle == 0); - *(unsigned long *)handle = 0; + if (WARN_ON(*(unsigned long *)handle == 0)) + return; + + zhdr = handle_to_z3fold_header(handle); slots = handle_to_slots(handle); + write_lock(&slots->lock); + *(unsigned long *)handle = 0; + write_unlock(&slots->lock); + if (zhdr->slots == slots) + return; /* simple case, nothing else to do */ + + /* we are freeing a foreign handle if we are here */ + zhdr->foreign_handles--; is_free = true; + read_lock(&slots->lock); + if (!test_bit(HANDLES_ORPHANED, &slots->pool)) { + read_unlock(&slots->lock); + return; + } for (i = 0; i <= BUDDY_MASK; i++) { if (slots->slot[i]) { is_free = false; break; } } + read_unlock(&slots->lock); if (is_free) { struct z3fold_pool *pool = slots_to_pool(slots); @@ -322,6 +422,7 @@ static struct z3fold_header *init_z3fold zhdr->first_num = 0; zhdr->start_middle = 0; zhdr->cpu = -1; + zhdr->foreign_handles = 0; zhdr->slots = slots; zhdr->pool = pool; INIT_LIST_HEAD(&zhdr->buddy); @@ -341,24 +442,6 @@ static void free_z3fold_page(struct page __free_page(page); } -/* Lock a z3fold page */ -static inline void z3fold_page_lock(struct z3fold_header *zhdr) -{ - spin_lock(&zhdr->page_lock); -} - -/* Try to lock a z3fold page */ -static inline int z3fold_page_trylock(struct z3fold_header *zhdr) -{ - return spin_trylock(&zhdr->page_lock); -} - -/* Unlock a z3fold page */ -static inline void z3fold_page_unlock(struct z3fold_header *zhdr) -{ - spin_unlock(&zhdr->page_lock); -} - /* Helper function to build the index */ static inline int __idx(struct z3fold_header *zhdr, enum buddy bud) { @@ -389,7 +472,9 @@ static unsigned long __encode_handle(str if (bud == LAST) h |= (zhdr->last_chunks << BUDDY_SHIFT); + write_lock(&slots->lock); slots->slot[idx] = h; + write_unlock(&slots->lock); return (unsigned long)&slots->slot[idx]; } @@ -398,22 +483,15 @@ static unsigned long encode_handle(struc return __encode_handle(zhdr, zhdr->slots, bud); } -/* Returns the z3fold page where a given handle is stored */ -static inline struct z3fold_header *handle_to_z3fold_header(unsigned long h) -{ - unsigned long addr = h; - - if (!(addr & (1 << PAGE_HEADLESS))) - addr = *(unsigned long *)h; - - return (struct z3fold_header *)(addr & PAGE_MASK); -} - /* only for LAST bud, returns zero otherwise */ static unsigned short handle_to_chunks(unsigned long handle) { - unsigned long addr = *(unsigned long *)handle; + struct z3fold_buddy_slots *slots = handle_to_slots(handle); + unsigned long addr; + read_lock(&slots->lock); + addr = *(unsigned long *)handle; + read_unlock(&slots->lock); return (addr & ~PAGE_MASK) >> BUDDY_SHIFT; } @@ -425,10 +503,13 @@ static unsigned short handle_to_chunks(u static enum buddy handle_to_buddy(unsigned long handle) { struct z3fold_header *zhdr; + struct z3fold_buddy_slots *slots = handle_to_slots(handle); unsigned long addr; + read_lock(&slots->lock); WARN_ON(handle & (1 << PAGE_HEADLESS)); addr = *(unsigned long *)handle; + read_unlock(&slots->lock); zhdr = (struct z3fold_header *)(addr & PAGE_MASK); return (addr - zhdr->first_num) & BUDDY_MASK; } @@ -442,6 +523,8 @@ static void __release_z3fold_page(struct { struct page *page = virt_to_page(zhdr); struct z3fold_pool *pool = zhdr_to_pool(zhdr); + bool is_free = true; + int i; WARN_ON(!list_empty(&zhdr->buddy)); set_bit(PAGE_STALE, &page->private); @@ -450,8 +533,25 @@ static void __release_z3fold_page(struct if (!list_empty(&page->lru)) list_del_init(&page->lru); spin_unlock(&pool->lock); + + /* If there are no foreign handles, free the handles array */ + read_lock(&zhdr->slots->lock); + for (i = 0; i <= BUDDY_MASK; i++) { + if (zhdr->slots->slot[i]) { + is_free = false; + break; + } + } + if (!is_free) + set_bit(HANDLES_ORPHANED, &zhdr->slots->pool); + read_unlock(&zhdr->slots->lock); + + if (is_free) + kmem_cache_free(pool->c_handle, zhdr->slots); + if (locked) z3fold_page_unlock(zhdr); + spin_lock(&pool->stale_lock); list_add(&zhdr->buddy, &pool->stale); queue_work(pool->release_wq, &pool->work); @@ -479,6 +579,7 @@ static void release_z3fold_page_locked_l struct z3fold_header *zhdr = container_of(ref, struct z3fold_header, refcount); struct z3fold_pool *pool = zhdr_to_pool(zhdr); + spin_lock(&pool->lock); list_del_init(&zhdr->buddy); spin_unlock(&pool->lock); @@ -559,6 +660,119 @@ static inline void *mchunk_memmove(struc zhdr->middle_chunks << CHUNK_SHIFT); } +static inline bool buddy_single(struct z3fold_header *zhdr) +{ + return !((zhdr->first_chunks && zhdr->middle_chunks) || + (zhdr->first_chunks && zhdr->last_chunks) || + (zhdr->middle_chunks && zhdr->last_chunks)); +} + +static struct z3fold_header *compact_single_buddy(struct z3fold_header *zhdr) +{ + struct z3fold_pool *pool = zhdr_to_pool(zhdr); + void *p = zhdr; + unsigned long old_handle = 0; + size_t sz = 0; + struct z3fold_header *new_zhdr = NULL; + int first_idx = __idx(zhdr, FIRST); + int middle_idx = __idx(zhdr, MIDDLE); + int last_idx = __idx(zhdr, LAST); + unsigned short *moved_chunks = NULL; + + /* + * No need to protect slots here -- all the slots are "local" and + * the page lock is already taken + */ + if (zhdr->first_chunks && zhdr->slots->slot[first_idx]) { + p += ZHDR_SIZE_ALIGNED; + sz = zhdr->first_chunks << CHUNK_SHIFT; + old_handle = (unsigned long)&zhdr->slots->slot[first_idx]; + moved_chunks = &zhdr->first_chunks; + } else if (zhdr->middle_chunks && zhdr->slots->slot[middle_idx]) { + p += zhdr->start_middle << CHUNK_SHIFT; + sz = zhdr->middle_chunks << CHUNK_SHIFT; + old_handle = (unsigned long)&zhdr->slots->slot[middle_idx]; + moved_chunks = &zhdr->middle_chunks; + } else if (zhdr->last_chunks && zhdr->slots->slot[last_idx]) { + p += PAGE_SIZE - (zhdr->last_chunks << CHUNK_SHIFT); + sz = zhdr->last_chunks << CHUNK_SHIFT; + old_handle = (unsigned long)&zhdr->slots->slot[last_idx]; + moved_chunks = &zhdr->last_chunks; + } + + if (sz > 0) { + enum buddy new_bud = HEADLESS; + short chunks = size_to_chunks(sz); + void *q; + + new_zhdr = __z3fold_alloc(pool, sz, false); + if (!new_zhdr) + return NULL; + + if (WARN_ON(new_zhdr == zhdr)) + goto out_fail; + + if (new_zhdr->first_chunks == 0) { + if (new_zhdr->middle_chunks != 0 && + chunks >= new_zhdr->start_middle) { + new_bud = LAST; + } else { + new_bud = FIRST; + } + } else if (new_zhdr->last_chunks == 0) { + new_bud = LAST; + } else if (new_zhdr->middle_chunks == 0) { + new_bud = MIDDLE; + } + q = new_zhdr; + switch (new_bud) { + case FIRST: + new_zhdr->first_chunks = chunks; + q += ZHDR_SIZE_ALIGNED; + break; + case MIDDLE: + new_zhdr->middle_chunks = chunks; + new_zhdr->start_middle = + new_zhdr->first_chunks + ZHDR_CHUNKS; + q += new_zhdr->start_middle << CHUNK_SHIFT; + break; + case LAST: + new_zhdr->last_chunks = chunks; + q += PAGE_SIZE - (new_zhdr->last_chunks << CHUNK_SHIFT); + break; + default: + goto out_fail; + } + new_zhdr->foreign_handles++; + memcpy(q, p, sz); + write_lock(&zhdr->slots->lock); + *(unsigned long *)old_handle = (unsigned long)new_zhdr + + __idx(new_zhdr, new_bud); + if (new_bud == LAST) + *(unsigned long *)old_handle |= + (new_zhdr->last_chunks << BUDDY_SHIFT); + write_unlock(&zhdr->slots->lock); + add_to_unbuddied(pool, new_zhdr); + z3fold_page_unlock(new_zhdr); + + *moved_chunks = 0; + } + + return new_zhdr; + +out_fail: + if (new_zhdr) { + if (kref_put(&new_zhdr->refcount, release_z3fold_page_locked)) + atomic64_dec(&pool->pages_nr); + else { + add_to_unbuddied(pool, new_zhdr); + z3fold_page_unlock(new_zhdr); + } + } + return NULL; + +} + #define BIG_CHUNK_GAP 3 /* Has to be called with lock held */ static int z3fold_compact_page(struct z3fold_header *zhdr) @@ -638,6 +852,15 @@ static void do_compact_page(struct z3fol return; } + if (!zhdr->foreign_handles && buddy_single(zhdr) && + zhdr->mapped_count == 0 && compact_single_buddy(zhdr)) { + if (kref_put(&zhdr->refcount, release_z3fold_page_locked)) + atomic64_dec(&pool->pages_nr); + else + z3fold_page_unlock(zhdr); + return; + } + z3fold_compact_page(zhdr); add_to_unbuddied(pool, zhdr); z3fold_page_unlock(zhdr); @@ -690,7 +913,8 @@ lookup: spin_unlock(&pool->lock); page = virt_to_page(zhdr); - if (test_bit(NEEDS_COMPACTING, &page->private)) { + if (test_bit(NEEDS_COMPACTING, &page->private) || + test_bit(PAGE_CLAIMED, &page->private)) { z3fold_page_unlock(zhdr); zhdr = NULL; put_cpu_ptr(pool->unbuddied); @@ -734,7 +958,8 @@ lookup: spin_unlock(&pool->lock); page = virt_to_page(zhdr); - if (test_bit(NEEDS_COMPACTING, &page->private)) { + if (test_bit(NEEDS_COMPACTING, &page->private) || + test_bit(PAGE_CLAIMED, &page->private)) { z3fold_page_unlock(zhdr); zhdr = NULL; if (can_sleep) @@ -1000,7 +1225,7 @@ static void z3fold_free(struct z3fold_po enum buddy bud; bool page_claimed; - zhdr = handle_to_z3fold_header(handle); + zhdr = get_z3fold_header(handle); page = virt_to_page(zhdr); page_claimed = test_and_set_bit(PAGE_CLAIMED, &page->private); @@ -1014,6 +1239,7 @@ static void z3fold_free(struct z3fold_po spin_lock(&pool->lock); list_del(&page->lru); spin_unlock(&pool->lock); + put_z3fold_header(zhdr); free_z3fold_page(page, true); atomic64_dec(&pool->pages_nr); } @@ -1021,7 +1247,6 @@ static void z3fold_free(struct z3fold_po } /* Non-headless case */ - z3fold_page_lock(zhdr); bud = handle_to_buddy(handle); switch (bud) { @@ -1037,11 +1262,13 @@ static void z3fold_free(struct z3fold_po default: pr_err("%s: unknown bud %d\n", __func__, bud); WARN_ON(1); - z3fold_page_unlock(zhdr); + put_z3fold_header(zhdr); + clear_bit(PAGE_CLAIMED, &page->private); return; } - free_handle(handle); + if (!page_claimed) + free_handle(handle); if (kref_put(&zhdr->refcount, release_z3fold_page_locked_list)) { atomic64_dec(&pool->pages_nr); return; @@ -1053,7 +1280,7 @@ static void z3fold_free(struct z3fold_po } if (unlikely(PageIsolated(page)) || test_and_set_bit(NEEDS_COMPACTING, &page->private)) { - z3fold_page_unlock(zhdr); + put_z3fold_header(zhdr); clear_bit(PAGE_CLAIMED, &page->private); return; } @@ -1063,14 +1290,14 @@ static void z3fold_free(struct z3fold_po spin_unlock(&pool->lock); zhdr->cpu = -1; kref_get(&zhdr->refcount); - do_compact_page(zhdr, true); clear_bit(PAGE_CLAIMED, &page->private); + do_compact_page(zhdr, true); return; } kref_get(&zhdr->refcount); - queue_work_on(zhdr->cpu, pool->compact_wq, &zhdr->work); clear_bit(PAGE_CLAIMED, &page->private); - z3fold_page_unlock(zhdr); + queue_work_on(zhdr->cpu, pool->compact_wq, &zhdr->work); + put_z3fold_header(zhdr); } /** @@ -1111,11 +1338,10 @@ static void z3fold_free(struct z3fold_po */ static int z3fold_reclaim_page(struct z3fold_pool *pool, unsigned int retries) { - int i, ret = 0; + int i, ret = -1; struct z3fold_header *zhdr = NULL; struct page *page = NULL; struct list_head *pos; - struct z3fold_buddy_slots slots; unsigned long first_handle = 0, middle_handle = 0, last_handle = 0; spin_lock(&pool->lock); @@ -1153,6 +1379,12 @@ static int z3fold_reclaim_page(struct z3 zhdr = NULL; continue; /* can't evict at this point */ } + if (zhdr->foreign_handles) { + clear_bit(PAGE_CLAIMED, &page->private); + z3fold_page_unlock(zhdr); + zhdr = NULL; + continue; /* can't evict such page */ + } kref_get(&zhdr->refcount); list_del_init(&zhdr->buddy); zhdr->cpu = -1; @@ -1176,39 +1408,38 @@ static int z3fold_reclaim_page(struct z3 last_handle = 0; middle_handle = 0; if (zhdr->first_chunks) - first_handle = __encode_handle(zhdr, &slots, - FIRST); + first_handle = encode_handle(zhdr, FIRST); if (zhdr->middle_chunks) - middle_handle = __encode_handle(zhdr, &slots, - MIDDLE); + middle_handle = encode_handle(zhdr, MIDDLE); if (zhdr->last_chunks) - last_handle = __encode_handle(zhdr, &slots, - LAST); + last_handle = encode_handle(zhdr, LAST); /* * it's safe to unlock here because we hold a * reference to this page */ z3fold_page_unlock(zhdr); } else { - first_handle = __encode_handle(zhdr, &slots, HEADLESS); + first_handle = encode_handle(zhdr, HEADLESS); last_handle = middle_handle = 0; } - /* Issue the eviction callback(s) */ if (middle_handle) { ret = pool->ops->evict(pool, middle_handle); if (ret) goto next; + free_handle(middle_handle); } if (first_handle) { ret = pool->ops->evict(pool, first_handle); if (ret) goto next; + free_handle(first_handle); } if (last_handle) { ret = pool->ops->evict(pool, last_handle); if (ret) goto next; + free_handle(last_handle); } next: if (test_bit(PAGE_HEADLESS, &page->private)) { @@ -1264,14 +1495,13 @@ static void *z3fold_map(struct z3fold_po void *addr; enum buddy buddy; - zhdr = handle_to_z3fold_header(handle); + zhdr = get_z3fold_header(handle); addr = zhdr; page = virt_to_page(zhdr); if (test_bit(PAGE_HEADLESS, &page->private)) goto out; - z3fold_page_lock(zhdr); buddy = handle_to_buddy(handle); switch (buddy) { case FIRST: @@ -1293,8 +1523,8 @@ static void *z3fold_map(struct z3fold_po if (addr) zhdr->mapped_count++; - z3fold_page_unlock(zhdr); out: + put_z3fold_header(zhdr); return addr; } @@ -1309,18 +1539,17 @@ static void z3fold_unmap(struct z3fold_p struct page *page; enum buddy buddy; - zhdr = handle_to_z3fold_header(handle); + zhdr = get_z3fold_header(handle); page = virt_to_page(zhdr); if (test_bit(PAGE_HEADLESS, &page->private)) return; - z3fold_page_lock(zhdr); buddy = handle_to_buddy(handle); if (buddy == MIDDLE) clear_bit(MIDDLE_CHUNK_MAPPED, &page->private); zhdr->mapped_count--; - z3fold_page_unlock(zhdr); + put_z3fold_header(zhdr); } /** @@ -1352,19 +1581,21 @@ static bool z3fold_page_isolate(struct p test_bit(PAGE_STALE, &page->private)) goto out; + if (zhdr->mapped_count != 0 || zhdr->foreign_handles != 0) + goto out; + pool = zhdr_to_pool(zhdr); + spin_lock(&pool->lock); + if (!list_empty(&zhdr->buddy)) + list_del_init(&zhdr->buddy); + if (!list_empty(&page->lru)) + list_del_init(&page->lru); + spin_unlock(&pool->lock); + + kref_get(&zhdr->refcount); + z3fold_page_unlock(zhdr); + return true; - if (zhdr->mapped_count == 0) { - kref_get(&zhdr->refcount); - if (!list_empty(&zhdr->buddy)) - list_del_init(&zhdr->buddy); - spin_lock(&pool->lock); - if (!list_empty(&page->lru)) - list_del(&page->lru); - spin_unlock(&pool->lock); - z3fold_page_unlock(zhdr); - return true; - } out: z3fold_page_unlock(zhdr); return false; @@ -1387,7 +1618,7 @@ static int z3fold_page_migrate(struct ad if (!z3fold_page_trylock(zhdr)) { return -EAGAIN; } - if (zhdr->mapped_count != 0) { + if (zhdr->mapped_count != 0 || zhdr->foreign_handles != 0) { z3fold_page_unlock(zhdr); return -EBUSY; } From patchwork Sun Dec 1 01:56:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268413 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AE89714B7 for ; Sun, 1 Dec 2019 01:56:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 71C122082E for ; Sun, 1 Dec 2019 01:56:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="mXybSIMf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 71C122082E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 902B26B0349; Sat, 30 Nov 2019 20:56:17 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 88D326B034B; Sat, 30 Nov 2019 20:56:17 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6E22A6B034C; Sat, 30 Nov 2019 20:56:17 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0137.hostedemail.com [216.40.44.137]) by kanga.kvack.org (Postfix) with ESMTP id 548246B0349 for ; Sat, 30 Nov 2019 20:56:17 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 08A63180AD817 for ; Sun, 1 Dec 2019 01:56:17 +0000 (UTC) X-FDA: 76214907594.25.wing73_36d1b283d713a X-Spam-Summary: 2,0,0,16d492d42cacd264,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hughd@google.com:linux-man@vger.kernel.org::lixinhai.lxh@gmail.com:mhocko@suse.com:mm-commits@vger.kernel.org:n-horiguchi@ah.jp.nec.com:torvalds@linux-foundation.org:vbabka@suse.cz,RULES_HIT:41:355:379:800:960:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1543:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2859:2895:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6121:6261:6653:7514:7576:8660:9025:9391:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:12555:12679:12783:12986:13148:13161:13229:13230:13846:14096:14181:14721:14849:21060:21080:21094:21220:21323:21433:21451:21627:21740:21819:21939:30012:30054:30070:30091,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Net check:no X-HE-Tag: wing73_36d1b283d713a X-Filterd-Recvd-Size: 4892 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:16 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 852FE2084D; Sun, 1 Dec 2019 01:56:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165375; bh=SaBTKBsd7mt6lHS/YOr773gnLBMgslZzLgdw1dfwTAY=; h=Date:From:To:Subject:From; b=mXybSIMfyXpr7KCPZHv3i9P7+vlKliXmzAKL57KTQqgx0WToYK7q/FvgWIBeOyrBI xTs3/MOSv4bQf5W6lQcVUwQJlAV4I6R4OE/evBwBOY+/6CCArOxCa0h6zQo6iXPAms UgxUNLu8VszY1bA6PqrR0JBTmOnMYzx/6It3gVG0= Date: Sat, 30 Nov 2019 17:56:15 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hughd@google.com, linux-man@vger.kernel.org, linux-mm@kvack.org, lixinhai.lxh@gmail.com, mhocko@suse.com, mm-commits@vger.kernel.org, n-horiguchi@ah.jp.nec.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 118/158] mm/mempolicy.c: check range first in queue_pages_test_walk Message-ID: <20191201015615.KPQyOXwfq%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Li Xinhai Subject: mm/mempolicy.c: check range first in queue_pages_test_walk Patch series "mm: Fix checking unmapped holes for mbind", v4. This patchset fix checking unmapped holes for mbind(). First patch makes sure the vma been correctly tracked in .test_walk(), so each time when .test_walk() is called, the neighborhood of two vma is correct. Current problem is that the !vma_migratable() check could cause return immediately without update tracking to vma. Second patch fix the inconsistent report of EFAULT when mbind() is called for MPOL_DEFAULT and non MPOL_DEFAULT cases, so application do not need to have workaround code to handle this special behavior. Currently there are two problems, one is that the .test_walk() can not know there is hole at tail side of range, because .test_walk() only call for vma not for hole. The other one is that mbind_range() checks for hole at head side of range but do not consider the MPOL_MF_DISCONTIG_OK flag as done in .test_walk(). This patch (of 2): Checking unmapped hole and updating the previous vma must be handled first, otherwise the unmapped hole could be calculated from a wrong previous vma. Several commits were relevant to this error: commit 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()"), This commit was correct, the VM_PFNMAP check was after updating previous vma commit 48684a65b4e3 (mm: pagewalk: fix misbehavior of walk_page_range for vma(VM_PFNMAP)), This commit added VM_PFNMAP check before updating previous vma. Then, there were two VM_PFNMAP check did same thing twice. commit acda0c334028 (mm/mempolicy.c: get rid of duplicated check for vma(VM_PFNMAP) in queue_page s_range()), This commit tried to fix the duplicated VM_PFNMAP check, but it wrongly removed the one which was after updating vma. Link: http://lkml.kernel.org/r/1573218104-11021-2-git-send-email-lixinhai.lxh@gmail.com Fixes: acda0c334028 (mm/mempolicy.c: get rid of duplicated check for vma(VM_PFNMAP) in queue_pages_range()) Signed-off-by: Li Xinhai Reviewed-by: Naoya Horiguchi Cc: Michal Hocko Cc: Vlastimil Babka Cc: Hugh Dickins Cc: linux-man Signed-off-by: Andrew Morton --- mm/mempolicy.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) --- a/mm/mempolicy.c~mm-check-range-first-in-queue_pages_test_walk +++ a/mm/mempolicy.c @@ -618,6 +618,16 @@ static int queue_pages_test_walk(unsigne unsigned long endvma = vma->vm_end; unsigned long flags = qp->flags; + /* range check first */ + if (!(flags & MPOL_MF_DISCONTIG_OK)) { + if (!vma->vm_next && vma->vm_end < end) + return -EFAULT; + if (qp->prev && qp->prev->vm_end < vma->vm_start) + return -EFAULT; + } + + qp->prev = vma; + /* * Need check MPOL_MF_STRICT to return -EIO if possible * regardless of vma_migratable @@ -631,15 +641,6 @@ static int queue_pages_test_walk(unsigne if (vma->vm_start > start) start = vma->vm_start; - if (!(flags & MPOL_MF_DISCONTIG_OK)) { - if (!vma->vm_next && vma->vm_end < end) - return -EFAULT; - if (qp->prev && qp->prev->vm_end < vma->vm_start) - return -EFAULT; - } - - qp->prev = vma; - if (flags & MPOL_MF_LAZY) { /* Similar to task_numa_work, skip inaccessible VMAs */ if (!is_vm_hugetlb_page(vma) && From patchwork Sun Dec 1 01:56:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268415 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0E52E14B7 for ; Sun, 1 Dec 2019 01:56:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C21AA215E5 for ; Sun, 1 Dec 2019 01:56:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="W3Y5A1YR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C21AA215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CDAB36B034B; Sat, 30 Nov 2019 20:56:20 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C5E856B034D; Sat, 30 Nov 2019 20:56:20 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AFF326B034E; Sat, 30 Nov 2019 20:56:20 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0018.hostedemail.com [216.40.44.18]) by kanga.kvack.org (Postfix) with ESMTP id 9347A6B034B for ; Sat, 30 Nov 2019 20:56:20 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 4D9BC181AEF0B for ; Sun, 1 Dec 2019 01:56:20 +0000 (UTC) X-FDA: 76214907720.05.pipe37_374a4bd4c0a08 X-Spam-Summary: 40,2.5,0,30d734a2e2a10957,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hughd@google.com:linux-man@vger.kernel.org::lixinhai.lxh@gmail.com:mhocko@suse.com:mm-commits@vger.kernel.org:n-horiguchi@ah.jp.nec.com:torvalds@linux-foundation.org:vbabka@suse.cz,RULES_HIT:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1535:1544:1605:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2740:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4117:4250:4321:4605:5007:6121:6261:6653:7514:7576:7903:9025:9391:9545:9592:10011:10913:11026:11473:11658:11914:12043:12048:12050:12219:12291:12295:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13141:13161:13180:13229:13230:13846:14096:14181:14721:14849:21060:21080:21094:21323:21324:21433:21451:21627:21740:21819:21939:30012:30054:30070:30074,0,RBL:e rror,Cac X-HE-Tag: pipe37_374a4bd4c0a08 X-Filterd-Recvd-Size: 6324 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:19 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D5A1D208C3; Sun, 1 Dec 2019 01:56:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165379; bh=Byuwr/l4raRkXxcjH1j0OkOqNWKztHVTFt5DeNjwufU=; h=Date:From:To:Subject:From; b=W3Y5A1YRGOKcLq6p6hbpxr+stkJ1zUdjPdfj2wES7CX+A3q8+cj1VBaWtHEW8skj2 EgDgVRw+mFGpio40zkVUTLfOL+LXsKjtmZXIctOWr/eBh5FGZPLwFOnQPQIxjGj8Vu xGEGfU6AfxairVpSdqo+Of3poB/cR/vSpShcG2Lg= Date: Sat, 30 Nov 2019 17:56:18 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hughd@google.com, linux-man@vger.kernel.org, linux-mm@kvack.org, lixinhai.lxh@gmail.com, mhocko@suse.com, mm-commits@vger.kernel.org, n-horiguchi@ah.jp.nec.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 119/158] mm/mempolicy.c: fix checking unmapped holes for mbind Message-ID: <20191201015618.fbLx16qnj%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Li Xinhai Subject: mm/mempolicy.c: fix checking unmapped holes for mbind mbind() is required to report EFAULT if range, specified by addr and len, contains unmapped holes. In current implementation, below rules are applied for this checking: 1: Unmapped holes at any part of the specified range should be reported as EFAULT if mbind() for none MPOL_DEFAULT cases; 2: Unmapped holes at any part of the specified range should be ignored (do not reprot EFAULT) if mbind() for MPOL_DEFAULT case; 3: The whole range in an unmapped hole should be reported as EFAULT; Note that rule 2 does not fullfill the mbind() API definition, but since that behavior has existed for long days (the internal flag MPOL_MF_DISCONTIG_OK is for this purpose), this patch does not plan to change it. In current code, application observed inconsistent behavior on rule 1 and rule 2 respectively. That inconsistency is fixed as below details. Cases of rule 1: 1) Hole at head side of range. Current code reprot EFAULT, no change by this patch. [ vma ][ hole ][ vma ] [ range ] 2) Hole at middle of range. Current code report EFAULT, no change by this patch. [ vma ][ hole ][ vma ] [ range ] 3) Hole at tail side of range. Current code do not report EFAULT, this patch fix it. [ vma ][ hole ][ vma ] [ range ] Cases of rule 2: 1) Hole at head side of range. Current code reprot EFAULT, this patch fix it. [ vma ][ hole ][ vma ] [ range ] 2) Hole at middle of range. Current code do not report EFAULT, no change by this patch. this patch. [ vma ][ hole ][ vma] [ range ] 3) Hole at tail side of range. Current code do not report EFAULT, no change by this patch. [ vma ][ hole ][ vma] [ range ] This patch has no changes to rule 3. The unmapped hole checking can also be handled by using .pte_hole(), instead of .test_walk(). But .pte_hole() is called for holes inside and outside vma, which causes more cost, so this patch keeps the original design with .test_walk(). Link: http://lkml.kernel.org/r/1573218104-11021-3-git-send-email-lixinhai.lxh@gmail.com Fixes: 6f4576e3687b ("mempolicy: apply page table walker on queue_pages_range()") Signed-off-by: Li Xinhai Reviewed-by: Naoya Horiguchi Cc: Michal Hocko Cc: Vlastimil Babka Cc: Hugh Dickins Cc: linux-man Signed-off-by: Andrew Morton --- mm/mempolicy.c | 40 +++++++++++++++++++++++++++------------- 1 file changed, 27 insertions(+), 13 deletions(-) --- a/mm/mempolicy.c~mm-fix-checking-unmapped-holes-for-mbind +++ a/mm/mempolicy.c @@ -410,7 +410,9 @@ struct queue_pages { struct list_head *pagelist; unsigned long flags; nodemask_t *nmask; - struct vm_area_struct *prev; + unsigned long start; + unsigned long end; + struct vm_area_struct *first; }; /* @@ -619,14 +621,20 @@ static int queue_pages_test_walk(unsigne unsigned long flags = qp->flags; /* range check first */ - if (!(flags & MPOL_MF_DISCONTIG_OK)) { - if (!vma->vm_next && vma->vm_end < end) - return -EFAULT; - if (qp->prev && qp->prev->vm_end < vma->vm_start) + VM_BUG_ON((vma->vm_start > start) || (vma->vm_end < end)); + + if (!qp->first) { + qp->first = vma; + if (!(flags & MPOL_MF_DISCONTIG_OK) && + (qp->start < vma->vm_start)) + /* hole at head side of range */ return -EFAULT; } - - qp->prev = vma; + if (!(flags & MPOL_MF_DISCONTIG_OK) && + ((vma->vm_end < qp->end) && + (!vma->vm_next || vma->vm_end < vma->vm_next->vm_start))) + /* hole at middle or tail of range */ + return -EFAULT; /* * Need check MPOL_MF_STRICT to return -EIO if possible @@ -638,8 +646,6 @@ static int queue_pages_test_walk(unsigne if (endvma > end) endvma = end; - if (vma->vm_start > start) - start = vma->vm_start; if (flags & MPOL_MF_LAZY) { /* Similar to task_numa_work, skip inaccessible VMAs */ @@ -682,14 +688,23 @@ queue_pages_range(struct mm_struct *mm, nodemask_t *nodes, unsigned long flags, struct list_head *pagelist) { + int err; struct queue_pages qp = { .pagelist = pagelist, .flags = flags, .nmask = nodes, - .prev = NULL, + .start = start, + .end = end, + .first = NULL, }; - return walk_page_range(mm, start, end, &queue_pages_walk_ops, &qp); + err = walk_page_range(mm, start, end, &queue_pages_walk_ops, &qp); + + if (!qp.first) + /* whole range in hole */ + err = -EFAULT; + + return err; } /* @@ -741,8 +756,7 @@ static int mbind_range(struct mm_struct unsigned long vmend; vma = find_vma(mm, start); - if (!vma || vma->vm_start > start) - return -EFAULT; + VM_BUG_ON(!vma); prev = vma->vm_prev; if (start > vma->vm_start) From patchwork Sun Dec 1 01:56:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268417 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 37334921 for ; Sun, 1 Dec 2019 01:56:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EF47D215E5 for ; Sun, 1 Dec 2019 01:56:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="EitGbBr/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EF47D215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C5F916B034D; Sat, 30 Nov 2019 20:56:23 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BEB9C6B034F; Sat, 30 Nov 2019 20:56:23 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3BF26B0351; Sat, 30 Nov 2019 20:56:23 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0243.hostedemail.com [216.40.44.243]) by kanga.kvack.org (Postfix) with ESMTP id 876446B034D for ; Sat, 30 Nov 2019 20:56:23 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 30B9540D3 for ; Sun, 1 Dec 2019 01:56:23 +0000 (UTC) X-FDA: 76214907846.07.grain60_37b6ddd47a21b X-Spam-Summary: 2,0,0,d5203c641ba4e07a,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:caoj.fnst@cn.fujitsu.com::mm-commits@vger.kernel.org:rppt@linux.ibm.com:torvalds@linux-foundation.org,RULES_HIT:41:69:355:379:800:960:966:967:973:982:988:989:1260:1263:1345:1381:1431:1437:1535:1544:1605:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4384:4385:4395:5007:6261:6653:7576:7875:7903:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12291:12297:12438:12517:12519:12555:12679:12683:12783:12986:14181:14721:14849:21080:21094:21323:21433:21451:21627:21939:30003:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: grain60_37b6ddd47a21b X-Filterd-Recvd-Size: 5301 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:22 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id EEE9D215F1; Sun, 1 Dec 2019 01:56:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165382; bh=8jRsnmvb2xlS/sdhnUeIfuR1fMdowT+VOHxWJjmg7Wg=; h=Date:From:To:Subject:From; b=EitGbBr/7ucR+jxAGpk7vqCQots+2fY0LqzyhZx5VOVsaQ2aVjK+Sb4D76PhgsZIi N6Wb7uu/jSn+QmN9W0zSUkbVlgFhYzB2N8ib4YUWpqbm8zawLuNuvYW25Nhseu2nRK YDR943PoVHH5gFB4AtCCtiO5KpUGk80MiAPUn2Go= Date: Sat, 30 Nov 2019 17:56:21 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, caoj.fnst@cn.fujitsu.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rppt@linux.ibm.com, torvalds@linux-foundation.org Subject: [patch 120/158] mm/memblock.c: cleanup doc Message-ID: <20191201015621.SA0S3phML%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Cao jin Subject: mm/memblock.c: cleanup doc fix typos for: elaboarte -> elaborate architecure -> architecture compltes -> completes And, convert the markup :c:func:`foo` to foo() as kernel documentation toolchain can recognize foo() as a function. Link: http://lkml.kernel.org/r/20190912123127.8694-1-caoj.fnst@cn.fujitsu.com Signed-off-by: Cao jin Suggested-by: Mike Rapoport Reviewed-by: Mike Rapoport Signed-off-by: Andrew Morton --- mm/memblock.c | 46 +++++++++++++++++++++------------------------- 1 file changed, 21 insertions(+), 25 deletions(-) --- a/mm/memblock.c~mm-memblock-cleanup-doc +++ a/mm/memblock.c @@ -57,42 +57,38 @@ * at build time. The region arrays for the "memory" and "reserved" * types are initially sized to %INIT_MEMBLOCK_REGIONS and for the * "physmap" type to %INIT_PHYSMEM_REGIONS. - * The :c:func:`memblock_allow_resize` enables automatic resizing of - * the region arrays during addition of new regions. This feature - * should be used with care so that memory allocated for the region - * array will not overlap with areas that should be reserved, for - * example initrd. + * The memblock_allow_resize() enables automatic resizing of the region + * arrays during addition of new regions. This feature should be used + * with care so that memory allocated for the region array will not + * overlap with areas that should be reserved, for example initrd. * * The early architecture setup should tell memblock what the physical - * memory layout is by using :c:func:`memblock_add` or - * :c:func:`memblock_add_node` functions. The first function does not - * assign the region to a NUMA node and it is appropriate for UMA - * systems. Yet, it is possible to use it on NUMA systems as well and - * assign the region to a NUMA node later in the setup process using - * :c:func:`memblock_set_node`. The :c:func:`memblock_add_node` - * performs such an assignment directly. + * memory layout is by using memblock_add() or memblock_add_node() + * functions. The first function does not assign the region to a NUMA + * node and it is appropriate for UMA systems. Yet, it is possible to + * use it on NUMA systems as well and assign the region to a NUMA node + * later in the setup process using memblock_set_node(). The + * memblock_add_node() performs such an assignment directly. * * Once memblock is setup the memory can be allocated using one of the * API variants: * - * * :c:func:`memblock_phys_alloc*` - these functions return the - * **physical** address of the allocated memory - * * :c:func:`memblock_alloc*` - these functions return the **virtual** - * address of the allocated memory. + * * memblock_phys_alloc*() - these functions return the **physical** + * address of the allocated memory + * * memblock_alloc*() - these functions return the **virtual** address + * of the allocated memory. * * Note, that both API variants use implict assumptions about allowed * memory ranges and the fallback methods. Consult the documentation - * of :c:func:`memblock_alloc_internal` and - * :c:func:`memblock_alloc_range_nid` functions for more elaboarte - * description. - * - * As the system boot progresses, the architecture specific - * :c:func:`mem_init` function frees all the memory to the buddy page - * allocator. + * of memblock_alloc_internal() and memblock_alloc_range_nid() + * functions for more elaborate description. * - * Unless an architecure enables %CONFIG_ARCH_KEEP_MEMBLOCK, the + * As the system boot progresses, the architecture specific mem_init() + * function frees all the memory to the buddy page allocator. + * + * Unless an architecture enables %CONFIG_ARCH_KEEP_MEMBLOCK, the * memblock data structures will be discarded after the system - * initialization compltes. + * initialization completes. */ #ifndef CONFIG_NEED_MULTIPLE_NODES From patchwork Sun Dec 1 01:56:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268419 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ACB04921 for ; Sun, 1 Dec 2019 01:56:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 78FDE2082E for ; Sun, 1 Dec 2019 01:56:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="eQxHSwwx" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 78FDE2082E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B00D36B034F; Sat, 30 Nov 2019 20:56:26 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A62456B0352; Sat, 30 Nov 2019 20:56:26 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9036F6B0353; Sat, 30 Nov 2019 20:56:26 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0075.hostedemail.com [216.40.44.75]) by kanga.kvack.org (Postfix) with ESMTP id 6F71A6B034F for ; Sat, 30 Nov 2019 20:56:26 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 16EC5180AD817 for ; Sun, 1 Dec 2019 01:56:26 +0000 (UTC) X-FDA: 76214907972.15.thing44_38254ffc3ad5f X-Spam-Summary: 2,0,0,9393c10087bcd035,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:caoj.fnst@cn.fujitsu.com::mm-commits@vger.kernel.org:rppt@linux.ibm.com:ruansy.fnst@cn.fujitsu.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2196:2199:2393:2507:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3867:3870:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:5007:6261:6653:7576:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12783:12986:13069:13311:13357:14181:14384:14721:14849:21080:21324:21451:21627:21939:30054:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:27,LUA_SUMMARY:none X-HE-Tag: thing44_38254ffc3ad5f X-Filterd-Recvd-Size: 2368 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:25 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id DD8862082E; Sun, 1 Dec 2019 01:56:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165385; bh=LyxDkHX8imJThbgeRz+O6oUpQ3frtgKLv3yZTTs1cE4=; h=Date:From:To:Subject:From; b=eQxHSwwx5i+BDtX+O/zIN81FdQTXTah44KNqjOVIw+XcB52wpBka0sZglxwYMEk0J lxk30h0TOzZRRI/9qsxYuRdYCQaZorap6UYDpBg070NU9dwRqvFptE24fG3v6Hhhbl h4O7mBEg0u5nduVS3bdHq6aMbO0Uhgifbtgb9a0Y= Date: Sat, 30 Nov 2019 17:56:24 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, caoj.fnst@cn.fujitsu.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rppt@linux.ibm.com, ruansy.fnst@cn.fujitsu.com, torvalds@linux-foundation.org Subject: [patch 121/158] mm/memblock: correct doc for function Message-ID: <20191201015624.joSflh_Ow%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Cao jin Subject: mm/memblock: correct doc for function Change "max_addr" to "end" for less confusion in memblock_alloc_range_nid comments. Link: http://lkml.kernel.org/r/20191113051822.3296-1-ruansy.fnst@cn.fujitsu.com Signed-off-by: Cao jin Signed-off-by: Shiyang Ruan Reviewed-by: Mike Rapoport Signed-off-by: Andrew Morton --- mm/memblock.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/memblock.c~mm-memblock-correct-doc-for-function +++ a/mm/memblock.c @@ -1321,7 +1321,7 @@ __next_mem_pfn_range_in_zone(u64 *idx, s * @nid: nid of the free area to find, %NUMA_NO_NODE for any node * * The allocation is performed from memory region limited by - * memblock.current_limit if @max_addr == %MEMBLOCK_ALLOC_ACCESSIBLE. + * memblock.current_limit if @end == %MEMBLOCK_ALLOC_ACCESSIBLE. * * If the specified node can not hold the requested memory the * allocation falls back to any node in the system From patchwork Sun Dec 1 01:56:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268421 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 986BE17E0 for ; Sun, 1 Dec 2019 01:56:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 58D6E205ED for ; Sun, 1 Dec 2019 01:56:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="wjKG3dak" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 58D6E205ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 201126B0352; Sat, 30 Nov 2019 20:56:30 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 18CB16B0354; Sat, 30 Nov 2019 20:56:30 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 02B276B0355; Sat, 30 Nov 2019 20:56:29 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0134.hostedemail.com [216.40.44.134]) by kanga.kvack.org (Postfix) with ESMTP id DE44F6B0352 for ; Sat, 30 Nov 2019 20:56:29 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 9743A824999B for ; Sun, 1 Dec 2019 01:56:29 +0000 (UTC) X-FDA: 76214908098.23.thumb38_38a4840de402d X-Spam-Summary: 2,0,0,68a7931270d20377,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:cai@lca.pw:dan.j.williams@intel.com:david@redhat.com::mm-commits@vger.kernel.org:osalvador@suse.de:richardw.yang@linux.intel.com:rppt@linux.ibm.com:torvalds@linux-foundation.org:yeyunfeng@huawei.com,RULES_HIT:2:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1535:1605:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2689:2693:2859:2892:2899:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4051:4120:4250:4321:4385:4605:5007:6117:6119:6261:6653:6737:7576:7875:7903:8603:9025:9121:9545:10004:10913:11026:11257:11473:11658:11914:12043:12048:12114:12291:12296:12297:12438:12517:12519:12555:12679:12683:12783:13255:13846:21080:21324:21433:21451:21627:21939:30003:30054:30064:30075,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none ,DomainC X-HE-Tag: thumb38_38a4840de402d X-Filterd-Recvd-Size: 9934 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:29 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 247AE215E5; Sun, 1 Dec 2019 01:56:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165388; bh=uMcIe2NSxM/CvCY1lj5SBFhZQSh3i41iKoSJVZB2KX0=; h=Date:From:To:Subject:From; b=wjKG3dakn4CpbQ8ukzCEcOdGV0U4iy7Qas9RXxQmHXlcop4beBc1Xj7+H5QY488mz G2h1E7Qe2kq/WOxbvM2Abj5xATs1JU339+PkFyv+hink3pJ2FqxHplvHuasCNXaJ5/ J0ktEtLVXQVOVJ9RzpSwcJFaY49NJdxLV8FeDy/g= Date: Sat, 30 Nov 2019 17:56:27 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, cai@lca.pw, dan.j.williams@intel.com, david@redhat.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, osalvador@suse.de, richardw.yang@linux.intel.com, rppt@linux.ibm.com, torvalds@linux-foundation.org, yeyunfeng@huawei.com Subject: [patch 122/158] mm: support memblock alloc on the exact node for sparse_buffer_init() Message-ID: <20191201015627.rHM34V-TS%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yunfeng Ye Subject: mm: support memblock alloc on the exact node for sparse_buffer_init() sparse_buffer_init() use memblock_alloc_try_nid_raw() to allocate memory for page management structure, if memory allocation fails from specified node, it will fall back to allocate from other nodes. Normally, the page management structure will not exceed 2% of the total memory, but a large continuous block of allocation is needed. In most cases, memory allocation from the specified node will succeed, but a node memory become highly fragmented will fail. we expect to allocate memory base section rather than by allocating a large block of memory from other NUMA nodes Add memblock_alloc_exact_nid_raw() for this situation, which allocate boot memory block on the exact node. If a large contiguous block memory allocate fail in sparse_buffer_init(), it will fall back to allocate small block memory base section. Link: http://lkml.kernel.org/r/66755ea7-ab10-8882-36fd-3e02b03775d5@huawei.com Signed-off-by: Yunfeng Ye Reviewed-by: Mike Rapoport Cc: Wei Yang Cc: Oscar Salvador Cc: Dan Williams Cc: David Hildenbrand Cc: Qian Cai Signed-off-by: Andrew Morton --- include/linux/memblock.h | 3 + mm/memblock.c | 65 ++++++++++++++++++++++++++++++------- mm/sparse.c | 2 - 3 files changed, 58 insertions(+), 12 deletions(-) --- a/include/linux/memblock.h~mm-support-memblock-alloc-on-the-exact-node-for-sparse_buffer_init +++ a/include/linux/memblock.h @@ -358,6 +358,9 @@ static inline phys_addr_t memblock_phys_ MEMBLOCK_ALLOC_ACCESSIBLE); } +void *memblock_alloc_exact_nid_raw(phys_addr_t size, phys_addr_t align, + phys_addr_t min_addr, phys_addr_t max_addr, + int nid); void *memblock_alloc_try_nid_raw(phys_addr_t size, phys_addr_t align, phys_addr_t min_addr, phys_addr_t max_addr, int nid); --- a/mm/memblock.c~mm-support-memblock-alloc-on-the-exact-node-for-sparse_buffer_init +++ a/mm/memblock.c @@ -1319,12 +1319,13 @@ __next_mem_pfn_range_in_zone(u64 *idx, s * @start: the lower bound of the memory region to allocate (phys address) * @end: the upper bound of the memory region to allocate (phys address) * @nid: nid of the free area to find, %NUMA_NO_NODE for any node + * @exact_nid: control the allocation fall back to other nodes * * The allocation is performed from memory region limited by * memblock.current_limit if @end == %MEMBLOCK_ALLOC_ACCESSIBLE. * - * If the specified node can not hold the requested memory the - * allocation falls back to any node in the system + * If the specified node can not hold the requested memory and @exact_nid + * is false, the allocation falls back to any node in the system. * * For systems with memory mirroring, the allocation is attempted first * from the regions with mirroring enabled and then retried from any @@ -1338,7 +1339,8 @@ __next_mem_pfn_range_in_zone(u64 *idx, s */ static phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size, phys_addr_t align, phys_addr_t start, - phys_addr_t end, int nid) + phys_addr_t end, int nid, + bool exact_nid) { enum memblock_flags flags = choose_memblock_flags(); phys_addr_t found; @@ -1358,7 +1360,7 @@ again: if (found && !memblock_reserve(found, size)) goto done; - if (nid != NUMA_NO_NODE) { + if (nid != NUMA_NO_NODE && !exact_nid) { found = memblock_find_in_range_node(size, align, start, end, NUMA_NO_NODE, flags); @@ -1406,7 +1408,8 @@ phys_addr_t __init memblock_phys_alloc_r phys_addr_t start, phys_addr_t end) { - return memblock_alloc_range_nid(size, align, start, end, NUMA_NO_NODE); + return memblock_alloc_range_nid(size, align, start, end, NUMA_NO_NODE, + false); } /** @@ -1425,7 +1428,7 @@ phys_addr_t __init memblock_phys_alloc_r phys_addr_t __init memblock_phys_alloc_try_nid(phys_addr_t size, phys_addr_t align, int nid) { return memblock_alloc_range_nid(size, align, 0, - MEMBLOCK_ALLOC_ACCESSIBLE, nid); + MEMBLOCK_ALLOC_ACCESSIBLE, nid, false); } /** @@ -1435,6 +1438,7 @@ phys_addr_t __init memblock_phys_alloc_t * @min_addr: the lower bound of the memory region to allocate (phys address) * @max_addr: the upper bound of the memory region to allocate (phys address) * @nid: nid of the free area to find, %NUMA_NO_NODE for any node + * @exact_nid: control the allocation fall back to other nodes * * Allocates memory block using memblock_alloc_range_nid() and * converts the returned physical address to virtual. @@ -1450,7 +1454,7 @@ phys_addr_t __init memblock_phys_alloc_t static void * __init memblock_alloc_internal( phys_addr_t size, phys_addr_t align, phys_addr_t min_addr, phys_addr_t max_addr, - int nid) + int nid, bool exact_nid) { phys_addr_t alloc; @@ -1465,11 +1469,13 @@ static void * __init memblock_alloc_inte if (max_addr > memblock.current_limit) max_addr = memblock.current_limit; - alloc = memblock_alloc_range_nid(size, align, min_addr, max_addr, nid); + alloc = memblock_alloc_range_nid(size, align, min_addr, max_addr, nid, + exact_nid); /* retry allocation without lower limit */ if (!alloc && min_addr) - alloc = memblock_alloc_range_nid(size, align, 0, max_addr, nid); + alloc = memblock_alloc_range_nid(size, align, 0, max_addr, nid, + exact_nid); if (!alloc) return NULL; @@ -1478,6 +1484,43 @@ static void * __init memblock_alloc_inte } /** + * memblock_alloc_exact_nid_raw - allocate boot memory block on the exact node + * without zeroing memory + * @size: size of memory block to be allocated in bytes + * @align: alignment of the region and block's size + * @min_addr: the lower bound of the memory region from where the allocation + * is preferred (phys address) + * @max_addr: the upper bound of the memory region from where the allocation + * is preferred (phys address), or %MEMBLOCK_ALLOC_ACCESSIBLE to + * allocate only from memory limited by memblock.current_limit value + * @nid: nid of the free area to find, %NUMA_NO_NODE for any node + * + * Public function, provides additional debug information (including caller + * info), if enabled. Does not zero allocated memory. + * + * Return: + * Virtual address of allocated memory block on success, NULL on failure. + */ +void * __init memblock_alloc_exact_nid_raw( + phys_addr_t size, phys_addr_t align, + phys_addr_t min_addr, phys_addr_t max_addr, + int nid) +{ + void *ptr; + + memblock_dbg("%s: %llu bytes align=0x%llx nid=%d from=%pa max_addr=%pa %pS\n", + __func__, (u64)size, (u64)align, nid, &min_addr, + &max_addr, (void *)_RET_IP_); + + ptr = memblock_alloc_internal(size, align, + min_addr, max_addr, nid, true); + if (ptr && size > 0) + page_init_poison(ptr, size); + + return ptr; +} + +/** * memblock_alloc_try_nid_raw - allocate boot memory block without zeroing * memory and without panicking * @size: size of memory block to be allocated in bytes @@ -1508,7 +1551,7 @@ void * __init memblock_alloc_try_nid_raw &max_addr, (void *)_RET_IP_); ptr = memblock_alloc_internal(size, align, - min_addr, max_addr, nid); + min_addr, max_addr, nid, false); if (ptr && size > 0) page_init_poison(ptr, size); @@ -1543,7 +1586,7 @@ void * __init memblock_alloc_try_nid( __func__, (u64)size, (u64)align, nid, &min_addr, &max_addr, (void *)_RET_IP_); ptr = memblock_alloc_internal(size, align, - min_addr, max_addr, nid); + min_addr, max_addr, nid, false); if (ptr) memset(ptr, 0, size); --- a/mm/sparse.c~mm-support-memblock-alloc-on-the-exact-node-for-sparse_buffer_init +++ a/mm/sparse.c @@ -486,7 +486,7 @@ static void __init sparse_buffer_init(un * and we want it to be properly aligned to the section size - this is * especially the case for VMEMMAP which maps memmap to PMDs */ - sparsemap_buf = memblock_alloc_try_nid_raw(size, section_map_size(), + sparsemap_buf = memblock_alloc_exact_nid_raw(size, section_map_size(), addr, MEMBLOCK_ALLOC_ACCESSIBLE, nid); sparsemap_buf_end = sparsemap_buf + size; } From patchwork Sun Dec 1 01:56:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268423 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DB22E17E0 for ; Sun, 1 Dec 2019 01:56:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9A9662082E for ; Sun, 1 Dec 2019 01:56:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="skJsZ9BV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9A9662082E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 50D066B0354; Sat, 30 Nov 2019 20:56:33 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4BD516B0356; Sat, 30 Nov 2019 20:56:33 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3D2D16B0357; Sat, 30 Nov 2019 20:56:33 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0041.hostedemail.com [216.40.44.41]) by kanga.kvack.org (Postfix) with ESMTP id 1D71E6B0354 for ; Sat, 30 Nov 2019 20:56:33 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id CEB5040F4 for ; Sun, 1 Dec 2019 01:56:32 +0000 (UTC) X-FDA: 76214908224.16.tax99_391d45a77450b X-Spam-Summary: 2,0,0,072f1db05e81f15d,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:david.bolvansky@gmail.com:dbueso@suse.de:ilie.halip@gmail.com::mike.kravetz@oracle.com:mm-commits@vger.kernel.org:natechancellor@gmail.com:ndesaulniers@google.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1535:1544:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4605:5007:6261:6653:7514:7576:7903:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:12555:12679:12783:12986:13221:13229:13846:14181:14721:14849:21063:21067:21080:21324:21451:21627:21939:30003:30012:30034:30054:30064:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,Dom ainCache X-HE-Tag: tax99_391d45a77450b X-Filterd-Recvd-Size: 5970 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:32 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 617632231B; Sun, 1 Dec 2019 01:56:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165391; bh=f1FV1pslOsBeDUz9OVAHXmZP0r+6iqfdf4Ra/+U191s=; h=Date:From:To:Subject:From; b=skJsZ9BV8XrzN1PLnhCFgFKC0HIFBxfTJnl4SH3TwsNco1knyJqsrYScGKVcNoAuS M0IUFvUlW55uHYQgjAAyYCzRx8BdJyy8uMLs6d0DkkOW0pTItVfMxRwpFpRr7iL9hL 8SuHJuPv5ukMZ2jP096/xcOaZt0gn7KXtw8aevmc= Date: Sat, 30 Nov 2019 17:56:30 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, david.bolvansky@gmail.com, dbueso@suse.de, ilie.halip@gmail.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, natechancellor@gmail.com, ndesaulniers@google.com, torvalds@linux-foundation.org Subject: [patch 123/158] hugetlbfs: hugetlb_fault_mutex_hash() cleanup Message-ID: <20191201015630.9_G7OSXVU%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: hugetlbfs: hugetlb_fault_mutex_hash() cleanup A new clang diagnostic (-Wsizeof-array-div) warns about the calculation to determine the number of u32's in an array of unsigned longs. Suppress warning by adding parentheses. While looking at the above issue, noticed that the 'address' parameter to hugetlb_fault_mutex_hash is no longer used. So, remove it from the definition and all callers. No functional change. Link: http://lkml.kernel.org/r/20190919011847.18400-1-mike.kravetz@oracle.com Signed-off-by: Mike Kravetz Reported-by: Nathan Chancellor Reviewed-by: Nathan Chancellor Reviewed-by: Davidlohr Bueso Reviewed-by: Andrew Morton Cc: Nick Desaulniers Cc: Ilie Halip Cc: David Bolvansky Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 4 ++-- include/linux/hugetlb.h | 2 +- mm/hugetlb.c | 10 +++++----- mm/userfaultfd.c | 2 +- 4 files changed, 9 insertions(+), 9 deletions(-) --- a/fs/hugetlbfs/inode.c~hugetlbfs-hugetlb_fault_mutex_hash-cleanup +++ a/fs/hugetlbfs/inode.c @@ -440,7 +440,7 @@ static void remove_inode_hugepages(struc u32 hash; index = page->index; - hash = hugetlb_fault_mutex_hash(h, mapping, index, 0); + hash = hugetlb_fault_mutex_hash(h, mapping, index); mutex_lock(&hugetlb_fault_mutex_table[hash]); /* @@ -644,7 +644,7 @@ static long hugetlbfs_fallocate(struct f addr = index * hpage_size; /* mutex taken here, fault path and hole punch */ - hash = hugetlb_fault_mutex_hash(h, mapping, index, addr); + hash = hugetlb_fault_mutex_hash(h, mapping, index); mutex_lock(&hugetlb_fault_mutex_table[hash]); /* See if already present in mapping to avoid alloc/free */ --- a/include/linux/hugetlb.h~hugetlbfs-hugetlb_fault_mutex_hash-cleanup +++ a/include/linux/hugetlb.h @@ -106,7 +106,7 @@ void free_huge_page(struct page *page); void hugetlb_fix_reserve_counts(struct inode *inode); extern struct mutex *hugetlb_fault_mutex_table; u32 hugetlb_fault_mutex_hash(struct hstate *h, struct address_space *mapping, - pgoff_t idx, unsigned long address); + pgoff_t idx); pte_t *huge_pmd_share(struct mm_struct *mm, unsigned long addr, pud_t *pud); --- a/mm/hugetlb.c~hugetlbfs-hugetlb_fault_mutex_hash-cleanup +++ a/mm/hugetlb.c @@ -3842,7 +3842,7 @@ retry: * handling userfault. Reacquire after handling * fault to make calling code simpler. */ - hash = hugetlb_fault_mutex_hash(h, mapping, idx, haddr); + hash = hugetlb_fault_mutex_hash(h, mapping, idx); mutex_unlock(&hugetlb_fault_mutex_table[hash]); ret = handle_userfault(&vmf, VM_UFFD_MISSING); mutex_lock(&hugetlb_fault_mutex_table[hash]); @@ -3970,7 +3970,7 @@ backout_unlocked: #ifdef CONFIG_SMP u32 hugetlb_fault_mutex_hash(struct hstate *h, struct address_space *mapping, - pgoff_t idx, unsigned long address) + pgoff_t idx) { unsigned long key[2]; u32 hash; @@ -3978,7 +3978,7 @@ u32 hugetlb_fault_mutex_hash(struct hsta key[0] = (unsigned long) mapping; key[1] = idx; - hash = jhash2((u32 *)&key, sizeof(key)/sizeof(u32), 0); + hash = jhash2((u32 *)&key, sizeof(key)/(sizeof(u32)), 0); return hash & (num_fault_mutexes - 1); } @@ -3988,7 +3988,7 @@ u32 hugetlb_fault_mutex_hash(struct hsta * return 0 and avoid the hashing overhead. */ u32 hugetlb_fault_mutex_hash(struct hstate *h, struct address_space *mapping, - pgoff_t idx, unsigned long address) + pgoff_t idx) { return 0; } @@ -4032,7 +4032,7 @@ vm_fault_t hugetlb_fault(struct mm_struc * get spurious allocation failures if two CPUs race to instantiate * the same page in the page cache. */ - hash = hugetlb_fault_mutex_hash(h, mapping, idx, haddr); + hash = hugetlb_fault_mutex_hash(h, mapping, idx); mutex_lock(&hugetlb_fault_mutex_table[hash]); entry = huge_ptep_get(ptep); --- a/mm/userfaultfd.c~hugetlbfs-hugetlb_fault_mutex_hash-cleanup +++ a/mm/userfaultfd.c @@ -269,7 +269,7 @@ retry: */ idx = linear_page_index(dst_vma, dst_addr); mapping = dst_vma->vm_file->f_mapping; - hash = hugetlb_fault_mutex_hash(h, mapping, idx, dst_addr); + hash = hugetlb_fault_mutex_hash(h, mapping, idx); mutex_lock(&hugetlb_fault_mutex_table[hash]); err = -ENOMEM; From patchwork Sun Dec 1 01:56:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268425 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BF476921 for ; Sun, 1 Dec 2019 01:56:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8C2DB205ED for ; Sun, 1 Dec 2019 01:56:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="SeDGABc7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8C2DB205ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 694CE6B0356; Sat, 30 Nov 2019 20:56:36 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 61FAD6B0358; Sat, 30 Nov 2019 20:56:36 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4973B6B0359; Sat, 30 Nov 2019 20:56:36 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0074.hostedemail.com [216.40.44.74]) by kanga.kvack.org (Postfix) with ESMTP id 307806B0356 for ; Sat, 30 Nov 2019 20:56:36 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id E663D181AEF0B for ; Sun, 1 Dec 2019 01:56:35 +0000 (UTC) X-FDA: 76214908350.27.ice69_39936ea11d63c X-Spam-Summary: 2,0,0,b77a849b08f82863,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:cgxu519@mykernel.net:dhowells@redhat.com::mike.kravetz@oracle.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org:viro@zeniv.linux.org.uk,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2693:2859:2898:2899:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:5007:6119:6261:6653:7576:9025:9545:10004:11026:11257:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13141:13221:13229:13230:13846:13870:13972:14181:14721:14849:21080:21433:21450:21451:21627:21939:30012:30054:30064:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_ rules:0: X-HE-Tag: ice69_39936ea11d63c X-Filterd-Recvd-Size: 3761 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:35 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9CDE02245C; Sun, 1 Dec 2019 01:56:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165395; bh=o/eH1qjEbTPSu3YScR5azrGpL5hdpewneNtOsaL5NzA=; h=Date:From:To:Subject:From; b=SeDGABc7NT6Umvskqvtr8F7b8RIRbA5LbK8CfvSjWn1UjWW98kdvOl1w+oy3xCIU9 D0qqyV0FDc/DPjn/xZo9eFS/7eyjp6ryzO9ugfCFzAmsAKxnUpqSONpkoPrhaDbbM1 8u/roByWt704JZB1YS7eoEdLqnSfrmOpl0sYLv/0= Date: Sat, 30 Nov 2019 17:56:34 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, cgxu519@mykernel.net, dhowells@redhat.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 124/158] mm/hugetlbfs: fix error handling when setting up mounts Message-ID: <20191201015634.09yQyxNwN%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: mm/hugetlbfs: fix error handling when setting up mounts It is assumed that the hugetlbfs_vfsmount[] array will contain either a valid vfsmount pointer or NULL for each hstate after initialization. Changes made while converting to use fs_context broke this assumption. While fixing the hugetlbfs_vfsmount issue, it was discovered that init_hugetlbfs_fs never did correctly clean up when encountering a vfs mount error. It was found during code inspection. A small memory allocation failure would be the most likely cause of taking a error path with the bug. This is unlikely to happen as this is early init code. Link: http://lkml.kernel.org/r/94b6244d-2c24-e269-b12c-e3ba694b242d@oracle.com Reported-by: Chengguang Xu Fixes: 32021982a324 ("hugetlbfs: Convert to fs_context") Signed-off-by: Mike Kravetz Cc: David Howells Cc: Al Viro Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 31 ++++++++++++++++++++++--------- 1 file changed, 22 insertions(+), 9 deletions(-) --- a/fs/hugetlbfs/inode.c~mm-hugetlbfs-fix-error-handling-when-setting-up-mounts +++ a/fs/hugetlbfs/inode.c @@ -1461,28 +1461,41 @@ static int __init init_hugetlbfs_fs(void sizeof(struct hugetlbfs_inode_info), 0, SLAB_ACCOUNT, init_once); if (hugetlbfs_inode_cachep == NULL) - goto out2; + goto out; error = register_filesystem(&hugetlbfs_fs_type); if (error) - goto out; + goto out_free; + + /* default hstate mount is required */ + mnt = mount_one_hugetlbfs(&hstates[default_hstate_idx]); + if (IS_ERR(mnt)) { + error = PTR_ERR(mnt); + goto out_unreg; + } + hugetlbfs_vfsmount[default_hstate_idx] = mnt; + /* other hstates are optional */ i = 0; for_each_hstate(h) { + if (i == default_hstate_idx) + continue; + mnt = mount_one_hugetlbfs(h); - if (IS_ERR(mnt) && i == 0) { - error = PTR_ERR(mnt); - goto out; - } - hugetlbfs_vfsmount[i] = mnt; + if (IS_ERR(mnt)) + hugetlbfs_vfsmount[i] = NULL; + else + hugetlbfs_vfsmount[i] = mnt; i++; } return 0; - out: + out_unreg: + (void)unregister_filesystem(&hugetlbfs_fs_type); + out_free: kmem_cache_destroy(hugetlbfs_inode_cachep); - out2: + out: return error; } fs_initcall(init_hugetlbfs_fs) From patchwork Sun Dec 1 01:56:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268427 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CD25F17E0 for ; Sun, 1 Dec 2019 01:56:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9AA8C205ED for ; Sun, 1 Dec 2019 01:56:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="07liVXMi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9AA8C205ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6B9466B0358; Sat, 30 Nov 2019 20:56:39 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 669096B035A; Sat, 30 Nov 2019 20:56:39 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 53B3C6B035B; Sat, 30 Nov 2019 20:56:39 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0188.hostedemail.com [216.40.44.188]) by kanga.kvack.org (Postfix) with ESMTP id 36CE16B0358 for ; Sat, 30 Nov 2019 20:56:39 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 00436181AEF0B for ; Sun, 1 Dec 2019 01:56:38 +0000 (UTC) X-FDA: 76214908476.30.wood56_3a06f9e842427 X-Spam-Summary: 2,0,0,c6a8cc69c19b4d9c,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:ben.dooks@codethink.co.uk:jgg@ziepe.ca::mike.kravetz@oracle.com:mm-commits@vger.kernel.org:mpe@ellerman.id.au:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4605:5007:6261:6653:7576:7903:8599:9025:9545:10004:10913:11026:11473:11657:11658:11914:12043:12048:12294:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:13255:13846:14093:14181:14721:14849:21080:21433:21451:21627:21939:30054:30064:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: wood56_3a06f9e842427 X-Filterd-Recvd-Size: 4249 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:38 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C03E9215F1; Sun, 1 Dec 2019 01:56:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165398; bh=uPTonfuwi4BA72Ekh3plL+XMoMqaJXOBm36svPLV29o=; h=Date:From:To:Subject:From; b=07liVXMi4/qhwY/DgE/QFl+TjplyQlvWNzlFKwQ/gUaDL4neTnyiJegBmseeTpFVs ON5mz/tM2/uVDjQ+OdXCdCnKiJ6Wg4M2vooyGF34Qhsn41QKpthGBsAcnmKDgTN1v2 /5Bzj03l+uh6oN1Qceg1RRC4CCnA2epjhbxrwP+Y= Date: Sat, 30 Nov 2019 17:56:37 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, ben.dooks@codethink.co.uk, jgg@ziepe.ca, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, mpe@ellerman.id.au, torvalds@linux-foundation.org Subject: [patch 125/158] powerpc/mm: remove pmd_huge/pud_huge stubs and include hugetlb.h Message-ID: <20191201015637.F-2eOeVoG%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: powerpc/mm: remove pmd_huge/pud_huge stubs and include hugetlb.h Patch series "hugetlbfs: convert macros to static inline, fix sparse warning". The definition for huge_pte_offset() in causes a sparse warning in the !CONFIG_HUGETLB_PAGE. Fix this as well as converting all macros in this block of definitions to static inlines for better type checking. When making the above changes, build errors were found in powerpc due to duplicate definitions. A separate powerpc specific patch is included as a requisite to remove the definitions and get them from . This patch (of 2): This removes the power specific stubs created by commit aad71e3928be ("powerpc/mm: Fix build break with RADIX=y & HUGETLBFS=n") used when !CONFIG_HUGETLB_PAGE. Instead, it addresses the build break by getting the definitions from . This allows the macros in to be replaced with static inlines. Link: http://lkml.kernel.org/r/20191112194558.139389-2-mike.kravetz@oracle.com Signed-off-by: Mike Kravetz Acked-by: Michael Ellerman Cc: Ben Dooks Cc: Jason Gunthorpe Signed-off-by: Andrew Morton --- arch/powerpc/include/asm/book3s/64/pgtable-4k.h | 3 --- arch/powerpc/include/asm/book3s/64/pgtable-64k.h | 3 --- arch/powerpc/mm/book3s64/radix_pgtable.c | 1 + 3 files changed, 1 insertion(+), 6 deletions(-) --- a/arch/powerpc/include/asm/book3s/64/pgtable-4k.h~powerpc-mm-remove-pmd_huge-pud_huge-stubs-and-include-hugetlbh +++ a/arch/powerpc/include/asm/book3s/64/pgtable-4k.h @@ -70,9 +70,6 @@ static inline int get_hugepd_cache_index /* should not reach */ } -#else /* !CONFIG_HUGETLB_PAGE */ -static inline int pmd_huge(pmd_t pmd) { return 0; } -static inline int pud_huge(pud_t pud) { return 0; } #endif /* CONFIG_HUGETLB_PAGE */ #endif /* __ASSEMBLY__ */ --- a/arch/powerpc/include/asm/book3s/64/pgtable-64k.h~powerpc-mm-remove-pmd_huge-pud_huge-stubs-and-include-hugetlbh +++ a/arch/powerpc/include/asm/book3s/64/pgtable-64k.h @@ -59,9 +59,6 @@ static inline int get_hugepd_cache_index BUG(); } -#else /* !CONFIG_HUGETLB_PAGE */ -static inline int pmd_huge(pmd_t pmd) { return 0; } -static inline int pud_huge(pud_t pud) { return 0; } #endif /* CONFIG_HUGETLB_PAGE */ static inline int remap_4k_pfn(struct vm_area_struct *vma, unsigned long addr, --- a/arch/powerpc/mm/book3s64/radix_pgtable.c~powerpc-mm-remove-pmd_huge-pud_huge-stubs-and-include-hugetlbh +++ a/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include From patchwork Sun Dec 1 01:56:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268429 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 118C9921 for ; Sun, 1 Dec 2019 01:56:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C4BBB20873 for ; Sun, 1 Dec 2019 01:56:43 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="gIZ5/ljz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C4BBB20873 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C10A76B035A; Sat, 30 Nov 2019 20:56:42 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B9B3E6B035C; Sat, 30 Nov 2019 20:56:42 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A8BF86B035D; Sat, 30 Nov 2019 20:56:42 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0021.hostedemail.com [216.40.44.21]) by kanga.kvack.org (Postfix) with ESMTP id 8D7826B035A for ; Sat, 30 Nov 2019 20:56:42 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 4F3A3180AD817 for ; Sun, 1 Dec 2019 01:56:42 +0000 (UTC) X-FDA: 76214908644.16.force97_3a7cf19b27a03 X-Spam-Summary: 2,0,0,95b580f323ef006a,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:ben.dooks@codethink.co.uk:jgg@ziepe.ca::mike.kravetz@oracle.com:mm-commits@vger.kernel.org:mpe@ellerman.id.au:torvalds@linux-foundation.org,RULES_HIT:2:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1535:1605:1606:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4117:4321:4385:5007:6261:6653:7576:8599:9010:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13221:13229:13846:14096:21080:21433:21451:21627:21740:21939:30003:30054:30064:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: force97_3a7cf19b27a03 X-Filterd-Recvd-Size: 6787 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:41 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id DA23B205ED; Sun, 1 Dec 2019 01:56:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165401; bh=KuGcGchCYk3yndZAIA7TWHVo+PZPwPz74T+3Sp7KzuM=; h=Date:From:To:Subject:From; b=gIZ5/ljzwpD/yp1eNoZLVZTLHtDS18aHyyB0G4BaZxsKoQ2R17TZFfYj1h/De7ew5 WVZ4wne377UWbuAubJ8td5bQBJnJMP8XcZShsH0FogCBQAx0FViHAqBbiHd/VZEFS2 yNRR/8it8K5e/dgOuYTvljPNIGoxhUF/U65Qe4WA= Date: Sat, 30 Nov 2019 17:56:40 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, ben.dooks@codethink.co.uk, jgg@ziepe.ca, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, mpe@ellerman.id.au, torvalds@linux-foundation.org Subject: [patch 126/158] hugetlbfs: convert macros to static inline, fix sparse warning Message-ID: <20191201015640.3h8dI1qa8%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: hugetlbfs: convert macros to static inline, fix sparse warning huge_pte_offset() produced a sparse warning due to an improper return type when the kernel was built with !CONFIG_HUGETLB_PAGE. Fix the bad type and also convert all the macros in this block to static inline wrappers. Two existing wrappers in this block had lines in excess of 80 columns so clean those up as well. No functional change. Link: http://lkml.kernel.org/r/20191112194558.139389-3-mike.kravetz@oracle.com Signed-off-by: Mike Kravetz Reported-by: Ben Dooks Suggested-by: Jason Gunthorpe Cc: Michael Ellerman Signed-off-by: Andrew Morton --- include/linux/hugetlb.h | 137 +++++++++++++++++++++++++++++++------- 1 file changed, 115 insertions(+), 22 deletions(-) --- a/include/linux/hugetlb.h~hugetlbfs-convert-macros-to-static-inline-fix-sparse-warning +++ a/include/linux/hugetlb.h @@ -164,38 +164,130 @@ static inline void adjust_range_if_pmd_s { } -#define follow_hugetlb_page(m,v,p,vs,a,b,i,w,n) ({ BUG(); 0; }) -#define follow_huge_addr(mm, addr, write) ERR_PTR(-EINVAL) -#define copy_hugetlb_page_range(src, dst, vma) ({ BUG(); 0; }) +static inline long follow_hugetlb_page(struct mm_struct *mm, + struct vm_area_struct *vma, struct page **pages, + struct vm_area_struct **vmas, unsigned long *position, + unsigned long *nr_pages, long i, unsigned int flags, + int *nonblocking) +{ + BUG(); + return 0; +} + +static inline struct page *follow_huge_addr(struct mm_struct *mm, + unsigned long address, int write) +{ + return ERR_PTR(-EINVAL); +} + +static inline int copy_hugetlb_page_range(struct mm_struct *dst, + struct mm_struct *src, struct vm_area_struct *vma) +{ + BUG(); + return 0; +} + static inline void hugetlb_report_meminfo(struct seq_file *m) { } -#define hugetlb_report_node_meminfo(n, buf) 0 + +static inline int hugetlb_report_node_meminfo(int nid, char *buf) +{ + return 0; +} + static inline void hugetlb_show_meminfo(void) { } -#define follow_huge_pd(vma, addr, hpd, flags, pdshift) NULL -#define follow_huge_pmd(mm, addr, pmd, flags) NULL -#define follow_huge_pud(mm, addr, pud, flags) NULL -#define follow_huge_pgd(mm, addr, pgd, flags) NULL -#define prepare_hugepage_range(file, addr, len) (-EINVAL) -#define pmd_huge(x) 0 -#define pud_huge(x) 0 -#define is_hugepage_only_range(mm, addr, len) 0 -#define hugetlb_free_pgd_range(tlb, addr, end, floor, ceiling) ({BUG(); 0; }) -#define hugetlb_mcopy_atomic_pte(dst_mm, dst_pte, dst_vma, dst_addr, \ - src_addr, pagep) ({ BUG(); 0; }) -#define huge_pte_offset(mm, address, sz) 0 + +static inline struct page *follow_huge_pd(struct vm_area_struct *vma, + unsigned long address, hugepd_t hpd, int flags, + int pdshift) +{ + return NULL; +} + +static inline struct page *follow_huge_pmd(struct mm_struct *mm, + unsigned long address, pmd_t *pmd, int flags) +{ + return NULL; +} + +static inline struct page *follow_huge_pud(struct mm_struct *mm, + unsigned long address, pud_t *pud, int flags) +{ + return NULL; +} + +static inline struct page *follow_huge_pgd(struct mm_struct *mm, + unsigned long address, pgd_t *pgd, int flags) +{ + return NULL; +} + +static inline int prepare_hugepage_range(struct file *file, + unsigned long addr, unsigned long len) +{ + return -EINVAL; +} + +static inline int pmd_huge(pmd_t pmd) +{ + return 0; +} + +static inline int pud_huge(pud_t pud) +{ + return 0; +} + +static inline int is_hugepage_only_range(struct mm_struct *mm, + unsigned long addr, unsigned long len) +{ + return 0; +} + +static inline void hugetlb_free_pgd_range(struct mmu_gather *tlb, + unsigned long addr, unsigned long end, + unsigned long floor, unsigned long ceiling) +{ + BUG(); +} + +static inline int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, + pte_t *dst_pte, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, + unsigned long src_addr, + struct page **pagep) +{ + BUG(); + return 0; +} + +static inline pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr, + unsigned long sz) +{ + return NULL; +} static inline bool isolate_huge_page(struct page *page, struct list_head *list) { return false; } -#define putback_active_hugepage(p) do {} while (0) -#define move_hugetlb_state(old, new, reason) do {} while (0) -static inline unsigned long hugetlb_change_protection(struct vm_area_struct *vma, - unsigned long address, unsigned long end, pgprot_t newprot) +static inline void putback_active_hugepage(struct page *page) +{ +} + +static inline void move_hugetlb_state(struct page *oldpage, + struct page *newpage, int reason) +{ +} + +static inline unsigned long hugetlb_change_protection( + struct vm_area_struct *vma, unsigned long address, + unsigned long end, pgprot_t newprot) { return 0; } @@ -213,9 +305,10 @@ static inline void __unmap_hugepage_rang { BUG(); } + static inline vm_fault_t hugetlb_fault(struct mm_struct *mm, - struct vm_area_struct *vma, unsigned long address, - unsigned int flags) + struct vm_area_struct *vma, unsigned long address, + unsigned int flags) { BUG(); return 0; From patchwork Sun Dec 1 01:56:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268431 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8DEAA17E0 for ; Sun, 1 Dec 2019 01:56:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5179920873 for ; Sun, 1 Dec 2019 01:56:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="zM9jxogX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5179920873 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7A35F6B035C; Sat, 30 Nov 2019 20:56:48 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 753836B035E; Sat, 30 Nov 2019 20:56:48 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6915D6B035F; Sat, 30 Nov 2019 20:56:48 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0232.hostedemail.com [216.40.44.232]) by kanga.kvack.org (Postfix) with ESMTP id 504EB6B035C for ; Sat, 30 Nov 2019 20:56:48 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 00444180AD817 for ; Sun, 1 Dec 2019 01:56:47 +0000 (UTC) X-FDA: 76214908854.16.drain48_3b52206931529 X-Spam-Summary: 2,0,0,235fef32f2721f35,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org::mhocko@kernel.org:mike.kravetz@oracle.com:mm-commits@vger.kernel.org:p.sarna@tlen.pl:torvalds@linux-foundation.org:viro@zeniv.linux.org.uk,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1543:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2914:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:6678:7576:7903:9025:9545:10004:11026:11257:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:13846:14096:14181:14721:14849:21080:21220:21451:21611:21627:21796:21939:30036:30054:30064,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: drain48_3b52206931529 X-Filterd-Recvd-Size: 4550 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:47 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AA90B2082E; Sun, 1 Dec 2019 01:56:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165407; bh=8IkV0xdigG+RxHdvudRBcYzmRn7IPMSuzYRw3AQUcpM=; h=Date:From:To:Subject:From; b=zM9jxogXgR8lRY6xr9AJPAJHZAfyLzdARLyVuNAyT6VF8YEaL2p+eqs9yNmEQuZKA yZVYGZ94YY2Pr3qdTHlhTi739BMqiMEiu3UTIr516JJ6wwSIu9tkxfwKOk4AuXRbxb QXkoWOmb1epxPdR8iZVJOlo4917Nvlbhfc7pAxss= Date: Sat, 30 Nov 2019 17:56:43 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, linux-mm@kvack.org, mhocko@kernel.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, p.sarna@tlen.pl, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 127/158] hugetlbfs: add O_TMPFILE support Message-ID: <20191201015643.6ro8I2lI-%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Piotr Sarna Subject: hugetlbfs: add O_TMPFILE support With hugetlbfs, a common pattern for mapping anonymous huge pages is to create a temporary file first. Currently libraries like libhugetlbfs and seastar create these with a standard mkstemp+unlink trick, but it would be more robust to be able to simply pass the O_TMPFILE flag to open(). O_TMPFILE is already supported by several file systems like ext4 and xfs. The implementation simply uses the existi= ng d_tmpfile utility function to instantiate the dcache entry for the file. Tested manually by successfully creating a temporary file by opening it with (O_TMPFILE|O_RDWR) on mounted hugetlbfs and successfully mapping 2M huge pages with it. Without the patch, trying to open a file with O_TMPFILE results in -ENOSUP. Link: http://lkml.kernel.org/r/bc9383eff6e1374d79f3a92257ae829ba1e6ae60.1573285189.git.p.sarna@tlen.pl Signed-off-by: Piotr Sarna Reviewed-by: Mike Kravetz Cc: Al Viro Cc: Michal Hocko Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) --- a/fs/hugetlbfs/inode.c~hugetlbfs-add-o_tmpfile-support +++ a/fs/hugetlbfs/inode.c @@ -815,8 +815,11 @@ static struct inode *hugetlbfs_get_inode /* * File creation. Allocate an inode, and we're done.. */ -static int hugetlbfs_mknod(struct inode *dir, - struct dentry *dentry, umode_t mode, dev_t dev) +static int do_hugetlbfs_mknod(struct inode *dir, + struct dentry *dentry, + umode_t mode, + dev_t dev, + bool tmpfile) { struct inode *inode; int error = -ENOSPC; @@ -824,13 +827,23 @@ static int hugetlbfs_mknod(struct inode inode = hugetlbfs_get_inode(dir->i_sb, dir, mode, dev); if (inode) { dir->i_ctime = dir->i_mtime = current_time(dir); - d_instantiate(dentry, inode); - dget(dentry); /* Extra count - pin the dentry in core */ + if (tmpfile) { + d_tmpfile(dentry, inode); + } else { + d_instantiate(dentry, inode); + dget(dentry);/* Extra count - pin the dentry in core */ + } error = 0; } return error; } +static int hugetlbfs_mknod(struct inode *dir, + struct dentry *dentry, umode_t mode, dev_t dev) +{ + return do_hugetlbfs_mknod(dir, dentry, mode, dev, false); +} + static int hugetlbfs_mkdir(struct inode *dir, struct dentry *dentry, umode_t mode) { int retval = hugetlbfs_mknod(dir, dentry, mode | S_IFDIR, 0); @@ -844,6 +857,12 @@ static int hugetlbfs_create(struct inode return hugetlbfs_mknod(dir, dentry, mode | S_IFREG, 0); } +static int hugetlbfs_tmpfile(struct inode *dir, + struct dentry *dentry, umode_t mode) +{ + return do_hugetlbfs_mknod(dir, dentry, mode | S_IFREG, 0, true); +} + static int hugetlbfs_symlink(struct inode *dir, struct dentry *dentry, const char *symname) { @@ -1102,6 +1121,7 @@ static const struct inode_operations hug .mknod = hugetlbfs_mknod, .rename = simple_rename, .setattr = hugetlbfs_setattr, + .tmpfile = hugetlbfs_tmpfile, }; static const struct inode_operations hugetlbfs_inode_operations = { From patchwork Sun Dec 1 01:56:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268433 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CDF9917E0 for ; Sun, 1 Dec 2019 01:56:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9110E215E5 for ; Sun, 1 Dec 2019 01:56:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="GHWZZ/kf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9110E215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B4C8B6B035E; Sat, 30 Nov 2019 20:56:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B24236B0360; Sat, 30 Nov 2019 20:56:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A148B6B0361; Sat, 30 Nov 2019 20:56:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0138.hostedemail.com [216.40.44.138]) by kanga.kvack.org (Postfix) with ESMTP id 87F786B035E for ; Sat, 30 Nov 2019 20:56:53 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 315DD824999B for ; Sun, 1 Dec 2019 01:56:53 +0000 (UTC) X-FDA: 76214909106.09.smell80_3c17e89c5c342 X-Spam-Summary: 2,0,0,0e880932872935ab,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:dave@stgolabs.net::longman@redhat.com:mike.kravetz@oracle.com:mingo@redhat.com:mm-commits@vger.kernel.org:peterz@infradead.org:torvalds@linux-foundation.org:will.deacon@arm.com:willy@infradead.org,RULES_HIT:41:355:379:421:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2693:2859:2898:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6119:6120:6261:6630:6653:6737:7576:7901:7903:8599:9025:9545:10004:10913:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:13846:13869:14037:14181:14721:14849:21063:21080:21222:21324:21451:21627:21939:30005:30054:30064:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCach e:0,MSF: X-HE-Tag: smell80_3c17e89c5c342 X-Filterd-Recvd-Size: 3582 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:52 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 45325205ED; Sun, 1 Dec 2019 01:56:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165412; bh=IwRVlB4vwjLVk8uI3zroRyCOxTdPijo+Y0tZsYNIwgY=; h=Date:From:To:Subject:From; b=GHWZZ/kfhrr2uqDtsSdtnysXyFIywBJ0Jr+0jpOS6yEMuw9dVSkYrRHUlRhTNooob WC1RWnEr0uJ/qNFmxPKp9OdpgmZfRSi/ozx2op+R6GtHF6E5JkrG2wwg3LfEkh36yG TpTdz2zdhditelrf59UgeYsLVSllTmTCwbhBfZLY= Date: Sat, 30 Nov 2019 17:56:49 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, dave@stgolabs.net, linux-mm@kvack.org, longman@redhat.com, mike.kravetz@oracle.com, mingo@redhat.com, mm-commits@vger.kernel.org, peterz@infradead.org, torvalds@linux-foundation.org, will.deacon@arm.com, willy@infradead.org Subject: [patch 128/158] hugetlbfs: take read_lock on i_mmap for PMD sharing Message-ID: <20191201015649.nrcYKXX9H%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Waiman Long Subject: hugetlbfs: take read_lock on i_mmap for PMD sharing A customer with large SMP systems (up to 16 sockets) with application that uses large amount of static hugepages (~500-1500GB) are experiencing random multisecond delays. These delays were caused by the long time it took to scan the VMA interval tree with mmap_sem held. The sharing of huge PMD does not require changes to the i_mmap at all. Therefore, we can just take the read lock and let other threads searching for the right VMA share it in parallel. Once the right VMA is found, either the PMD lock (2M huge page for x86-64) or the mm->page_table_lock will be acquired to perform the actual PMD sharing. Lock contention, if present, will happen in the spinlock. That is much better than contention in the rwsem where the time needed to scan the the interval tree is indeterminate. With this patch applied, the customer is seeing significant performance improvement over the unpatched kernel. Link: http://lkml.kernel.org/r/20191107211809.9539-1-longman@redhat.com Signed-off-by: Waiman Long Suggested-by: Mike Kravetz Reviewed-by: Mike Kravetz Cc: Davidlohr Bueso Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Will Deacon Cc: Matthew Wilcox Signed-off-by: Andrew Morton --- mm/hugetlb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/mm/hugetlb.c~hugetlbfs-take-read_lock-on-i_mmap-for-pmd-sharing +++ a/mm/hugetlb.c @@ -4769,7 +4769,7 @@ pte_t *huge_pmd_share(struct mm_struct * if (!vma_shareable(vma, addr)) return (pte_t *)pmd_alloc(mm, pud, addr); - i_mmap_lock_write(mapping); + i_mmap_lock_read(mapping); vma_interval_tree_foreach(svma, &mapping->i_mmap, idx, idx) { if (svma == vma) continue; @@ -4799,7 +4799,7 @@ pte_t *huge_pmd_share(struct mm_struct * spin_unlock(ptl); out: pte = (pte_t *)pmd_alloc(mm, pud, addr); - i_mmap_unlock_write(mapping); + i_mmap_unlock_read(mapping); return pte; } From patchwork Sun Dec 1 01:56:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268435 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D0D0217E0 for ; Sun, 1 Dec 2019 01:56:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9414F20873 for ; Sun, 1 Dec 2019 01:56:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="zKmusuWL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9414F20873 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9C6356B0360; Sat, 30 Nov 2019 20:56:58 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 953036B0362; Sat, 30 Nov 2019 20:56:58 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 88C366B0363; Sat, 30 Nov 2019 20:56:58 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0072.hostedemail.com [216.40.44.72]) by kanga.kvack.org (Postfix) with ESMTP id 6E57A6B0360 for ; Sat, 30 Nov 2019 20:56:58 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 275E42A87 for ; Sun, 1 Dec 2019 01:56:58 +0000 (UTC) X-FDA: 76214909316.30.watch14_3ccb8275cc61a X-Spam-Summary: 2,0,0,c45ceb87fc3b818e,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:almasrymina@google.com:gthelen@google.com::mike.kravetz@oracle.com:mm-commits@vger.kernel.org:rientjes@google.com:shakeelb@google.com:torvalds@linux-foundation.org,RULES_HIT:2:41:69:355:379:800:960:966:967:968:973:982:988:989:1260:1263:1345:1381:1431:1437:1535:1605:1606:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4118:4321:4385:5007:6119:6261:6653:7576:7903:7904:8599:8603:8660:9010:9025:9545:9592:10004:10913:11026:11232:11473:11658:11914:12043:12048:12295:12296:12297:12517:12519:12555:12679:12683:12783:12986:13141:13148:13230:13846:14096:21080:21324:21451:21627:21740:21939:30012:30034:30054:30064:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCach e:0,MSF: X-HE-Tag: watch14_3ccb8275cc61a X-Filterd-Recvd-Size: 7236 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:56:57 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 46362205ED; Sun, 1 Dec 2019 01:56:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165417; bh=STVnvHPfhDITO2O4R7JGMd7F6UZcUMpfVAqf5P9eLc0=; h=Date:From:To:Subject:From; b=zKmusuWLKx1l2yiQwCEVT5OgJX3dxqFXPNbfhcBcha6P0ipDRBphfnB+4AUjAjbSQ 6DQaO8NUl1DVinkQPg7Ss0MW+RW/DBWarZvXtXpDykgmkcNsRzyQejM844QCzszkJV AJXSeZfzAsy+2FfrdrYA6qkBFJd3YB8mZF6Q+6yA= Date: Sat, 30 Nov 2019 17:56:54 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, almasrymina@google.com, gthelen@google.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, rientjes@google.com, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 129/158] hugetlb: region_chg provides only cache entry Message-ID: <20191201015654.AlAAC9Y5f%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mina Almasry Subject: hugetlb: region_chg provides only cache entry Current behavior is that region_chg provides both a cache entry in resv->region_cache, AND a placeholder entry in resv->regions. region_add first tries to use the placeholder, and if it finds that the placeholder has been deleted by a racing region_del call, it uses the cache entry. This behavior is completely unnecessary and is removed in this patch for a couple of reasons: 1. region_add needs to either find a cached file_region entry in resv->region_cache, or find an entry in resv->regions to expand. It does not need both. 2. region_chg adding a placeholder entry in resv->regions opens up a possible race with region_del, where region_chg adds a placeholder region in resv->regions, and this region is deleted by a racing call to region_del during region_chg execution or before region_add is called. Removing the race makes the code easier to reason about and maintain. In addition, a follow up patch in another series that disables region coalescing, which would be further complicated if the race with region_del exists. Link: http://lkml.kernel.org/r/20190919200428.188797-2-almasrymina@google.com Signed-off-by: Mina Almasry Reviewed-by: Mike Kravetz Cc: David Rientjes Cc: Shakeel Butt Cc: Greg Thelen Signed-off-by: Andrew Morton --- mm/hugetlb.c | 63 ++++++++----------------------------------------- 1 file changed, 11 insertions(+), 52 deletions(-) --- a/mm/hugetlb.c~hugetlb-region_chg-provides-only-cache-entry +++ a/mm/hugetlb.c @@ -246,14 +246,10 @@ struct file_region { /* * Add the huge page range represented by [f, t) to the reserve - * map. In the normal case, existing regions will be expanded - * to accommodate the specified range. Sufficient regions should - * exist for expansion due to the previous call to region_chg - * with the same range. However, it is possible that region_del - * could have been called after region_chg and modifed the map - * in such a way that no region exists to be expanded. In this - * case, pull a region descriptor from the cache associated with - * the map and use that for the new range. + * map. Existing regions will be expanded to accommodate the specified + * range, or a region will be taken from the cache. Sufficient regions + * must exist in the cache due to the previous call to region_chg with + * the same range. * * Return the number of new huge pages added to the map. This * number is greater than or equal to zero. @@ -272,9 +268,8 @@ static long region_add(struct resv_map * /* * If no region exists which can be expanded to include the - * specified range, the list must have been modified by an - * interleving call to region_del(). Pull a region descriptor - * from the cache and use it for this range. + * specified range, pull a region descriptor from the cache + * and use it for this range. */ if (&rg->link == head || t < rg->from) { VM_BUG_ON(resv->region_cache_count <= 0); @@ -339,15 +334,9 @@ out_locked: * call to region_add that will actually modify the reserve * map to add the specified range [f, t). region_chg does * not change the number of huge pages represented by the - * map. However, if the existing regions in the map can not - * be expanded to represent the new range, a new file_region - * structure is added to the map as a placeholder. This is - * so that the subsequent region_add call will have all the - * regions it needs and will not fail. - * - * Upon entry, region_chg will also examine the cache of region descriptors - * associated with the map. If there are not enough descriptors cached, one - * will be allocated for the in progress add operation. + * map. A new file_region structure is added to the cache + * as a placeholder, so that the subsequent region_add + * call will have all the regions it needs and will not fail. * * Returns the number of huge pages that need to be added to the existing * reservation map for the range [f, t). This number is greater or equal to @@ -357,10 +346,9 @@ out_locked: static long region_chg(struct resv_map *resv, long f, long t) { struct list_head *head = &resv->regions; - struct file_region *rg, *nrg = NULL; + struct file_region *rg; long chg = 0; -retry: spin_lock(&resv->lock); retry_locked: resv->adds_in_progress++; @@ -378,10 +366,8 @@ retry_locked: spin_unlock(&resv->lock); trg = kmalloc(sizeof(*trg), GFP_KERNEL); - if (!trg) { - kfree(nrg); + if (!trg) return -ENOMEM; - } spin_lock(&resv->lock); list_add(&trg->link, &resv->region_cache); @@ -394,28 +380,6 @@ retry_locked: if (f <= rg->to) break; - /* If we are below the current region then a new region is required. - * Subtle, allocate a new region at the position but make it zero - * size such that we can guarantee to record the reservation. */ - if (&rg->link == head || t < rg->from) { - if (!nrg) { - resv->adds_in_progress--; - spin_unlock(&resv->lock); - nrg = kmalloc(sizeof(*nrg), GFP_KERNEL); - if (!nrg) - return -ENOMEM; - - nrg->from = f; - nrg->to = f; - INIT_LIST_HEAD(&nrg->link); - goto retry; - } - - list_add(&nrg->link, rg->link.prev); - chg = t - f; - goto out_nrg; - } - /* Round our left edge to the current segment if it encloses us. */ if (f > rg->from) f = rg->from; @@ -440,11 +404,6 @@ retry_locked: out: spin_unlock(&resv->lock); - /* We already know we raced and no longer need the new region */ - kfree(nrg); - return chg; -out_nrg: - spin_unlock(&resv->lock); return chg; } From patchwork Sun Dec 1 01:56:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268437 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5FBA1195A for ; Sun, 1 Dec 2019 01:57:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 205B32084D for ; Sun, 1 Dec 2019 01:57:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="ChAi0I8B" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 205B32084D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 078606B0362; Sat, 30 Nov 2019 20:57:02 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 027776B0364; Sat, 30 Nov 2019 20:57:01 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E81326B0366; Sat, 30 Nov 2019 20:57:01 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0023.hostedemail.com [216.40.44.23]) by kanga.kvack.org (Postfix) with ESMTP id CF7076B0362 for ; Sat, 30 Nov 2019 20:57:01 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 858C4181AEF0B for ; Sun, 1 Dec 2019 01:57:01 +0000 (UTC) X-FDA: 76214909442.11.comb69_3d4a8afe70417 X-Spam-Summary: 2,0,0,8d6824c4caf3e45a,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:almasrymina@google.com:gthelen@google.com::mike.kravetz@oracle.com:mm-commits@vger.kernel.org:rientjes@google.com:shakeelb@google.com:torvalds@linux-foundation.org,RULES_HIT:2:41:69:355:379:800:960:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1535:1605:1606:1730:1747:1777:1792:2194:2196:2199:2200:2393:2525:2559:2563:2682:2685:2693:2859:2890:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4042:4117:4321:4385:4419:5007:6119:6261:6653:7576:7903:7904:8599:9010:9025:9545:9592:10004:10913:11026:11658:11914:12043:12048:12257:12291:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13846:14096:21080:21450:21451:21627:21939:30001:30012:30034:30054:30064:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk, SPF:fp,M X-HE-Tag: comb69_3d4a8afe70417 X-Filterd-Recvd-Size: 6400 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:01 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1D9CE205ED; Sun, 1 Dec 2019 01:57:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165420; bh=s/wx9Zi63EKyrdYcRZvB8VemzqhfC/U4sys0Rr6jS90=; h=Date:From:To:Subject:From; b=ChAi0I8BnmlZI4pytfTAyUTpRTPJ5jVsYMI+xSQ7gpdgEwGgqSiGhOMxzZVQkqVk8 utTyeHnBbnEUDuDo4WSE8YJcuw3/ZXvePDQrVJY2Osok4S+cbJ6jH3/nesVZ80qwRL yiG4u9/+LpENodzkzLQoR0OmdPImoiJrhN8h8j1s= Date: Sat, 30 Nov 2019 17:56:59 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, almasrymina@google.com, gthelen@google.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, rientjes@google.com, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 130/158] hugetlb: remove duplicated code Message-ID: <20191201015659.FITgNH5bV%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mina Almasry Subject: hugetlb: remove duplicated code Remove duplicated code between region_chg and region_add, and refactor it into a common function, add_reservation_in_range. This is mostly done because there is a follow up change in another series that disables region coalescing in region_add, and I want to make that change in one place only. It should improve maintainability anyway on its own. [akpm@linux-foundation.org: coding style fixes] Link: http://lkml.kernel.org/r/20190919200428.188797-3-almasrymina@google.com Signed-off-by: Mina Almasry Reviewed-by: Mike Kravetz Cc: David Rientjes Cc: Shakeel Butt Cc: Greg Thelen Signed-off-by: Andrew Morton --- mm/hugetlb.c | 119 +++++++++++++++++++++++-------------------------- 1 file changed, 57 insertions(+), 62 deletions(-) --- a/mm/hugetlb.c~hugetlb-remove-duplicated-code +++ a/mm/hugetlb.c @@ -244,6 +244,60 @@ struct file_region { long to; }; +/* Must be called with resv->lock held. Calling this with count_only == true + * will count the number of pages to be added but will not modify the linked + * list. + */ +static long add_reservation_in_range(struct resv_map *resv, long f, long t, + bool count_only) +{ + long chg = 0; + struct list_head *head = &resv->regions; + struct file_region *rg = NULL, *trg = NULL, *nrg = NULL; + + /* Locate the region we are before or in. */ + list_for_each_entry(rg, head, link) + if (f <= rg->to) + break; + + /* Round our left edge to the current segment if it encloses us. */ + if (f > rg->from) + f = rg->from; + + chg = t - f; + + /* Check for and consume any regions we now overlap with. */ + nrg = rg; + list_for_each_entry_safe(rg, trg, rg->link.prev, link) { + if (&rg->link == head) + break; + if (rg->from > t) + break; + + /* We overlap with this area, if it extends further than + * us then we must extend ourselves. Account for its + * existing reservation. + */ + if (rg->to > t) { + chg += rg->to - t; + t = rg->to; + } + chg -= rg->to - rg->from; + + if (!count_only && rg != nrg) { + list_del(&rg->link); + kfree(rg); + } + } + + if (!count_only) { + nrg->from = f; + nrg->to = t; + } + + return chg; +} + /* * Add the huge page range represented by [f, t) to the reserve * map. Existing regions will be expanded to accommodate the specified @@ -257,7 +311,7 @@ struct file_region { static long region_add(struct resv_map *resv, long f, long t) { struct list_head *head = &resv->regions; - struct file_region *rg, *nrg, *trg; + struct file_region *rg, *nrg; long add = 0; spin_lock(&resv->lock); @@ -287,38 +341,7 @@ static long region_add(struct resv_map * goto out_locked; } - /* Round our left edge to the current segment if it encloses us. */ - if (f > rg->from) - f = rg->from; - - /* Check for and consume any regions we now overlap with. */ - nrg = rg; - list_for_each_entry_safe(rg, trg, rg->link.prev, link) { - if (&rg->link == head) - break; - if (rg->from > t) - break; - - /* If this area reaches higher then extend our area to - * include it completely. If this is not the first area - * which we intend to reuse, free it. */ - if (rg->to > t) - t = rg->to; - if (rg != nrg) { - /* Decrement return value by the deleted range. - * Another range will span this area so that by - * end of routine add will be >= zero - */ - add -= (rg->to - rg->from); - list_del(&rg->link); - kfree(rg); - } - } - - add += (nrg->from - f); /* Added to beginning of region */ - nrg->from = f; - add += t - nrg->to; /* Added to end of region */ - nrg->to = t; + add = add_reservation_in_range(resv, f, t, false); out_locked: resv->adds_in_progress--; @@ -345,8 +368,6 @@ out_locked: */ static long region_chg(struct resv_map *resv, long f, long t) { - struct list_head *head = &resv->regions; - struct file_region *rg; long chg = 0; spin_lock(&resv->lock); @@ -375,34 +396,8 @@ retry_locked: goto retry_locked; } - /* Locate the region we are before or in. */ - list_for_each_entry(rg, head, link) - if (f <= rg->to) - break; - - /* Round our left edge to the current segment if it encloses us. */ - if (f > rg->from) - f = rg->from; - chg = t - f; - - /* Check for and consume any regions we now overlap with. */ - list_for_each_entry(rg, rg->link.prev, link) { - if (&rg->link == head) - break; - if (rg->from > t) - goto out; - - /* We overlap with this area, if it extends further than - * us then we must extend ourselves. Account for its - * existing reservation. */ - if (rg->to > t) { - chg += rg->to - t; - t = rg->to; - } - chg -= rg->to - rg->from; - } + chg = add_reservation_in_range(resv, f, t, true); -out: spin_unlock(&resv->lock); return chg; } From patchwork Sun Dec 1 01:57:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268439 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9B98917E0 for ; Sun, 1 Dec 2019 01:57:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 51435205ED for ; Sun, 1 Dec 2019 01:57:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="jF+pdDTo" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 51435205ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1FDE56B0364; Sat, 30 Nov 2019 20:57:05 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 189876B0367; Sat, 30 Nov 2019 20:57:05 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09DD96B0368; Sat, 30 Nov 2019 20:57:05 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0237.hostedemail.com [216.40.44.237]) by kanga.kvack.org (Postfix) with ESMTP id E00F16B0364 for ; Sat, 30 Nov 2019 20:57:04 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id A1AB6824999B for ; Sun, 1 Dec 2019 01:57:04 +0000 (UTC) X-FDA: 76214909568.23.brick71_3dbe4ba380b23 X-Spam-Summary: 2,0,0,4fe3bb39c4e657cf,d41d8cd98f00b204,akpm@linux-foundation.org,:aarcange@redhat.com:akpm@linux-foundation.org:cai@lca.pw:hughd@google.com::mike.kravetz@oracle.com:mm-commits@vger.kernel.org:richardw.yang@linux.intel.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1535:1544:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2564:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4117:4250:4321:4385:4605:5007:6119:6261:6653:7576:8599:9025:9391:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:13846:14181:14721:14849:21063:21080:21433:21451:21611:21627:21819:21939:30012:30054:30064,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutra l,Custom X-HE-Tag: brick71_3dbe4ba380b23 X-Filterd-Recvd-Size: 6102 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:04 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 499F2215F1; Sun, 1 Dec 2019 01:57:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165423; bh=7xJ6jlOKMcPtgM7pAkHN7Pum7TlqeF85BF77diWuUDI=; h=Date:From:To:Subject:From; b=jF+pdDToWuRJLllk4KzSUWPF4edmoCjJoF4i6nqmHd57kfCWmfEDK/e9TfwvfkU9d 1moaC4UiYNw8MmdfSK9DCefInQnDTN2u7UbgJX02g8/O9xtzomQHwg0SNHLlAQtPrM 5v26Gf52vbJZM9RXoeqhEXL28PpnIaPRD2kCSoNI= Date: Sat, 30 Nov 2019 17:57:02 -0800 From: akpm@linux-foundation.org To: aarcange@redhat.com, akpm@linux-foundation.org, cai@lca.pw, hughd@google.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, richardw.yang@linux.intel.com, torvalds@linux-foundation.org Subject: [patch 131/158] hugetlb: remove unused hstate in hugetlb_fault_mutex_hash() Message-ID: <20191201015702.Y00XKxInb%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wei Yang Subject: hugetlb: remove unused hstate in hugetlb_fault_mutex_hash() The first parameter hstate in function hugetlb_fault_mutex_hash() is not used anymore. This patch removes it. [akpm@linux-foundation.org: various build fixes] [cai@lca.pw: fix a GCC compilation warning] Link: http://lkml.kernel.org/r/1570544108-32331-1-git-send-email-cai@lca.pw Link: http://lkml.kernel.org/r/20191005003302.785-1-richardw.yang@linux.intel.com Signed-off-by: Wei Yang Signed-off-by: Qian Cai Suggested-by: Andrew Morton Reviewed-by: Andrew Morton Cc: Mike Kravetz Cc: Hugh Dickins Cc: Andrea Arcangeli Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 4 ++-- include/linux/hugetlb.h | 3 +-- mm/hugetlb.c | 10 ++++------ mm/userfaultfd.c | 5 +---- 4 files changed, 8 insertions(+), 14 deletions(-) --- a/fs/hugetlbfs/inode.c~hugetlb-remove-unused-hstate-in-hugetlb_fault_mutex_hash +++ a/fs/hugetlbfs/inode.c @@ -440,7 +440,7 @@ static void remove_inode_hugepages(struc u32 hash; index = page->index; - hash = hugetlb_fault_mutex_hash(h, mapping, index); + hash = hugetlb_fault_mutex_hash(mapping, index); mutex_lock(&hugetlb_fault_mutex_table[hash]); /* @@ -644,7 +644,7 @@ static long hugetlbfs_fallocate(struct f addr = index * hpage_size; /* mutex taken here, fault path and hole punch */ - hash = hugetlb_fault_mutex_hash(h, mapping, index); + hash = hugetlb_fault_mutex_hash(mapping, index); mutex_lock(&hugetlb_fault_mutex_table[hash]); /* See if already present in mapping to avoid alloc/free */ --- a/include/linux/hugetlb.h~hugetlb-remove-unused-hstate-in-hugetlb_fault_mutex_hash +++ a/include/linux/hugetlb.h @@ -105,8 +105,7 @@ void move_hugetlb_state(struct page *old void free_huge_page(struct page *page); void hugetlb_fix_reserve_counts(struct inode *inode); extern struct mutex *hugetlb_fault_mutex_table; -u32 hugetlb_fault_mutex_hash(struct hstate *h, struct address_space *mapping, - pgoff_t idx); +u32 hugetlb_fault_mutex_hash(struct address_space *mapping, pgoff_t idx); pte_t *huge_pmd_share(struct mm_struct *mm, unsigned long addr, pud_t *pud); --- a/mm/hugetlb.c~hugetlb-remove-unused-hstate-in-hugetlb_fault_mutex_hash +++ a/mm/hugetlb.c @@ -3796,7 +3796,7 @@ retry: * handling userfault. Reacquire after handling * fault to make calling code simpler. */ - hash = hugetlb_fault_mutex_hash(h, mapping, idx); + hash = hugetlb_fault_mutex_hash(mapping, idx); mutex_unlock(&hugetlb_fault_mutex_table[hash]); ret = handle_userfault(&vmf, VM_UFFD_MISSING); mutex_lock(&hugetlb_fault_mutex_table[hash]); @@ -3923,8 +3923,7 @@ backout_unlocked: } #ifdef CONFIG_SMP -u32 hugetlb_fault_mutex_hash(struct hstate *h, struct address_space *mapping, - pgoff_t idx) +u32 hugetlb_fault_mutex_hash(struct address_space *mapping, pgoff_t idx) { unsigned long key[2]; u32 hash; @@ -3941,8 +3940,7 @@ u32 hugetlb_fault_mutex_hash(struct hsta * For uniprocesor systems we always use a single mutex, so just * return 0 and avoid the hashing overhead. */ -u32 hugetlb_fault_mutex_hash(struct hstate *h, struct address_space *mapping, - pgoff_t idx) +u32 hugetlb_fault_mutex_hash(struct address_space *mapping, pgoff_t idx) { return 0; } @@ -3986,7 +3984,7 @@ vm_fault_t hugetlb_fault(struct mm_struc * get spurious allocation failures if two CPUs race to instantiate * the same page in the page cache. */ - hash = hugetlb_fault_mutex_hash(h, mapping, idx); + hash = hugetlb_fault_mutex_hash(mapping, idx); mutex_lock(&hugetlb_fault_mutex_table[hash]); entry = huge_ptep_get(ptep); --- a/mm/userfaultfd.c~hugetlb-remove-unused-hstate-in-hugetlb_fault_mutex_hash +++ a/mm/userfaultfd.c @@ -184,7 +184,6 @@ static __always_inline ssize_t __mcopy_a unsigned long src_addr, dst_addr; long copied; struct page *page; - struct hstate *h; unsigned long vma_hpagesize; pgoff_t idx; u32 hash; @@ -256,8 +255,6 @@ retry: goto out_unlock; } - h = hstate_vma(dst_vma); - while (src_addr < src_start + len) { pte_t dst_pteval; @@ -269,7 +266,7 @@ retry: */ idx = linear_page_index(dst_vma, dst_addr); mapping = dst_vma->vm_file->f_mapping; - hash = hugetlb_fault_mutex_hash(h, mapping, idx); + hash = hugetlb_fault_mutex_hash(mapping, idx); mutex_lock(&hugetlb_fault_mutex_table[hash]); err = -ENOMEM; From patchwork Sun Dec 1 01:57:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268441 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D1C9317E0 for ; Sun, 1 Dec 2019 01:57:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 95574205ED for ; Sun, 1 Dec 2019 01:57:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="HjD6xP8A" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 95574205ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 61DE46B0367; Sat, 30 Nov 2019 20:57:08 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5CE2A6B0369; Sat, 30 Nov 2019 20:57:08 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 50BDB6B036B; Sat, 30 Nov 2019 20:57:08 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0191.hostedemail.com [216.40.44.191]) by kanga.kvack.org (Postfix) with ESMTP id 385036B0367 for ; Sat, 30 Nov 2019 20:57:08 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id E3723180AD817 for ; Sun, 1 Dec 2019 01:57:07 +0000 (UTC) X-FDA: 76214909694.05.scene21_3e3abac5c0610 X-Spam-Summary: 10,1,0,eafa94c0bceb896c,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hzhongzhang@tencent.com:knightzhang@tencent.com::mike.kravetz@oracle.com:mm-commits@vger.kernel.org:tonnylu@tencent.com:torvalds@linux-foundation.org:willy@infradead.org,RULES_HIT:41:355:379:588:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2892:2895:2898:2902:2924:2926:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4605:5007:6261:6630:6653:7514:7576:9025:9121:9545:10004:10913:11026:11233:11658:11914:12043:12048:12295:12296:12297:12438:12517:12519:12555:12679:12783:12986:13225:13229:13255:13846:13870:14181:14721:14849:21080:21324:21433:21451:21627:21819:21939:30054:30056:30064:30070:30075,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,S PF:fp,MS X-HE-Tag: scene21_3e3abac5c0610 X-Filterd-Recvd-Size: 3532 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:07 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 82493215E5; Sun, 1 Dec 2019 01:57:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165426; bh=7+wvV3gZq5vFyvYUTM8y6xf9bmqVI1iWgdN4mIOvVH0=; h=Date:From:To:Subject:From; b=HjD6xP8AfTV8V/eyM/XH0vQ/MuZA/QH00B8yBnBMMowMsVtiAGmirJMQJ4zywcTmF CehhpEjTLVpdO3mfztEho7oXeD1HbzkQWEsZeA9ykxiX5G9atVHjEaFF7yiLpm0jSJ 7oshZL35rZtY+iMFeSgbLOBKQ73n/owYC1C1VGPQ= Date: Sat, 30 Nov 2019 17:57:06 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hzhongzhang@tencent.com, knightzhang@tencent.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, tonnylu@tencent.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 132/158] mm/hugetlb: avoid looping to the same hugepage if !pages and !vmas Message-ID: <20191201015706.RdZ5u8gLH%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zhigang Lu Subject: mm/hugetlb: avoid looping to the same hugepage if !pages and !vmas When mmapping an existing hugetlbfs file with MAP_POPULATE, we find it is very time consuming. For example, mmapping a 128GB file takes about 50 milliseconds. Sampling with perfevent shows it spends 99% time in the same_page loop in follow_hugetlb_page(). samples: 205 of event 'cycles', Event count (approx.): 136686374 - 99.04% test_mmap_huget [kernel.kallsyms] [k] follow_hugetlb_page follow_hugetlb_page __get_user_pages __mlock_vma_pages_range __mm_populate vm_mmap_pgoff sys_mmap_pgoff sys_mmap system_call_fastpath __mmap64 follow_hugetlb_page() is called with pages=NULL and vmas=NULL, so for each hugepage, we run into the same_page loop for pages_per_huge_page() times, but doing nothing. With this change, it takes less then 1 millisecond to mmap a 128GB file in hugetlbfs. Link: http://lkml.kernel.org/r/1567581712-5992-1-git-send-email-totty.lu@gmail.com Signed-off-by: Zhigang Lu Reviewed-by: Haozhong Zhang Reviewed-by: Zongming Zhang Reviewed-by: Mike Kravetz Acked-by: Matthew Wilcox Signed-off-by: Andrew Morton --- mm/hugetlb.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) --- a/mm/hugetlb.c~mm-hugetlb-avoid-looping-to-the-same-hugepage-if-pages-and-vmas +++ a/mm/hugetlb.c @@ -4338,6 +4338,21 @@ long follow_hugetlb_page(struct mm_struc break; } } + + /* + * If subpage information not requested, update counters + * and skip the same_page loop below. + */ + if (!pages && !vmas && !pfn_offset && + (vaddr + huge_page_size(h) < vma->vm_end) && + (remainder >= pages_per_huge_page(h))) { + vaddr += huge_page_size(h); + remainder -= pages_per_huge_page(h); + i += pages_per_huge_page(h); + spin_unlock(ptl); + continue; + } + same_page: if (pages) { pages[i] = mem_map_offset(page, pfn_offset); From patchwork Sun Dec 1 01:57:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268443 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 08E4717E0 for ; Sun, 1 Dec 2019 01:57:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CA66620873 for ; Sun, 1 Dec 2019 01:57:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="nui0ph9t" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CA66620873 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 950EB6B0369; Sat, 30 Nov 2019 20:57:11 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 901176B036C; Sat, 30 Nov 2019 20:57:11 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F1836B036D; Sat, 30 Nov 2019 20:57:11 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0238.hostedemail.com [216.40.44.238]) by kanga.kvack.org (Postfix) with ESMTP id 66B456B0369 for ; Sat, 30 Nov 2019 20:57:11 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 1DC582C0C for ; Sun, 1 Dec 2019 01:57:11 +0000 (UTC) X-FDA: 76214909862.13.meat95_3eb0cd983d53b X-Spam-Summary: 2,0,0,487f7f1749c6582d,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hughd@google.com:kirill.shutemov@linux.intel.com::mhocko@suse.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org:vbabka@suse.cz:zhongjiang@huawei.com,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1714:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3350:3867:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12114:12297:12438:12517:12519:12555:12679:12783:12986:13069:13255:13311:13357:13846:14181:14384:14721:14777:14849:21080:21433:21451:21627:21819:21939:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: meat95_3eb0cd983d53b X-Filterd-Recvd-Size: 2508 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:10 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B6BFF2084D; Sun, 1 Dec 2019 01:57:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165430; bh=7ncTcxlXfaOb7fvdSSpCu1rxXYjT0Tbd0bQ5hgbkeXU=; h=Date:From:To:Subject:From; b=nui0ph9txSWjd5Ox94gFeCzQnKGp1xNaVg4S72tXSgjGH8czOldYeBRpiZIylLi3h okEs/G5YgfYHHqollR4zi/TqFYy7O+dfHM6ZQsKg50sVS3EFxEvBoMYhF2j9bkFCIK 6HefAxtBNpvNf1Whn6u+T5z+8Awh73wEDtfiwji0= Date: Sat, 30 Nov 2019 17:57:09 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hughd@google.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, zhongjiang@huawei.com Subject: [patch 133/158] mm/huge_memory.c: split_huge_pages_fops should be defined with DEFINE_DEBUGFS_ATTRIBUTE Message-ID: <20191201015709.OXeUBhujO%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: zhong jiang Subject: mm/huge_memory.c: split_huge_pages_fops should be defined with DEFINE_DEBUGFS_ATTRIBUTE split_huge_pages_fops is used for debugfs file. hence, it is more clear to use DEFINE_DEBUGFS_ATTRIBUTE. Link: http://lkml.kernel.org/r/1572347674-8111-1-git-send-email-zhongjiang@huawei.com Signed-off-by: zhong jiang Reviewed-by: Andrew Morton Cc: Kirill A. Shutemov Cc: Michal Hocko Cc: Vlastimil Babka Cc: Hugh Dickins Signed-off-by: Andrew Morton --- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/huge_memory.c~mm-split_huge_pages_fops-should-be-defined-with-define_debugfs_attribute +++ a/mm/huge_memory.c @@ -3003,7 +3003,7 @@ next: return 0; } -DEFINE_SIMPLE_ATTRIBUTE(split_huge_pages_fops, NULL, split_huge_pages_set, +DEFINE_DEBUGFS_ATTRIBUTE(split_huge_pages_fops, NULL, split_huge_pages_set, "%llu\n"); static int __init split_huge_pages_debugfs(void) From patchwork Sun Dec 1 01:57:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268445 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E0E6217F0 for ; Sun, 1 Dec 2019 01:57:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AE21720873 for ; Sun, 1 Dec 2019 01:57:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="jOOcWklY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AE21720873 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9EDB56B036C; Sat, 30 Nov 2019 20:57:14 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 99DC16B036E; Sat, 30 Nov 2019 20:57:14 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 88DD46B036F; Sat, 30 Nov 2019 20:57:14 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0248.hostedemail.com [216.40.44.248]) by kanga.kvack.org (Postfix) with ESMTP id 6A93F6B036C for ; Sat, 30 Nov 2019 20:57:14 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 190D82C0C for ; Sun, 1 Dec 2019 01:57:14 +0000 (UTC) X-FDA: 76214909988.18.verse70_3f21308c3d228 X-Spam-Summary: 2,0,0,c215b60682a148e4,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org::mgorman@techsingularity.net:mhocko@suse.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org:vbabka@suse.cz:yang.shi@linux.alibaba.com,RULES_HIT:41:355:379:800:960:965:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4390:4395:5007:6261:6653:7576:7903:8957:9025:9121:9545:9592:10004:10913:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:14096:14181:14721:14849:21060:21080:21433:21451:21627:21819:21939:30012:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUM MARY:non X-HE-Tag: verse70_3f21308c3d228 X-Filterd-Recvd-Size: 3403 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:13 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D54D0217AB; Sun, 1 Dec 2019 01:57:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165433; bh=/2rtnSvaqSIdsqgtXHziAl7RiYz6VlgkUwR9UcaBHYE=; h=Date:From:To:Subject:From; b=jOOcWklY9gIsFgepSNSYsQ8HUZrOpl0+ETnaXrzCH20ajpSA916iYcXWMwXbf85VX Bfd0AdeyqmGgCmil7KQ+XWKEZh7r/vJdRNVWamztnrT9ySOjIcxXcVzrLlb75PGI8h ERVlGpZRqEWqwaWsOHO6rUMuAqgJdB6CGugOPgNg= Date: Sat, 30 Nov 2019 17:57:12 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@suse.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, yang.shi@linux.alibaba.com Subject: [patch 134/158] mm/migrate.c: handle freed page at the first place Message-ID: <20191201015712.oGwSkH_BM%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Shi Subject: mm/migrate.c: handle freed page at the first place When doing migration if the freed page is met, we just return without migrating it since it is pointless to migrate a freed page. But, the current code allocates target page unconditionally before handling freed page, if the page is freed, the newly allocated will be just freed. It doesn't make too much sense and is just a waste of time although migrating freed page is rare. So, handle freed page at the before that to avoid unnecessary page allocation and free. Link: http://lkml.kernel.org/r/1573755869-106954-1-git-send-email-yang.shi@linux.alibaba.com Signed-off-by: Yang Shi Acked-by: Michal Hocko Reviewed-by: Andrew Morton Cc: Mel Gorman Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- mm/migrate.c | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) --- a/mm/migrate.c~mm-migrate-handle-freed-page-at-the-first-place +++ a/mm/migrate.c @@ -1168,15 +1168,11 @@ static ICE_noinline int unmap_and_move(n enum migrate_reason reason) { int rc = MIGRATEPAGE_SUCCESS; - struct page *newpage; + struct page *newpage = NULL; if (!thp_migration_supported() && PageTransHuge(page)) return -ENOMEM; - newpage = get_new_page(page, private); - if (!newpage) - return -ENOMEM; - if (page_count(page) == 1) { /* page was freed from under us. So we are done. */ ClearPageActive(page); @@ -1187,13 +1183,13 @@ static ICE_noinline int unmap_and_move(n __ClearPageIsolated(page); unlock_page(page); } - if (put_new_page) - put_new_page(newpage, private); - else - put_page(newpage); goto out; } + newpage = get_new_page(page, private); + if (!newpage) + return -ENOMEM; + rc = __unmap_and_move(page, newpage, force, mode); if (rc == MIGRATEPAGE_SUCCESS) set_page_owner_migrate_reason(newpage, reason); From patchwork Sun Dec 1 01:57:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268447 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6947A195A for ; Sun, 1 Dec 2019 01:57:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 36DEF205ED for ; Sun, 1 Dec 2019 01:57:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="O6MRrFdF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 36DEF205ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1FD276B036E; Sat, 30 Nov 2019 20:57:18 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1D5BC6B0370; Sat, 30 Nov 2019 20:57:18 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09D5B6B0371; Sat, 30 Nov 2019 20:57:18 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0054.hostedemail.com [216.40.44.54]) by kanga.kvack.org (Postfix) with ESMTP id E46706B036E for ; Sat, 30 Nov 2019 20:57:17 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 998EA180AD817 for ; Sun, 1 Dec 2019 01:57:17 +0000 (UTC) X-FDA: 76214910114.25.time65_3fa6390f1731b X-Spam-Summary: 2,0,0,17124e4a2afd7e2d,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:kirill.shutemov@linux.intel.com:kirill@shutemov.name::mhocko@kernel.org:mm-commits@vger.kernel.org:torvalds@linux-foundation.org:yang.shi@linux.alibaba.com,RULES_HIT:41:355:379:800:960:966:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:5007:6261:6653:7576:7903:8599:8957:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12783:12986:13069:13161:13229:13311:13357:14181:14384:14721:14849:21060:21080:21451:21627:21939:30054:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:4,LUA_SUMMARY:none X-HE-Tag: time65_3fa6390f1731b X-Filterd-Recvd-Size: 3029 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:17 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E5FEA20873; Sun, 1 Dec 2019 01:57:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165436; bh=GkZ0k2Q12lDqrJYbxGBpJ394Oqy+QeZOEOYWxn8aoMo=; h=Date:From:To:Subject:From; b=O6MRrFdF6R+06GtgZ+oD2pIVxgpPNHTUXQoEw4xzeNa3WejHcboGZEtomHEHic9RU qiUxDZXaEgx+GHkVoOxQcJNWp4K4ZJ5JI9XgFXlypgslPpgX9KxDq2rEMb8f/VSG1X JOYo+ZV3W50wX1WTQPyuJQfAiN4fH5uP1wlS/WCU= Date: Sat, 30 Nov 2019 17:57:15 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, kirill.shutemov@linux.intel.com, kirill@shutemov.name, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, yang.shi@linux.alibaba.com Subject: [patch 135/158] mm, thp: do not queue fully unmapped pages for deferred split Message-ID: <20191201015715.7Go-jCWUJ%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Kirill A. Shutemov" Subject: mm, thp: do not queue fully unmapped pages for deferred split Adding fully unmapped pages into deferred split queue is not productive: these pages are about to be freed or they are pinned and cannot be split anyway. Link: http://lkml.kernel.org/r/20190913091849.11151-1-kirill.shutemov@linux.intel.com Signed-off-by: Kirill A. Shutemov Reviewed-by: Yang Shi Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/rmap.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) --- a/mm/rmap.c~mm-thp-do-not-queue-fully-unmapped-pages-for-deferred-split +++ a/mm/rmap.c @@ -1292,12 +1292,20 @@ static void page_remove_anon_compound_rm if (TestClearPageDoubleMap(page)) { /* * Subpages can be mapped with PTEs too. Check how many of - * themi are still mapped. + * them are still mapped. */ for (i = 0, nr = 0; i < HPAGE_PMD_NR; i++) { if (atomic_add_negative(-1, &page[i]._mapcount)) nr++; } + + /* + * Queue the page for deferred split if at least one small + * page of the compound page is unmapped, but at least one + * small page is still mapped. + */ + if (nr && nr < HPAGE_PMD_NR) + deferred_split_huge_page(page); } else { nr = HPAGE_PMD_NR; } @@ -1305,10 +1313,8 @@ static void page_remove_anon_compound_rm if (unlikely(PageMlocked(page))) clear_page_mlock(page); - if (nr) { + if (nr) __mod_node_page_state(page_pgdat(page), NR_ANON_MAPPED, -nr); - deferred_split_huge_page(page); - } } /** From patchwork Sun Dec 1 01:57:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268449 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A905317F0 for ; Sun, 1 Dec 2019 01:57:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 765702082E for ; Sun, 1 Dec 2019 01:57:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="X48K46Hz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 765702082E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 61CD06B0370; Sat, 30 Nov 2019 20:57:21 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5F3786B0373; Sat, 30 Nov 2019 20:57:21 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 530C26B0375; Sat, 30 Nov 2019 20:57:21 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0137.hostedemail.com [216.40.44.137]) by kanga.kvack.org (Postfix) with ESMTP id 3998C6B0370 for ; Sat, 30 Nov 2019 20:57:21 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id E6AB5181AEF0B for ; Sun, 1 Dec 2019 01:57:20 +0000 (UTC) X-FDA: 76214910240.17.edge78_401df609de730 X-Spam-Summary: 2,0,0,017ec6ca98c0ede4,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hannes@cmpxchg.org:hughd@google.com:kirill.shutemov@linux.intel.com::mm-commits@vger.kernel.org:songliubraving@fb.com:torvalds@linux-foundation.org:william.kucharski@oracle.com:willy@infradead.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:8599:8660:8957:9025:9545:10004:10913:11026:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12783:12986:13069:13148:13184:13229:13230:13255:13311:13357:14096:14181:14384:14721:14849:21080:21433:21451:21611:21627:21740:21939:30054:30064,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL: neutral, X-HE-Tag: edge78_401df609de730 X-Filterd-Recvd-Size: 3175 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:20 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 89387205ED; Sun, 1 Dec 2019 01:57:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165439; bh=BDLE4/Rlprx1ErkpuDLXCnGyWIoYaXR+PrjZR0eY0n4=; h=Date:From:To:Subject:From; b=X48K46HzBfjDWbDa+8CqNFKGPSrqioRwPkHpUjOjA83fQRyAYzWzZ+4N57JIqEEnp gpy2+bRkoMt8TPwgRQznmvRJagi+2e4IeVL7nY3Gf0clygVsrOmcigI7EYLykZx7mj wRESj7boKrZK7wwrWICgOc7Q+Bryx5eYll0iHMxM= Date: Sat, 30 Nov 2019 17:57:19 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hannes@cmpxchg.org, hughd@google.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, songliubraving@fb.com, torvalds@linux-foundation.org, william.kucharski@oracle.com, willy@infradead.org Subject: [patch 136/158] mm/thp: flush file for !is_shmem PageDirty() case in collapse_file() Message-ID: <20191201015719.31Qrkfi64%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Song Liu Subject: mm/thp: flush file for !is_shmem PageDirty() case in collapse_file() For non-shmem file THPs, khugepaged only collapses read only .text mapping (VM_DENYWRITE). These pages should not be dirty except the case where the file hasn't been flushed since first write. Call filemap_flush() in collapse_file() to accelerate the write back in such cases. Link: http://lkml.kernel.org/r/20191106060930.2571389-3-songliubraving@fb.com Signed-off-by: Song Liu Cc: Kirill A. Shutemov Cc: Hugh Dickins Cc: William Kucharski Cc: Johannes Weiner Cc: Matthew Wilcox Signed-off-by: Andrew Morton --- mm/khugepaged.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) --- a/mm/khugepaged.c~mm-thp-flush-file-for-is_shmem-pagedirty-case-in-collapse_file +++ a/mm/khugepaged.c @@ -1602,6 +1602,24 @@ static void collapse_file(struct mm_stru result = SCAN_FAIL; goto xa_unlocked; } + } else if (PageDirty(page)) { + /* + * khugepaged only works on read-only fd, + * so this page is dirty because it hasn't + * been flushed since first write. There + * won't be new dirty pages. + * + * Trigger async flush here and hope the + * writeback is done when khugepaged + * revisits this page. + * + * This is a one-off situation. We are not + * forcing writeback in loop. + */ + xas_unlock_irq(&xas); + filemap_flush(mapping); + result = SCAN_FAIL; + goto xa_unlocked; } else if (trylock_page(page)) { get_page(page); xas_unlock_irq(&xas); From patchwork Sun Dec 1 01:57:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268451 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CC71B17E0 for ; Sun, 1 Dec 2019 01:57:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 99A5A205ED for ; Sun, 1 Dec 2019 01:57:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="Y21dyyiB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 99A5A205ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 99EA56B0373; Sat, 30 Nov 2019 20:57:24 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 94F1C6B0376; Sat, 30 Nov 2019 20:57:24 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83EA56B0377; Sat, 30 Nov 2019 20:57:24 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0200.hostedemail.com [216.40.44.200]) by kanga.kvack.org (Postfix) with ESMTP id 6A4FD6B0373 for ; Sat, 30 Nov 2019 20:57:24 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 1EE604403 for ; Sun, 1 Dec 2019 01:57:24 +0000 (UTC) X-FDA: 76214910408.25.box31_4098480a8a944 X-Spam-Summary: 2,0,0,f3b257210b197ac9,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:andreyknvl@google.com:aryabinin@virtuozzo.com:huyue2@yulong.com::mm-commits@vger.kernel.org:opendmb@gmail.com:peng.fan@nxp.com:rppt@linux.ibm.com:ryh.szk.cmnty@gmail.com:tglx@linutronix.de:torvalds@linux-foundation.org:yeyunfeng@huawei.com,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3867:3870:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:5007:6119:6261:6653:6737:7514:7576:7903:9025:9036:9545:9592:10004:11026:11257:11473:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12783:12986:13069:13311:13357:13846:14181:14384:14721:14849:21080:21451:21627:21939:30029:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF :not bul X-HE-Tag: box31_4098480a8a944 X-Filterd-Recvd-Size: 3128 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:23 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id BE5702082E; Sun, 1 Dec 2019 01:57:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165443; bh=+tg2i4I7sKkCCl2JudpXly90jX+u0Q9Ju38U1lCdQyE=; h=Date:From:To:Subject:From; b=Y21dyyiBO+lkYbEZjTCWEQEs8p/iuXAjxu3Vud8ovylZdCDorjhfZmzTGJ1xi6pFW 2hZgZ+qITTz9Bwt133BzefzempSjkB9Cpv50aryDk3uQ5Kxg06cw97zPISpnJcnevJ 2/6FFRPjJTlXa8S60VB/wwIAwld7O/WkCH0oauTY= Date: Sat, 30 Nov 2019 17:57:22 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, huyue2@yulong.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, opendmb@gmail.com, peng.fan@nxp.com, rppt@linux.ibm.com, ryh.szk.cmnty@gmail.com, tglx@linutronix.de, torvalds@linux-foundation.org, yeyunfeng@huawei.com Subject: [patch 137/158] mm/cma.c: switch to bitmap_zalloc() for cma bitmap allocation Message-ID: <20191201015722.ZKSvOZdeb%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yunfeng Ye Subject: mm/cma.c: switch to bitmap_zalloc() for cma bitmap allocation kzalloc() is used for cma bitmap allocation in cma_activate_area(), switch to bitmap_zalloc() for clarity. Link: http://lkml.kernel.org/r/895d4627-f115-c77a-d454-c0a196116426@huawei.com Signed-off-by: Yunfeng Ye Reviewed-by: Andrew Morton Cc: Mike Rapoport Cc: Yue Hu Cc: Peng Fan Cc: Andrey Ryabinin Cc: Ryohei Suzuki Cc: Andrey Konovalov Cc: Doug Berger Cc: Thomas Gleixner Signed-off-by: Andrew Morton --- mm/cma.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) --- a/mm/cma.c~mm-cmac-switch-to-bitmap_zalloc-for-cma-bitmap-allocation +++ a/mm/cma.c @@ -95,13 +95,11 @@ static void cma_clear_bitmap(struct cma static int __init cma_activate_area(struct cma *cma) { - int bitmap_size = BITS_TO_LONGS(cma_bitmap_maxno(cma)) * sizeof(long); unsigned long base_pfn = cma->base_pfn, pfn = base_pfn; unsigned i = cma->count >> pageblock_order; struct zone *zone; - cma->bitmap = kzalloc(bitmap_size, GFP_KERNEL); - + cma->bitmap = bitmap_zalloc(cma_bitmap_maxno(cma), GFP_KERNEL); if (!cma->bitmap) { cma->count = 0; return -ENOMEM; @@ -139,7 +137,7 @@ static int __init cma_activate_area(stru not_in_zone: pr_err("CMA area %s could not be activated\n", cma->name); - kfree(cma->bitmap); + bitmap_free(cma->bitmap); cma->count = 0; return -EINVAL; } From patchwork Sun Dec 1 01:57:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268453 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2EDDF17E0 for ; Sun, 1 Dec 2019 01:57:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F080A205ED for ; Sun, 1 Dec 2019 01:57:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="H4F/vxHg" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F080A205ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E5A636B0376; Sat, 30 Nov 2019 20:57:27 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DE3116B0378; Sat, 30 Nov 2019 20:57:27 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CF9EB6B0379; Sat, 30 Nov 2019 20:57:27 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0040.hostedemail.com [216.40.44.40]) by kanga.kvack.org (Postfix) with ESMTP id B6A2D6B0376 for ; Sat, 30 Nov 2019 20:57:27 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 780CF824999B for ; Sun, 1 Dec 2019 01:57:27 +0000 (UTC) X-FDA: 76214910534.17.able99_4111eec80a124 X-Spam-Summary: 2,0,0,0a220b439ff0b9b0,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:gregkh@linuxfoundation.org:huyue2@yulong.com::mm-commits@vger.kernel.org:torvalds@linux-foundation.org:zhongjiang@huawei.com,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3867:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:5007:6261:6653:7576:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:12555:12679:12783:12986:13255:13846:14181:14721:14849:21080:21220:21433:21451:21627:21819:21939:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: able99_4111eec80a124 X-Filterd-Recvd-Size: 3641 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:27 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 023E920873; Sun, 1 Dec 2019 01:57:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165446; bh=kIkuU3mCd5IDI/yDSAIuEnNBzb9oL1jchP2is6d8yLk=; h=Date:From:To:Subject:From; b=H4F/vxHgqjHrhJYs56+bSwxyDkHSIIpLAbCYEVBvrKoXoruopiZGdz6xw/YzRkRjU PmN3/2tvVV5DT5bPRec7QrMFsbUJTam10guMmNRwyUm6QssGyrKM5Kcb2KwOPvAvw5 5WCGUTjl2p9+vrDhbL6ALxs9X0XPdiiNdM61m2ac= Date: Sat, 30 Nov 2019 17:57:25 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, gregkh@linuxfoundation.org, huyue2@yulong.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, zhongjiang@huawei.com Subject: [patch 138/158] mm/cma_debug.c: use DEFINE_DEBUGFS_ATTRIBUTE to define debugfs fops Message-ID: <20191201015725.vA60n4PQg%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: zhong jiang Subject: mm/cma_debug.c: use DEFINE_DEBUGFS_ATTRIBUTE to define debugfs fops It is more clear to use DEFINE_DEBUGFS_ATTRIBUTE to define debugfs file operation rather than DEFINE_SIMPLE_ATTRIBUTE. Link: http://lkml.kernel.org/r/1572348687-9951-1-git-send-email-zhongjiang@huawei.com Signed-off-by: zhong jiang Reviewed-by: Andrew Morton Cc: Yue Hu Cc: Greg Kroah-Hartman Signed-off-by: Andrew Morton --- mm/cma_debug.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) --- a/mm/cma_debug.c~mm-cma_debug-use-define_debugfs_attribute-to-define-debugfs-fops +++ a/mm/cma_debug.c @@ -29,7 +29,7 @@ static int cma_debugfs_get(void *data, u return 0; } -DEFINE_SIMPLE_ATTRIBUTE(cma_debugfs_fops, cma_debugfs_get, NULL, "%llu\n"); +DEFINE_DEBUGFS_ATTRIBUTE(cma_debugfs_fops, cma_debugfs_get, NULL, "%llu\n"); static int cma_used_get(void *data, u64 *val) { @@ -44,7 +44,7 @@ static int cma_used_get(void *data, u64 return 0; } -DEFINE_SIMPLE_ATTRIBUTE(cma_used_fops, cma_used_get, NULL, "%llu\n"); +DEFINE_DEBUGFS_ATTRIBUTE(cma_used_fops, cma_used_get, NULL, "%llu\n"); static int cma_maxchunk_get(void *data, u64 *val) { @@ -66,7 +66,7 @@ static int cma_maxchunk_get(void *data, return 0; } -DEFINE_SIMPLE_ATTRIBUTE(cma_maxchunk_fops, cma_maxchunk_get, NULL, "%llu\n"); +DEFINE_DEBUGFS_ATTRIBUTE(cma_maxchunk_fops, cma_maxchunk_get, NULL, "%llu\n"); static void cma_add_to_cma_mem_list(struct cma *cma, struct cma_mem *mem) { @@ -126,7 +126,7 @@ static int cma_free_write(void *data, u6 return cma_free_mem(cma, pages); } -DEFINE_SIMPLE_ATTRIBUTE(cma_free_fops, NULL, cma_free_write, "%llu\n"); +DEFINE_DEBUGFS_ATTRIBUTE(cma_free_fops, NULL, cma_free_write, "%llu\n"); static int cma_alloc_mem(struct cma *cma, int count) { @@ -158,7 +158,7 @@ static int cma_alloc_write(void *data, u return cma_alloc_mem(cma, pages); } -DEFINE_SIMPLE_ATTRIBUTE(cma_alloc_fops, NULL, cma_alloc_write, "%llu\n"); +DEFINE_DEBUGFS_ATTRIBUTE(cma_alloc_fops, NULL, cma_alloc_write, "%llu\n"); static void cma_debugfs_add_one(struct cma *cma, struct dentry *root_dentry) { From patchwork Sun Dec 1 01:57:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268455 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AE27117F0 for ; Sun, 1 Dec 2019 01:57:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 71CA12082E for ; Sun, 1 Dec 2019 01:57:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="toYOZA5C" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 71CA12082E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 691066B0378; Sat, 30 Nov 2019 20:57:31 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6427C6B037A; Sat, 30 Nov 2019 20:57:31 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 558326B037B; Sat, 30 Nov 2019 20:57:31 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0051.hostedemail.com [216.40.44.51]) by kanga.kvack.org (Postfix) with ESMTP id 3D9666B0378 for ; Sat, 30 Nov 2019 20:57:31 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id D9C52180AD817 for ; Sun, 1 Dec 2019 01:57:30 +0000 (UTC) X-FDA: 76214910660.24.order40_419287a2d5607 X-Spam-Summary: 2,0,0,5134204738c710ef,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:dan.j.williams@intel.com:dave.hansen@linux.intel.com:fengguang.wu@intel.com::mgorman@suse.de:mhocko@suse.com:mingo@kernel.org:mm-commits@vger.kernel.org:peterz@infradead.org:riel@redhat.com:torvalds@linux-foundation.org:ying.huang@intel.com,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2196:2199:2393:2525:2553:2559:2563:2682:2685:2859:2897:2898:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4605:5007:6119:6261:6653:6737:7576:8599:8603:8660:9025:9121:9545:10004:10913:11026:11233:11658:11914:12043:12048:12114:12294:12295:12296:12297:12438:12517:12519:12555:12679:12783:12986:13148:13161:13229:13230:13846:14181:14721:14849:21063:21080:21451:21524:21627:21939:30034:30054: 30064:30 X-HE-Tag: order40_419287a2d5607 X-Filterd-Recvd-Size: 3914 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:30 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 67B47205ED; Sun, 1 Dec 2019 01:57:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165449; bh=idAMszw1QipJjTueGsRHrgbfWvc2GZFiUMdabpZkV14=; h=Date:From:To:Subject:From; b=toYOZA5ChrYPH/VbUIDTOQ0h/flPrgj7eCGFDaRbHe1vwguyopj+Ye/YsXcemNj7m UwxBeGrD5qyTYzfd7pwqXJtOmmKG53QpBby5Nk+uYEwI6IB8xsNjT2MVuFntmN21xz IeN0EH/lJ3sWGONR/zdwlQp3N0I0j9m0u2hlQN7o= Date: Sat, 30 Nov 2019 17:57:28 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, dan.j.williams@intel.com, dave.hansen@linux.intel.com, fengguang.wu@intel.com, linux-mm@kvack.org, mgorman@suse.de, mhocko@suse.com, mingo@kernel.org, mm-commits@vger.kernel.org, peterz@infradead.org, riel@redhat.com, torvalds@linux-foundation.org, ying.huang@intel.com Subject: [patch 139/158] autonuma: fix watermark checking in migrate_balanced_pgdat() Message-ID: <20191201015728.Z9V4wI2df%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Huang Ying Subject: autonuma: fix watermark checking in migrate_balanced_pgdat() When zone_watermark_ok() is called in migrate_balanced_pgdat() to check migration target node, the parameter classzone_idx (for requested zone) is specified as 0 (ZONE_DMA). But when allocating memory for autonuma in alloc_misplaced_dst_page(), the requested zone from GFP flags is ZONE_MOVABLE. That is, the requested zone is different. The size of lowmem_reserve for the different requested zone is different. And this may cause some issues. For example, in the zoneinfo of a test machine as below, Node 0, zone DMA32 pages free 61592 min 29 low 454 high 879 spanned 1044480 present 442306 managed 425921 protection: (0, 0, 62457, 62457, 62457) The free page number of ZONE_DMA32 is greater than "high watermark + lowmem_reserve[ZONE_DMA]", but less than "high watermark + lowmem_reserve[ZONE_MOVABLE]". And because __alloc_pages_node() in alloc_misplaced_dst_page() requests ZONE_MOVABLE, the zone_watermark_ok() on ZONE_DMA32 in migrate_balanced_pgdat() may always return true. So, autonuma may not stop even when memory pressure in node 0 is heavy. To fix the issue, ZONE_MOVABLE is used as parameter to call zone_watermark_ok() in migrate_balanced_pgdat(). This makes it same as requested zone in alloc_misplaced_dst_page(). So that migrate_balanced_pgdat() returns false when memory pressure is heavy. Link: http://lkml.kernel.org/r/20191101075727.26683-2-ying.huang@intel.com Signed-off-by: "Huang, Ying" Acked-by: Mel Gorman Cc: Michal Hocko Cc: Rik van Riel Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Dave Hansen Cc: Dan Williams Cc: Fengguang Wu Signed-off-by: Andrew Morton --- mm/migrate.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/migrate.c~autonuma-fix-watermark-checking-in-migrate_balanced_pgdat +++ a/mm/migrate.c @@ -1859,7 +1859,7 @@ static bool migrate_balanced_pgdat(struc if (!zone_watermark_ok(zone, 0, high_wmark_pages(zone) + nr_migrate_pages, - 0, 0)) + ZONE_MOVABLE, 0)) continue; return true; } From patchwork Sun Dec 1 01:57:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268457 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 032F817F0 for ; Sun, 1 Dec 2019 01:57:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C4BF620873 for ; Sun, 1 Dec 2019 01:57:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="D5wkfYio" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C4BF620873 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BE8EB6B037A; Sat, 30 Nov 2019 20:57:34 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B992B6B037C; Sat, 30 Nov 2019 20:57:34 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A61976B037D; Sat, 30 Nov 2019 20:57:34 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0064.hostedemail.com [216.40.44.64]) by kanga.kvack.org (Postfix) with ESMTP id 8E5016B037A for ; Sat, 30 Nov 2019 20:57:34 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 43FC2824999B for ; Sun, 1 Dec 2019 01:57:34 +0000 (UTC) X-FDA: 76214910828.21.glue13_42106ce10ea37 X-Spam-Summary: 2,0,0,b935c683d633a9b4,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:dan.j.williams@intel.com:dave.hansen@linux.intel.com:fengguang.wu@intel.com::mgorman@suse.de:mhocko@suse.com:mingo@kernel.org:mm-commits@vger.kernel.org:peterz@infradead.org:riel@redhat.com:torvalds@linux-foundation.org:ying.huang@intel.com,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2898:2901:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3834:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4560:5007:6119:6261:6653:6737:7576:8599:8957:9025:9545:9592:10004:10913:11026:11658:11914:12043:12048:12297:12517:12519:12555:12679:12783:12986:13161:13229:13846:14181:14721:14849:21080:21433:21451:21627:21939:30003:30045:30054:30055:30064:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck :none,Do X-HE-Tag: glue13_42106ce10ea37 X-Filterd-Recvd-Size: 3606 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:33 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id CF13F2082E; Sun, 1 Dec 2019 01:57:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165453; bh=3psEDXDdfYuBV/kjrYa967oPAqNh0lcS+E4+zQo8hos=; h=Date:From:To:Subject:From; b=D5wkfYioaze+lr5DOEGzZDX4zppA+v4BYnHSgsEOctq3TLSqno8QFin06LtqJvNvy bxCu70eCpWVvShuhuMiVHmBmBSbzau1GsSrNcTcs+X6IHraq13ub587iHnWUJDsYwD tOwvw+1M/b9BBpj4mQwzsNbX9wVMekj2P13HQ1yM= Date: Sat, 30 Nov 2019 17:57:32 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, dan.j.williams@intel.com, dave.hansen@linux.intel.com, fengguang.wu@intel.com, linux-mm@kvack.org, mgorman@suse.de, mhocko@suse.com, mingo@kernel.org, mm-commits@vger.kernel.org, peterz@infradead.org, riel@redhat.com, torvalds@linux-foundation.org, ying.huang@intel.com Subject: [patch 140/158] autonuma: reduce cache footprint when scanning page tables Message-ID: <20191201015732.Ecjld8a-U%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Huang Ying Subject: autonuma: reduce cache footprint when scanning page tables In auto NUMA balancing page table scanning, if the pte_protnone() is true, the PTE needs not to be changed because it's in target state already. So other checking on corresponding struct page is unnecessary too. So, if we check pte_protnone() firstly for each PTE, we can avoid unnecessary struct page accessing, so that reduce the cache footprint of NUMA balancing page table scanning. In the performance test of pmbench memory accessing benchmark with 80:20 read/write ratio and normal access address distribution on a 2 socket Intel server with Optance DC Persistent Memory, perf profiling shows that the autonuma page table scanning time reduces from 1.23% to 0.97% (that is, reduced 21%) with the patch. Link: http://lkml.kernel.org/r/20191101075727.26683-3-ying.huang@intel.com Signed-off-by: "Huang, Ying" Acked-by: Mel Gorman Cc: Michal Hocko Cc: Rik van Riel Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Dave Hansen Cc: Dan Williams Cc: Fengguang Wu Signed-off-by: Andrew Morton --- mm/mprotect.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) --- a/mm/mprotect.c~autonuma-reduce-cache-footprint-when-scanning-page-tables +++ a/mm/mprotect.c @@ -80,6 +80,10 @@ static unsigned long change_pte_range(st if (prot_numa) { struct page *page; + /* Avoid TLB flush if possible */ + if (pte_protnone(oldpte)) + continue; + page = vm_normal_page(vma, addr, oldpte); if (!page || PageKsm(page)) continue; @@ -97,10 +101,6 @@ static unsigned long change_pte_range(st if (page_is_file_cache(page) && PageDirty(page)) continue; - /* Avoid TLB flush if possible */ - if (pte_protnone(oldpte)) - continue; - /* * Don't mess with PTEs if page is already on the node * a single-threaded process is running on. From patchwork Sun Dec 1 01:57:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268459 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4222417E0 for ; Sun, 1 Dec 2019 01:57:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 05D3A222D9 for ; Sun, 1 Dec 2019 01:57:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="C/btsZTw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 05D3A222D9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 029386B037C; Sat, 30 Nov 2019 20:57:38 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F1C036B037F; Sat, 30 Nov 2019 20:57:37 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E59D06B0380; Sat, 30 Nov 2019 20:57:37 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0021.hostedemail.com [216.40.44.21]) by kanga.kvack.org (Postfix) with ESMTP id CBD556B037C for ; Sat, 30 Nov 2019 20:57:37 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 94E7B181AEF0B for ; Sun, 1 Dec 2019 01:57:37 +0000 (UTC) X-FDA: 76214910954.28.pot71_428edfa7cdb19 X-Spam-Summary: 2,0,0,834ab5a0968c3900,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org::mm-commits@vger.kernel.org:n-horiguchi@ah.jp.nec.com:torvalds@linux-foundation.org:zhongjiang@huawei.com,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1714:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3350:3867:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:9025:9391:9545:10004:10913:11026:11473:11658:11914:12043:12048:12114:12297:12438:12517:12519:12555:12679:12783:12986:13069:13255:13311:13357:13846:14181:14384:14721:14777:14849:21060:21080:21433:21451:21627:21819:21939:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: pot71_428edfa7cdb19 X-Filterd-Recvd-Size: 2451 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:37 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 6EBF320873; Sun, 1 Dec 2019 01:57:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165456; bh=6c1N0s91gUhGLw6YMHitqdKLECJHoerq6PY0MuVRd8A=; h=Date:From:To:Subject:From; b=C/btsZTwYx51hLw5eqkyDX4We4ngrRYcPhxaTwRsep/pe9LTdM5jMLNGYNmXCl6ea fmF9MvT2SUwJRgn+irs7qaMoRCehz4ujrbomlsbC9niFIs1bd3OgcO1/Cpd9NCGG7T naZKEFaMklVLof7YFg3Vkv4AgApoqN7L8ePya75M= Date: Sat, 30 Nov 2019 17:57:35 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, n-horiguchi@ah.jp.nec.com, torvalds@linux-foundation.org, zhongjiang@huawei.com Subject: [patch 141/158] mm/hwpoison-inject: use DEFINE_DEBUGFS_ATTRIBUTE to define debugfs fops Message-ID: <20191201015735.f9AAXx8Ed%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: zhong jiang Subject: mm/hwpoison-inject: use DEFINE_DEBUGFS_ATTRIBUTE to define debugfs fops It is more clear to use DEFINE_DEBUGFS_ATTRIBUTE to define debugfs file operation rather than DEFINE_SIMPLE_ATTRIBUTE. Link: http://lkml.kernel.org/r/1572403660-44718-1-git-send-email-zhongjiang@huawei.com Signed-off-by: zhong jiang Acked-by: Naoya Horiguchi Signed-off-by: Andrew Morton --- mm/hwpoison-inject.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/mm/hwpoison-inject.c~mm-hwpoison-inject-use-define_debugfs_attribute-to-define-debugfs-fops +++ a/mm/hwpoison-inject.c @@ -67,8 +67,8 @@ static int hwpoison_unpoison(void *data, return unpoison_memory(val); } -DEFINE_SIMPLE_ATTRIBUTE(hwpoison_fops, NULL, hwpoison_inject, "%lli\n"); -DEFINE_SIMPLE_ATTRIBUTE(unpoison_fops, NULL, hwpoison_unpoison, "%lli\n"); +DEFINE_DEBUGFS_ATTRIBUTE(hwpoison_fops, NULL, hwpoison_inject, "%lli\n"); +DEFINE_DEBUGFS_ATTRIBUTE(unpoison_fops, NULL, hwpoison_unpoison, "%lli\n"); static void pfn_inject_exit(void) { From patchwork Sun Dec 1 01:57:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268461 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D53CE17F0 for ; Sun, 1 Dec 2019 01:57:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A208420873 for ; Sun, 1 Dec 2019 01:57:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="q1zajdH+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A208420873 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9C5A96B037F; Sat, 30 Nov 2019 20:57:41 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 976B06B0381; Sat, 30 Nov 2019 20:57:41 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B45E6B0383; Sat, 30 Nov 2019 20:57:41 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0053.hostedemail.com [216.40.44.53]) by kanga.kvack.org (Postfix) with ESMTP id 720406B037F for ; Sat, 30 Nov 2019 20:57:41 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 379022A87 for ; Sun, 1 Dec 2019 01:57:41 +0000 (UTC) X-FDA: 76214911122.09.lace81_4312809a3981e X-Spam-Summary: 2,0,0,d0d3af6a154e1422,d41d8cd98f00b204,akpm@linux-foundation.org,:aarcange@redhat.com:akpm@linux-foundation.org:catalin.marinas@arm.com:darrick.wong@oracle.com:dave.hansen@linux.intel.com:jannh@google.com::mhocko@suse.com:mm-commits@vger.kernel.org:richardw.yang@linux.intel.com:rppt@linux.ibm.com:steve.capper@arm.com:tiny.windzz@gmail.com:torvalds@linux-foundation.org:walken@google.com:will@kernel.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:1801:2393:2525:2559:2563:2682:2685:2692:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3865:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4605:5007:6117:6119:6261:6653:6737:7514:7576:7903:8599:8784:9025:9121:9149:9545:10004:10913:11026:11233:11658:11914:12043:12048:12295:12297:12517:12519:12555:12679:12783:12986:14096:14181:14721:14849:21080:21451:21627:21939,0,RBL:err or,Cache X-HE-Tag: lace81_4312809a3981e X-Filterd-Recvd-Size: 4206 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:40 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9DC62208E4; Sun, 1 Dec 2019 01:57:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165460; bh=9BZUYEqPfJabm0sXj7iT6KNitd/1eNtHYRGahbxfQ3w=; h=Date:From:To:Subject:From; b=q1zajdH+WZx8wm883tzqsRPpDEDhzLji1YWtg3dhuGerCK8lFI1wUnosRhMSkf+PR LA9t0O2zVRqsCCZqzss3usrfRZoflCGOzNVMEcklmNEFwoDlTku3XNmR/yY7Tl6pDv 7bpnA54q1e6AeFhDBT56QZVxzpW+QmGK0t+balJw= Date: Sat, 30 Nov 2019 17:57:39 -0800 From: akpm@linux-foundation.org To: aarcange@redhat.com, akpm@linux-foundation.org, catalin.marinas@arm.com, darrick.wong@oracle.com, dave.hansen@linux.intel.com, jannh@google.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, richardw.yang@linux.intel.com, rppt@linux.ibm.com, steve.capper@arm.com, tiny.windzz@gmail.com, torvalds@linux-foundation.org, walken@google.com, will@kernel.org Subject: [patch 142/158] mm/mmap.c: make vma_merge() comment more easy to understand Message-ID: <20191201015739.gU8SfaYW2%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wei Yang Subject: mm/mmap.c: make vma_merge() comment more easy to understand Case 1/6, 2/7 and 3/8 have the same pattern and we handle them in the same logic. Rearrange the comment to make it a little easy for audience to understand. Link: http://lkml.kernel.org/r/20191030012445.16944-1-richardw.yang@linux.intel.com Signed-off-by: Wei Yang Cc: Mike Rapoport Cc: Will Deacon Cc: Michal Hocko Cc: Catalin Marinas Cc: Andrea Arcangeli Cc: Jann Horn Cc: Darrick J. Wong Cc: Steve Capper Cc: Michel Lespinasse Cc: Dave Hansen Cc: Yangtao Li Signed-off-by: Andrew Morton --- mm/mmap.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) --- a/mm/mmap.c~mm-mmapc-make-vma_merge-comment-more-easy-to-understand +++ a/mm/mmap.c @@ -1091,15 +1091,18 @@ can_vma_merge_after(struct vm_area_struc * the area passed down from mprotect_fixup, never extending beyond one * vma, PPPPPP is the prev vma specified, and NNNNNN the next vma after: * - * AAAA AAAA AAAA AAAA - * PPPPPPNNNNNN PPPPPPNNNNNN PPPPPPNNNNNN PPPPNNNNXXXX - * cannot merge might become might become might become - * PPNNNNNNNNNN PPPPPPPPPPNN PPPPPPPPPPPP 6 or - * mmap, brk or case 4 below case 5 below PPPPPPPPXXXX 7 or - * mremap move: PPPPXXXXXXXX 8 - * AAAA - * PPPP NNNN PPPPPPPPPPPP PPPPPPPPNNNN PPPPNNNNNNNN - * might become case 1 below case 2 below case 3 below + * AAAA AAAA AAAA + * PPPPPPNNNNNN PPPPPPNNNNNN PPPPPPNNNNNN + * cannot merge might become might become + * PPNNNNNNNNNN PPPPPPPPPPNN + * mmap, brk or case 4 below case 5 below + * mremap move: + * AAAA AAAA + * PPPP NNNN PPPPNNNNXXXX + * might become might become + * PPPPPPPPPPPP 1 or PPPPPPPPPPPP 6 or + * PPPPPPPPNNNN 2 or PPPPPPPPXXXX 7 or + * PPPPNNNNNNNN 3 PPPPXXXXXXXX 8 * * It is important for case 8 that the vma NNNN overlapping the * region AAAA is never going to extended over XXXX. Instead XXXX must From patchwork Sun Dec 1 01:57:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268463 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B6F117F0 for ; Sun, 1 Dec 2019 01:57:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3F308215F1 for ; Sun, 1 Dec 2019 01:57:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZQYSm60I" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3F308215F1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 496546B0381; Sat, 30 Nov 2019 20:57:45 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 446FD6B0384; Sat, 30 Nov 2019 20:57:45 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 382E36B0385; Sat, 30 Nov 2019 20:57:45 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0098.hostedemail.com [216.40.44.98]) by kanga.kvack.org (Postfix) with ESMTP id 1EA966B0381 for ; Sat, 30 Nov 2019 20:57:45 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id CB0ED2C0C for ; Sun, 1 Dec 2019 01:57:44 +0000 (UTC) X-FDA: 76214911248.01.shape68_4397eb6b28919 X-Spam-Summary: 2,0,0,18e368f94197cbb9,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hushiyuan@huawei.com:jack@suse.cz:jgg@ziepe.ca:linfeilong@huawei.com::mhocko@suse.com:minchan@kernel.org:mm-commits@vger.kernel.org:peterz@infradead.org:rientjes@google.com:rppt@linux.ibm.com:torvalds@linux-foundation.org:yeyunfeng@huawei.com,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2198:2199:2393:2525:2553:2559:2563:2682:2685:2693:2731:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:5007:6261:6653:6737:7576:8603:9025:9545:10004:11026:11257:11473:11658:11914:12043:12048:12114:12297:12438:12517:12519:12679:12783:13069:13255:13311:13357:13846:14096:14181:14384:14721:14849:21080:21451:21627:21809:21939:30054:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SP X-HE-Tag: shape68_4397eb6b28919 X-Filterd-Recvd-Size: 3331 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:44 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 36BC720873; Sun, 1 Dec 2019 01:57:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165463; bh=HRgs0zNdmd4j/EEgkwNVV1vqEorM5hljuy9FBmeAnAY=; h=Date:From:To:Subject:From; b=ZQYSm60IdwF2VkZAHg9ZRGpJ7nxSwgZfQA2hNxtXz/WioTuIX0b4Fyz0j1PRmUFZS mEsdBfoiPyhIRTH8BfBVOaEmpiwzDJWh/Rwb/0J0DMovsSaafBIZWKhh00ZTR1bmyl j9mCmR68Ev7Z3U3vg+MfS8/tuQfRMYA5evXHOCuU= Date: Sat, 30 Nov 2019 17:57:42 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hushiyuan@huawei.com, jack@suse.cz, jgg@ziepe.ca, linfeilong@huawei.com, linux-mm@kvack.org, mhocko@suse.com, minchan@kernel.org, mm-commits@vger.kernel.org, peterz@infradead.org, rientjes@google.com, rppt@linux.ibm.com, torvalds@linux-foundation.org, yeyunfeng@huawei.com Subject: [patch 143/158] mm/madvise.c: replace with page_size() in madvise_inject_error() Message-ID: <20191201015742.tXGpYlwt0%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yunfeng Ye Subject: mm/madvise.c: replace with page_size() in madvise_inject_error() page_size() is supported after the commit a50b854e073c ("mm: introduce page_size()"). Use page_size() in madvise_inject_error() for readability. [akpm@linux-foundation.org: use ulong for `size', per David] Link: http://lkml.kernel.org/r/29dce60c-38d6-0220-f292-e298f0c78c4d@huawei.com Signed-off-by: Yunfeng Ye Reviewed-by: Andrew Morton Acked-by: David Rientjes Cc: Jason Gunthorpe Cc: Michal Hocko Cc: Minchan Kim Cc: Peter Zijlstra Cc: Jan Kara Cc: Mike Rapoport Cc: Hu Shiyuan Cc: Feilong Lin Signed-off-by: Andrew Morton --- mm/madvise.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) --- a/mm/madvise.c~mm-madvise-replace-with-page_size-in-madvise_inject_error +++ a/mm/madvise.c @@ -864,13 +864,13 @@ static int madvise_inject_error(int beha { struct page *page; struct zone *zone; - unsigned int order; + unsigned long size; if (!capable(CAP_SYS_ADMIN)) return -EPERM; - for (; start < end; start += PAGE_SIZE << order) { + for (; start < end; start += size) { unsigned long pfn; int ret; @@ -882,9 +882,9 @@ static int madvise_inject_error(int beha /* * When soft offlining hugepages, after migrating the page * we dissolve it, therefore in the second loop "page" will - * no longer be a compound page, and order will be 0. + * no longer be a compound page. */ - order = compound_order(compound_head(page)); + size = page_size(compound_head(page)); if (PageHWPoison(page)) { put_page(page); From patchwork Sun Dec 1 01:57:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268465 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9D10C17E0 for ; Sun, 1 Dec 2019 01:57:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6AA2F208C3 for ; Sun, 1 Dec 2019 01:57:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="RRXkFsFe" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6AA2F208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 61C9F6B0384; Sat, 30 Nov 2019 20:57:48 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5CBC06B0386; Sat, 30 Nov 2019 20:57:48 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4BC3A6B0387; Sat, 30 Nov 2019 20:57:48 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0199.hostedemail.com [216.40.44.199]) by kanga.kvack.org (Postfix) with ESMTP id 2D10C6B0384 for ; Sat, 30 Nov 2019 20:57:48 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id D2E5F180AD817 for ; Sun, 1 Dec 2019 01:57:47 +0000 (UTC) X-FDA: 76214911374.06.start82_440b3c3a33e42 X-Spam-Summary: 2,0,0,d45c8555d450cbe5,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org::mm-commits@vger.kernel.org:richardw.yang@linux.intel.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1540:1711:1714:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3351:3867:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6642:6653:7576:8599:9025:9545:10004:10913:11026:11658:11914:12043:12048:12297:12438:12517:12519:12555:12663:12679:12783:12986:13069:13311:13357:14181:14384:14721:14849:21080:21451:21627:21939:30054:30070,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: start82_440b3c3a33e42 X-Filterd-Recvd-Size: 2193 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:47 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A8C50215E5; Sun, 1 Dec 2019 01:57:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165466; bh=MPhSfNyCJiHCWYvKXZZ/vEXfiImlJSzzZYrO8xIwEPU=; h=Date:From:To:Subject:From; b=RRXkFsFeLk+HrSg+AxHVKucm6BiZYm7WFwMPa0sdEaUv7Bs5zDF9k5Sz9ovAAD30R nDTbyKqp+GKosPmDrVjP3qRpsPnwdCuiHAPgDcKG+MRtp3A20pxWIUovOD39bbEzq4 xCswL93fFR4itNhQijt2QZxMIoVNHiSSRwouwZPc= Date: Sat, 30 Nov 2019 17:57:46 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, richardw.yang@linux.intel.com, torvalds@linux-foundation.org Subject: [patch 144/158] mm/madvise.c: use PAGE_ALIGN[ED] for range checking Message-ID: <20191201015746.s24EMeDji%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wei Yang Subject: mm/madvise.c: use PAGE_ALIGN[ED] for range checking Improve readability, no functional change. Link: http://lkml.kernel.org/r/20191118032857.22683-1-richardw.yang@linux.intel.com Signed-off-by: Wei Yang Reviewed-by: Andrew Morton Signed-off-by: Andrew Morton --- mm/madvise.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/mm/madvise.c~madvise-use-page_align-for-range-checking +++ a/mm/madvise.c @@ -1059,9 +1059,9 @@ SYSCALL_DEFINE3(madvise, unsigned long, if (!madvise_behavior_valid(behavior)) return error; - if (start & ~PAGE_MASK) + if (!PAGE_ALIGNED(start)) return error; - len = (len_in + ~PAGE_MASK) & PAGE_MASK; + len = PAGE_ALIGN(len_in); /* Check to see whether len was rounded up from small -ve to zero */ if (len_in && !len) From patchwork Sun Dec 1 01:57:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268467 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D0B6017F0 for ; Sun, 1 Dec 2019 01:57:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9E7BF215F1 for ; Sun, 1 Dec 2019 01:57:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="0OMFLI/L" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9E7BF215F1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8B4846B0386; Sat, 30 Nov 2019 20:57:51 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 865226B0389; Sat, 30 Nov 2019 20:57:51 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 77B236B038A; Sat, 30 Nov 2019 20:57:51 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0140.hostedemail.com [216.40.44.140]) by kanga.kvack.org (Postfix) with ESMTP id 5DD8A6B0386 for ; Sat, 30 Nov 2019 20:57:51 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 10CCD2C0C for ; Sun, 1 Dec 2019 01:57:51 +0000 (UTC) X-FDA: 76214911542.13.sand00_44827552cab03 X-Spam-Summary: 2,0,0,a04ff43a70d6eb6c,d41d8cd98f00b204,akpm@linux-foundation.org,:aarcange@redhat.com:akpm@linux-foundation.org:hughd@google.com::mike.kravetz@oracle.com:mm-commits@vger.kernel.org:richardw.yang@linux.intel.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2736:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:7903:8599:8603:9025:9545:10004:10913:11026:11232:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13069:13255:13311:13357:14181:14384:14721:14849:21080:21451:21627:21939:30051:30054:30064,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: sand00_44827552cab03 X-Filterd-Recvd-Size: 3063 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:50 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id BBDEB208C3; Sun, 1 Dec 2019 01:57:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165470; bh=QABGqKEOmpC2ieUi0s5vO6cx1R2A+h0maehlOLHMO3A=; h=Date:From:To:Subject:From; b=0OMFLI/LxwfwSUvgKqJKFdxLEPRgFtQt+esZKXUkKNVDezrge96XNqQVdcBdUh470 IzuQB5aIn601YUtSKpqcMnOPRBSLIdMMZhFB7Fe1u6zVWNtberyQXvxbZL+K5xRULG 44cHeV85l2FCqnGO+0qXG6y8Wwvy2R8BqurDAYzQ= Date: Sat, 30 Nov 2019 17:57:49 -0800 From: akpm@linux-foundation.org To: aarcange@redhat.com, akpm@linux-foundation.org, hughd@google.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, richardw.yang@linux.intel.com, torvalds@linux-foundation.org Subject: [patch 145/158] userfaultfd: use vma_pagesize for all huge page size calculation Message-ID: <20191201015749.QHBpAEQOY%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wei Yang Subject: userfaultfd: use vma_pagesize for all huge page size calculation In __mcopy_atomic_hugetlb() we use two variables to deal with huge page size: vma_hpagesize and huge_page_size. Since they are the same, it is not necessary to use two different mechanism. This patch makes it consistent by all using vma_hpagesize. Link: http://lkml.kernel.org/r/20190927070032.2129-1-richardw.yang@linux.intel.com Signed-off-by: Wei Yang Reviewed-by: Mike Kravetz Cc: Andrea Arcangeli Cc: Hugh Dickins Signed-off-by: Andrew Morton --- mm/userfaultfd.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) --- a/mm/userfaultfd.c~userfaultfd-use-vma_pagesize-for-all-huge-page-size-calculation +++ a/mm/userfaultfd.c @@ -259,7 +259,7 @@ retry: pte_t dst_pteval; BUG_ON(dst_addr >= dst_start + len); - VM_BUG_ON(dst_addr & ~huge_page_mask(h)); + VM_BUG_ON(dst_addr & (vma_hpagesize - 1)); /* * Serialize via hugetlb_fault_mutex @@ -270,7 +270,7 @@ retry: mutex_lock(&hugetlb_fault_mutex_table[hash]); err = -ENOMEM; - dst_pte = huge_pte_alloc(dst_mm, dst_addr, huge_page_size(h)); + dst_pte = huge_pte_alloc(dst_mm, dst_addr, vma_hpagesize); if (!dst_pte) { mutex_unlock(&hugetlb_fault_mutex_table[hash]); goto out_unlock; @@ -297,7 +297,8 @@ retry: err = copy_huge_page_from_user(page, (const void __user *)src_addr, - pages_per_huge_page(h), true); + vma_hpagesize / PAGE_SIZE, + true); if (unlikely(err)) { err = -EFAULT; goto out; From patchwork Sun Dec 1 01:57:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268469 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E44C417F0 for ; Sun, 1 Dec 2019 01:57:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A83C3215E5 for ; Sun, 1 Dec 2019 01:57:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="mKABw++v" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A83C3215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 977096B0389; Sat, 30 Nov 2019 20:57:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9284D6B038B; Sat, 30 Nov 2019 20:57:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83E386B038C; Sat, 30 Nov 2019 20:57:54 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0232.hostedemail.com [216.40.44.232]) by kanga.kvack.org (Postfix) with ESMTP id 6A57C6B0389 for ; Sat, 30 Nov 2019 20:57:54 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 22192181AEF0B for ; Sun, 1 Dec 2019 01:57:54 +0000 (UTC) X-FDA: 76214911668.21.fang53_44f1da7e4f706 X-Spam-Summary: 2,0,0,734126a3f14b9640,d41d8cd98f00b204,akpm@linux-foundation.org,:aarcange@redhat.com:akpm@linux-foundation.org:hughd@google.com::mike.kravetz@oracle.com:mm-commits@vger.kernel.org:richardw.yang@linux.intel.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2693:2859:2902:2903:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3867:3868:3870:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:7875:8599:8603:9025:9545:9592:10004:10913:11026:11232:11473:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:12555:12679:12783:12986:13069:13146:13161:13229:13230:13255:13311:13357:14181:14384:14721:14849:21080:21451:21627:21939:30003:30054:30064,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,L Ftime:2, X-HE-Tag: fang53_44f1da7e4f706 X-Filterd-Recvd-Size: 2799 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:53 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C6FEF208C3; Sun, 1 Dec 2019 01:57:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165473; bh=c2LVyF8obRtSg4YajpZuLtO5IzDUd0Glgf6DXh1jr20=; h=Date:From:To:Subject:From; b=mKABw++v+fByoWbvjgNzU3lHgVl1+94tD5hLP82w9W7M2Tidi52mRPh7bCRxABL3S tFXX3I9HAAs/xVWXvItJIE80Shp8nQ5FGyk45SNz0qFRPanoBpb15Ba43IY5VLhBPz v9Elcs1GHZKDQ6hRRoeb5BHjG6UtLK1LETKiVPkA= Date: Sat, 30 Nov 2019 17:57:52 -0800 From: akpm@linux-foundation.org To: aarcange@redhat.com, akpm@linux-foundation.org, hughd@google.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, richardw.yang@linux.intel.com, torvalds@linux-foundation.org Subject: [patch 146/158] userfaultfd: remove unnecessary WARN_ON() in __mcopy_atomic_hugetlb() Message-ID: <20191201015752.ffTMDO_ic%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wei Yang Subject: userfaultfd: remove unnecessary WARN_ON() in __mcopy_atomic_hugetlb() These warning here is to make sure address(dst_addr) and length(len - copied) are huge page size aligned. While this is ensured by: dst_start and len is huge page size aligned dst_addr equals to dst_start and increase huge page size each time copied increase huge page size each time This means these warnings will never be triggered. Link: http://lkml.kernel.org/r/20190927070032.2129-2-richardw.yang@linux.intel.com Signed-off-by: Wei Yang Reviewed-by: Mike Kravetz Cc: Andrea Arcangeli Cc: Hugh Dickins Signed-off-by: Andrew Morton --- mm/userfaultfd.c | 5 ----- 1 file changed, 5 deletions(-) --- a/mm/userfaultfd.c~userfaultfd-remove-unnecessary-warn_on-in-__mcopy_atomic_hugetlb +++ a/mm/userfaultfd.c @@ -242,10 +242,6 @@ retry: vm_shared = dst_vma->vm_flags & VM_SHARED; } - if (WARN_ON(dst_addr & (vma_hpagesize - 1) || - (len - copied) & (vma_hpagesize - 1))) - goto out_unlock; - /* * If not shared, ensure the dst_vma has a anon_vma. */ @@ -259,7 +255,6 @@ retry: pte_t dst_pteval; BUG_ON(dst_addr >= dst_start + len); - VM_BUG_ON(dst_addr & (vma_hpagesize - 1)); /* * Serialize via hugetlb_fault_mutex From patchwork Sun Dec 1 01:57:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268471 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F375E17F0 for ; Sun, 1 Dec 2019 01:57:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B74B8208C3 for ; Sun, 1 Dec 2019 01:57:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="tBGzYmoG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B74B8208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A3A4E6B038B; Sat, 30 Nov 2019 20:57:57 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9EB4A6B038D; Sat, 30 Nov 2019 20:57:57 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 928646B038E; Sat, 30 Nov 2019 20:57:57 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0224.hostedemail.com [216.40.44.224]) by kanga.kvack.org (Postfix) with ESMTP id 763576B038B for ; Sat, 30 Nov 2019 20:57:57 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 38F434403 for ; Sun, 1 Dec 2019 01:57:57 +0000 (UTC) X-FDA: 76214911794.05.stick75_456581cf0a856 X-Spam-Summary: 2,0,0,2485ea2e2c9388d0,d41d8cd98f00b204,akpm@linux-foundation.org,:aarcange@redhat.com:akpm@linux-foundation.org:hughd@google.com::mike.kravetz@oracle.com:mm-commits@vger.kernel.org:richardw.yang@linux.intel.com:torvalds@linux-foundation.org,RULES_HIT:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1543:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3865:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6119:6261:6653:7576:7903:8599:9025:9545:9592:10004:10913:11026:11658:11914:12043:12048:12291:12296:12297:12517:12519:12555:12679:12683:12783:12986:13141:13230:13255:13972:14096:14181:14721:14849:21080:21324:21451:21627:21939:30054:30064:30070:30076,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: stick75_456581cf0a856 X-Filterd-Recvd-Size: 4408 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:56 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id DEA9122464; Sun, 1 Dec 2019 01:57:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165476; bh=R7jmxi+Pah3kqYv9zh5OH1OwGl+7e/f8Hd2LZ3Y3yyE=; h=Date:From:To:Subject:From; b=tBGzYmoGvZc7cqd7bNlzbB89rY6OEN6mnxNEnhjac1gbvB+SGLPWWrpKGxz92sZRT EOQdgKDHhSS3zd3mu6FG1lUP0RC/Z9p6hQR/PpN+vmibvl2wJXcvy43pEB3aQmjdZZ FSWX6i7Eslj15+zK8DUIUJyqUhu7Jp2h43cBc/84= Date: Sat, 30 Nov 2019 17:57:55 -0800 From: akpm@linux-foundation.org To: aarcange@redhat.com, akpm@linux-foundation.org, hughd@google.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, richardw.yang@linux.intel.com, torvalds@linux-foundation.org Subject: [patch 147/158] userfaultfd: wrap the common dst_vma check into an inlined function Message-ID: <20191201015755.26Xc_9w8u%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wei Yang Subject: userfaultfd: wrap the common dst_vma check into an inlined function When doing UFFDIO_COPY, it is necessary to find the correct destination vma and make sure fault range is in it. Since there are two places need to do the same task, just wrap those common check into an inlined function. Link: http://lkml.kernel.org/r/20190927070032.2129-3-richardw.yang@linux.intel.com Signed-off-by: Wei Yang Reviewed-by: Mike Kravetz Cc: Andrea Arcangeli Cc: Hugh Dickins Signed-off-by: Andrew Morton --- mm/userfaultfd.c | 56 +++++++++++++++++++++++++-------------------- 1 file changed, 32 insertions(+), 24 deletions(-) --- a/mm/userfaultfd.c~userfaultfd-wrap-the-common-dst_vma-check-into-an-inlined-function +++ a/mm/userfaultfd.c @@ -18,6 +18,36 @@ #include #include "internal.h" +static __always_inline +struct vm_area_struct *find_dst_vma(struct mm_struct *dst_mm, + unsigned long dst_start, + unsigned long len) +{ + /* + * Make sure that the dst range is both valid and fully within a + * single existing vma. + */ + struct vm_area_struct *dst_vma; + + dst_vma = find_vma(dst_mm, dst_start); + if (!dst_vma) + return NULL; + + if (dst_start < dst_vma->vm_start || + dst_start + len > dst_vma->vm_end) + return NULL; + + /* + * Check the vma is registered in uffd, this is required to + * enforce the VM_MAYWRITE check done at uffd registration + * time. + */ + if (!dst_vma->vm_userfaultfd_ctx.ctx) + return NULL; + + return dst_vma; +} + static int mcopy_atomic_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, struct vm_area_struct *dst_vma, @@ -220,20 +250,9 @@ retry: */ if (!dst_vma) { err = -ENOENT; - dst_vma = find_vma(dst_mm, dst_start); + dst_vma = find_dst_vma(dst_mm, dst_start, len); if (!dst_vma || !is_vm_hugetlb_page(dst_vma)) goto out_unlock; - /* - * Check the vma is registered in uffd, this is - * required to enforce the VM_MAYWRITE check done at - * uffd registration time. - */ - if (!dst_vma->vm_userfaultfd_ctx.ctx) - goto out_unlock; - - if (dst_start < dst_vma->vm_start || - dst_start + len > dst_vma->vm_end) - goto out_unlock; err = -EINVAL; if (vma_hpagesize != vma_kernel_pagesize(dst_vma)) @@ -468,20 +487,9 @@ retry: * both valid and fully within a single existing vma. */ err = -ENOENT; - dst_vma = find_vma(dst_mm, dst_start); + dst_vma = find_dst_vma(dst_mm, dst_start, len); if (!dst_vma) goto out_unlock; - /* - * Check the vma is registered in uffd, this is required to - * enforce the VM_MAYWRITE check done at uffd registration - * time. - */ - if (!dst_vma->vm_userfaultfd_ctx.ctx) - goto out_unlock; - - if (dst_start < dst_vma->vm_start || - dst_start + len > dst_vma->vm_end) - goto out_unlock; err = -EINVAL; /* From patchwork Sun Dec 1 01:57:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268473 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E743114B7 for ; Sun, 1 Dec 2019 01:58:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AA8C1215E5 for ; Sun, 1 Dec 2019 01:58:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="XiwIswuE" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AA8C1215E5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 88DF86B038D; Sat, 30 Nov 2019 20:58:00 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 83D686B038F; Sat, 30 Nov 2019 20:58:00 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 77B086B0391; Sat, 30 Nov 2019 20:58:00 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0252.hostedemail.com [216.40.44.252]) by kanga.kvack.org (Postfix) with ESMTP id 5F9E06B038D for ; Sat, 30 Nov 2019 20:58:00 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 1C8CA181AEF0B for ; Sun, 1 Dec 2019 01:58:00 +0000 (UTC) X-FDA: 76214911920.26.quilt37_45d49aef01e12 X-Spam-Summary: 13,1.2,0,042e0810d7eb33fd,d41d8cd98f00b204,akpm@linux-foundation.org,:aarcange@redhat.com:akpm@linux-foundation.org::mm-commits@vger.kernel.org:richardw.yang@linux.intel.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3867:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:8599:9025:9545:10008:10913:11026:11473:11658:11914:12043:12048:12114:12297:12438:12517:12519:12555:12679:12783:12986:13069:13221:13229:13255:13311:13357:13846:13972:14181:14384:14721:14849:21080:21451:21627:21939:30054:30076,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: quilt37_45d49aef01e12 X-Filterd-Recvd-Size: 2586 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:57:59 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E5E69222D9; Sun, 1 Dec 2019 01:57:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165479; bh=3lefjcA1D4sFNBdV7/3pICovYKekFm9dyFtZX4KzLhg=; h=Date:From:To:Subject:From; b=XiwIswuE5aS1eKmoueDepTpQME9wyyxvI6gArGE1JNy90zeZIPEGHW/oHqTcEytb/ DVbi5xF/NW6IrApZxjmDTQsbWrhodpsWDRz9GWoV3XVXShPE9ECeSPD4uQIEeqgQUO 4NJRCt8wCCeoMoWjrkXIHb8X7FQ+Cq2zbsdUzzhA= Date: Sat, 30 Nov 2019 17:57:58 -0800 From: akpm@linux-foundation.org To: aarcange@redhat.com, akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, richardw.yang@linux.intel.com, torvalds@linux-foundation.org Subject: [patch 148/158] fs/userfaultfd.c: wp: clear VM_UFFD_MISSING or VM_UFFD_WP during userfaultfd_register() Message-ID: <20191201015758.51WEakx-u%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrea Arcangeli Subject: fs/userfaultfd.c: wp: clear VM_UFFD_MISSING or VM_UFFD_WP during userfaultfd_register() If the registration is repeated without VM_UFFD_MISSING or VM_UFFD_WP they need to be cleared. Currently setting UFFDIO_REGISTER_MODE_WP returns -EINVAL, so this patch is a noop until the UFFDIO_REGISTER_MODE_WP support is applied. Link: http://lkml.kernel.org/r/20191004232834.GP13922@redhat.com Signed-off-by: Andrea Arcangeli Reported-by: Wei Yang Reviewed-by: Wei Yang Signed-off-by: Andrew Morton --- fs/userfaultfd.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) --- a/fs/userfaultfd.c~uffd-wp-clear-vm_uffd_missing-or-vm_uffd_wp-during-userfaultfd_register +++ a/fs/userfaultfd.c @@ -1460,7 +1460,8 @@ static int userfaultfd_register(struct u start = vma->vm_start; vma_end = min(end, vma->vm_end); - new_flags = (vma->vm_flags & ~vm_flags) | vm_flags; + new_flags = (vma->vm_flags & + ~(VM_UFFD_MISSING|VM_UFFD_WP)) | vm_flags; prev = vma_merge(mm, prev, start, vma_end, new_flags, vma->anon_vma, vma->vm_file, vma->vm_pgoff, vma_policy(vma), From patchwork Sun Dec 1 01:58:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268475 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 43D0815AB for ; Sun, 1 Dec 2019 01:58:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 07B38208C3 for ; Sun, 1 Dec 2019 01:58:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="J/Wlg/yI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 07B38208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EA7D46B038F; Sat, 30 Nov 2019 20:58:03 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E7F9C6B0392; Sat, 30 Nov 2019 20:58:03 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D6FD96B0393; Sat, 30 Nov 2019 20:58:03 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0218.hostedemail.com [216.40.44.218]) by kanga.kvack.org (Postfix) with ESMTP id BD5BD6B038F for ; Sat, 30 Nov 2019 20:58:03 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 65AE440FE for ; Sun, 1 Dec 2019 01:58:03 +0000 (UTC) X-FDA: 76214912046.24.scent09_464edb36f9856 X-Spam-Summary: 2,0,0,ab14305480175e56,d41d8cd98f00b204,akpm@linux-foundation.org,:aarcange@redhat.com:akpm@linux-foundation.org:cyphar@cyphar.com:dancol@google.com:jannh@google.com::lokeshgidra@google.com:mm-commits@vger.kernel.org:nnk@google.com:nosh@google.com:ovzxemul@gmail.com:rppt@linux.ibm.com:timmurray@google.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:982:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2393:2525:2553:2559:2564:2682:2685:2859:2894:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4470:5007:6261:6653:6737:7514:7576:7903:8784:9025:9545:10004:10913:11026:11257:11473:11658:11914:12043:12048:12114:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:14181:14721:14819:14849:21067:21080:21433:21451:21627:21795:21819:21939:30051:30054:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netchec k:none,D X-HE-Tag: scent09_464edb36f9856 X-Filterd-Recvd-Size: 4251 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:58:02 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id F3F0E215E5; Sun, 1 Dec 2019 01:58:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165482; bh=xCCD7HQOTr3XutaMpcMzo8KZ72AWoWeTDPh2UIfQAa0=; h=Date:From:To:Subject:From; b=J/Wlg/yIRCkqDP3RR54hYfs8KWN5Xzy16Iv0DKA7IqtZXy5MQuREbfPNsVc/AsRfg vZliaGcXtRxDA8jf/Zf62/2c3sqi9Pt5YU+CKnPBROlaWkmNfYx01OWNOKyAu6Ffez XKjpNzC+CjrjhgNitPJcrluNBgHo65FaViRjSDH8= Date: Sat, 30 Nov 2019 17:58:01 -0800 From: akpm@linux-foundation.org To: aarcange@redhat.com, akpm@linux-foundation.org, cyphar@cyphar.com, dancol@google.com, jannh@google.com, linux-mm@kvack.org, lokeshgidra@google.com, mm-commits@vger.kernel.org, nnk@google.com, nosh@google.com, ovzxemul@gmail.com, rppt@linux.ibm.com, timmurray@google.com, torvalds@linux-foundation.org Subject: [patch 149/158] userfaultfd: require CAP_SYS_PTRACE for UFFD_FEATURE_EVENT_FORK Message-ID: <20191201015801.SW5FAPUDb%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Rapoport Subject: userfaultfd: require CAP_SYS_PTRACE for UFFD_FEATURE_EVENT_FORK A while ago Andy noticed (http://lkml.kernel.org/r/CALCETrWY+5ynDct7eU_nDUqx=okQvjm=Y5wJvA4ahBja=CQXGw@mail.gmail.com) that UFFD_FEATURE_EVENT_FORK used by an unprivileged user may have security implications. As the first step of the solution the following patch limits the availably of UFFD_FEATURE_EVENT_FORK only for those having CAP_SYS_PTRACE. The usage of CAP_SYS_PTRACE ensures compatibility with CRIU. Yet, if there are other users of non-cooperative userfaultfd that run without CAP_SYS_PTRACE, they would be broken :( Current implementation of UFFD_FEATURE_EVENT_FORK modifies the file descriptor table from the read() implementation of uffd, which may have security implications for unprivileged use of the userfaultfd. Limit availability of UFFD_FEATURE_EVENT_FORK only for callers that have CAP_SYS_PTRACE. Link: http://lkml.kernel.org/r/1572967777-8812-2-git-send-email-rppt@linux.ibm.com Signed-off-by: Mike Rapoport Reviewed-by: Andrea Arcangeli Cc: Daniel Colascione Cc: Jann Horn Cc: Lokesh Gidra Cc: Nick Kralevich Cc: Nosh Minwalla Cc: Pavel Emelyanov Cc: Tim Murray Cc: Aleksa Sarai Signed-off-by: Andrew Morton --- fs/userfaultfd.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) --- a/fs/userfaultfd.c~userfaultfd-require-cap_sys_ptrace-for-uffd_feature_event_fork +++ a/fs/userfaultfd.c @@ -1835,13 +1835,12 @@ static int userfaultfd_api(struct userfa if (copy_from_user(&uffdio_api, buf, sizeof(uffdio_api))) goto out; features = uffdio_api.features; - if (uffdio_api.api != UFFD_API || (features & ~UFFD_API_FEATURES)) { - memset(&uffdio_api, 0, sizeof(uffdio_api)); - if (copy_to_user(buf, &uffdio_api, sizeof(uffdio_api))) - goto out; - ret = -EINVAL; - goto out; - } + ret = -EINVAL; + if (uffdio_api.api != UFFD_API || (features & ~UFFD_API_FEATURES)) + goto err_out; + ret = -EPERM; + if ((features & UFFD_FEATURE_EVENT_FORK) && !capable(CAP_SYS_PTRACE)) + goto err_out; /* report all available features and ioctls to userland */ uffdio_api.features = UFFD_API_FEATURES; uffdio_api.ioctls = UFFD_API_IOCTLS; @@ -1854,6 +1853,11 @@ static int userfaultfd_api(struct userfa ret = 0; out: return ret; +err_out: + memset(&uffdio_api, 0, sizeof(uffdio_api)); + if (copy_to_user(buf, &uffdio_api, sizeof(uffdio_api))) + ret = -EFAULT; + goto out; } static long userfaultfd_ioctl(struct file *file, unsigned cmd, From patchwork Sun Dec 1 01:58:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268477 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 325AC15AB for ; Sun, 1 Dec 2019 01:58:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E7F33208C3 for ; Sun, 1 Dec 2019 01:58:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="bdvTpLHB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E7F33208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C37676B0392; Sat, 30 Nov 2019 20:58:06 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BE6886B0394; Sat, 30 Nov 2019 20:58:06 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AFEBE6B0395; Sat, 30 Nov 2019 20:58:06 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0080.hostedemail.com [216.40.44.80]) by kanga.kvack.org (Postfix) with ESMTP id 984AF6B0392 for ; Sat, 30 Nov 2019 20:58:06 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 54825824999B for ; Sun, 1 Dec 2019 01:58:06 +0000 (UTC) X-FDA: 76214912172.05.hour82_46bc0753da261 X-Spam-Summary: 2,0,0,75daabdec21350e8,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:colin.king@canonical.com:hughd@google.com::mm-commits@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2553:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6119:6261:6653:7576:7903:8599:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12895:12986:13069:13255:13311:13357:13846:14094:14181:14384:14721:14849:21080:21433:21451:21627:21939:30012:30054:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: hour82_46bc0753da261 X-Filterd-Recvd-Size: 2510 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:58:05 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 34A34215F1; Sun, 1 Dec 2019 01:58:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165485; bh=G5CtEnc8ux9RHPf8KNnC8NrhnXVfQXVgYPJn7u3kcOo=; h=Date:From:To:Subject:From; b=bdvTpLHB6/AkV+l2PKjbAZXaig6/yGy5asxjc0W3x7ubKd7jKRuKjaQCK2O4nXSNv f27kAlPaFOqySzXTwntx9o2KtBmkRWS/PnNgwfwadxwMZFdXyE2xrEwMsX03Kk+GqK uiJIoNVzMjhtZVMFUNwrshZJ8ZOkbl3RbaJ8KwVQ= Date: Sat, 30 Nov 2019 17:58:04 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, colin.king@canonical.com, hughd@google.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 150/158] mm/shmem.c: make array 'values' static const, makes object smaller Message-ID: <20191201015804.qRDuXlQiq%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Colin Ian King Subject: mm/shmem.c: make array 'values' static const, makes object smaller Don't populate the array 'values' on the stack but instead make it static const. Makes the object code smaller by 111 bytes. Before: text data bss dec hex filename 108612 11169 512 120293 1d5e5 mm/shmem.o After: text data bss dec hex filename 108437 11233 512 120182 1d576 mm/shmem.o (gcc version 9.2.1, amd64) Link: http://lkml.kernel.org/r/20190906143012.28698-1-colin.king@canonical.com Signed-off-by: Colin Ian King Reviewed-by: Andrew Morton Cc: Hugh Dickins Signed-off-by: Andrew Morton --- mm/shmem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/shmem.c~mm-shmemc-make-array-values-static-const-makes-object-smaller +++ a/mm/shmem.c @@ -3932,7 +3932,7 @@ out2: static ssize_t shmem_enabled_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { - int values[] = { + static const int values[] = { SHMEM_HUGE_ALWAYS, SHMEM_HUGE_WITHIN_SIZE, SHMEM_HUGE_ADVISE, From patchwork Sun Dec 1 01:58:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268479 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 85DCC14B7 for ; Sun, 1 Dec 2019 01:58:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 49D3A2082E for ; Sun, 1 Dec 2019 01:58:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="xvapJZuk" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 49D3A2082E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 30FF36B0394; Sat, 30 Nov 2019 20:58:10 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 298636B0396; Sat, 30 Nov 2019 20:58:10 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1AED76B0398; Sat, 30 Nov 2019 20:58:10 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0144.hostedemail.com [216.40.44.144]) by kanga.kvack.org (Postfix) with ESMTP id 00FE76B0394 for ; Sat, 30 Nov 2019 20:58:09 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id AAE78180AD817 for ; Sun, 1 Dec 2019 01:58:09 +0000 (UTC) X-FDA: 76214912298.09.alley86_473683458275b X-Spam-Summary: 2,0,0,222000898cc52b10,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:hughd@google.com::mhocko@kernel.org:mm-commits@vger.kernel.org:torvalds@linux-foundation.org:yang.shi@linux.alibaba.com,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2553:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3867:3870:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6119:6120:6261:6653:7576:7901:7903:8660:9025:9391:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13069:13148:13161:13229:13230:13311:13357:14181:14384:14721:14777:14849:21060:21080:21433:21451:21627:21819:21939:30054:30075:30079:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:n one X-HE-Tag: alley86_473683458275b X-Filterd-Recvd-Size: 3011 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:58:09 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 55430205ED; Sun, 1 Dec 2019 01:58:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165488; bh=sKq6bJk0VtGEbufB7B4NAPI6JrHBFNY4zcNOFHjJ54c=; h=Date:From:To:Subject:From; b=xvapJZukyDNs6bpp841u8W3B8WxfUPUnh5sifIMUgSlRE8lBH3UmmO21flhhwxa00 IGsOe7v07qyfpXHZf6uie60/UtCC00KtYiuy60T4s5C82HpNCUy0bm31OnWxxkrd62 dUAM5NaprkfBgHpf+7R9754/fp6vk35L5DhhRt+o= Date: Sat, 30 Nov 2019 17:58:07 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, hughd@google.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, yang.shi@linux.alibaba.com Subject: [patch 151/158] mm: shmem: use proper gfp flags for shmem_writepage() Message-ID: <20191201015807.hMeXscL91%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Shi Subject: mm: shmem: use proper gfp flags for shmem_writepage() The shmem_writepage() uses GFP_ATOMIC to allocate swap cache. GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH | __GFP_ATOMIC | __GFP_KSWAPD_RECLAIM. However, shmem_writepage() should write out to swap only in response to memory pressure, so __GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself or in direct reclaim already. In addition, XArray node allocations from PF_MEMALLOC contexts could completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency reserves from being allocated. Here just copy the gfp flags used by add_to_swap(). Hugh: "a cleanup to make the two calls look the same when they don't need to be different (whereas the call from __read_swap_cache_async() rightly uses a lower priority gfp)". Link: http://lkml.kernel.org/r/1572991351-86061-1-git-send-email-yang.shi@linux.alibaba.com Signed-off-by: Yang Shi Acked-by: Hugh Dickins Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/shmem.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) --- a/mm/shmem.c~mm-shmem-use-proper-gfp-flags-for-shmem_writepage +++ a/mm/shmem.c @@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page * if (list_empty(&info->swaplist)) list_add(&info->swaplist, &shmem_swaplist); - if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) { + if (add_to_swap_cache(page, swap, + __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) { spin_lock_irq(&info->lock); shmem_recalc_inode(inode); info->swapped++; From patchwork Sun Dec 1 01:58:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268481 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9334915AB for ; Sun, 1 Dec 2019 01:58:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5F716208C3 for ; Sun, 1 Dec 2019 01:58:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="MufvndhX" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5F716208C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3D3756B0396; Sat, 30 Nov 2019 20:58:13 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 384286B0399; Sat, 30 Nov 2019 20:58:13 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 29B376B039A; Sat, 30 Nov 2019 20:58:13 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 0CC746B0396 for ; Sat, 30 Nov 2019 20:58:13 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id C297E181AEF0B for ; Sun, 1 Dec 2019 01:58:12 +0000 (UTC) X-FDA: 76214912424.21.suit02_47aced2024d36 X-Spam-Summary: 2,0,0,e72a69d8384abb24,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:cai@lca.pw:chenjun102@huawei.com:hughd@google.com::mm-commits@vger.kernel.org:torvalds@linux-foundation.org:wangkefeng.wang@huawei.com,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2393:2525:2553:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3867:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6119:6238:6261:6653:7576:7875:7903:8660:9025:9036:9163:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12783:12895:12986:13148:13161:13229:13230:13846:13870:14181:14721:14849:21080:21324:21433:21451:21611:21627:21796:21819:21939:30029:30036:30054:30056:30070:30090,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral, Custom_r X-HE-Tag: suit02_47aced2024d36 X-Filterd-Recvd-Size: 3922 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:58:12 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9205A205ED; Sun, 1 Dec 2019 01:58:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165491; bh=7U814T7aIKNwCnIDPSbFK+hkgGtpf5VTYAlK4X6w/8Q=; h=Date:From:To:Subject:From; b=MufvndhX2TMgn2DkovLOm1Mly0U9IseWCqtGX/6Pz6i89GshgxfTQPufWPQyT5BTZ jrZd5qs3JN4G9FYrIX6MfFdnTeS7gcsQ4pT5NHWTtfAPKtWQpByuQ6NMtOfS9oA03t xhtY2Bg2NeQBKKWwQfuUwwP+2rOMWbLCWbZGtGaw= Date: Sat, 30 Nov 2019 17:58:11 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, cai@lca.pw, chenjun102@huawei.com, hughd@google.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, wangkefeng.wang@huawei.com Subject: [patch 152/158] mm/shmem.c: cast the type of unmap_start to u64 Message-ID: <20191201015811.1qCFcdjbI%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Chen Jun Subject: mm/shmem.c: cast the type of unmap_start to u64 In 64bit system. sb->s_maxbytes of shmem filesystem is MAX_LFS_FILESIZE, which equal LLONG_MAX. If offset > LLONG_MAX - PAGE_SIZE, offset + len < LLONG_MAX in shmem_fallocate, which will pass the checking in vfs_fallocate. /* Check for wrap through zero too */ if (((offset + len) > inode->i_sb->s_maxbytes) || ((offset + len) < 0)) return -EFBIG; loff_t unmap_start = round_up(offset, PAGE_SIZE) in shmem_fallocate causes a overflow. Syzkaller reports a overflow problem in mm/shmem: UBSAN: Undefined behaviour in mm/shmem.c:2014:10 signed integer overflow: '9223372036854775807 + 1' cannot be represented in type 'long long int' CPU: 0 PID:17076 Comm: syz-executor0 Not tainted 4.1.46+ #1 Hardware name: linux, dummy-virt (DT) Call trace: [] dump_backtrace+0x0/0x2c8 arch/arm64/kernel/traps.c:100 [] show_stack+0x20/0x30 arch/arm64/kernel/traps.c:238 [] __dump_stack lib/dump_stack.c:15 [inline] [] ubsan_epilogue+0x18/0x70 lib/ubsan.c:164 [] handle_overflow+0x158/0x1b0 lib/ubsan.c:195 [] shmem_fallocate+0x6d0/0x820 mm/shmem.c:2104 [] vfs_fallocate+0x238/0x428 fs/open.c:312 [] SYSC_fallocate fs/open.c:335 [inline] [] SyS_fallocate+0x54/0xc8 fs/open.c:239 The highest bit of unmap_start will be appended with sign bit 1 (overflow) when calculate shmem_falloc.start: shmem_falloc.start = unmap_start >> PAGE_SHIFT. Fix it by casting the type of unmap_start to u64, when right shifted. This bug is found in LTS Linux 4.1. It also seems to exist in mainline. Link: http://lkml.kernel.org/r/1573867464-5107-1-git-send-email-chenjun102@huawei.com Signed-off-by: Chen Jun Reviewed-by: Andrew Morton Cc: Hugh Dickins Cc: Qian Cai Cc: Kefeng Wang Signed-off-by: Andrew Morton --- mm/shmem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/shmem.c~mm-cast-the-type-of-unmap_start-to-u64 +++ a/mm/shmem.c @@ -2747,7 +2747,7 @@ static long shmem_fallocate(struct file } shmem_falloc.waitq = &shmem_falloc_waitq; - shmem_falloc.start = unmap_start >> PAGE_SHIFT; + shmem_falloc.start = (u64)unmap_start >> PAGE_SHIFT; shmem_falloc.next = (unmap_end + 1) >> PAGE_SHIFT; spin_lock(&inode->i_lock); inode->i_private = &shmem_falloc; From patchwork Sun Dec 1 01:58:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268483 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A94E914B7 for ; Sun, 1 Dec 2019 01:58:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6CF50205ED for ; Sun, 1 Dec 2019 01:58:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="vqhFtUZE" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6CF50205ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5CDF36B0399; Sat, 30 Nov 2019 20:58:16 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5A5C66B039B; Sat, 30 Nov 2019 20:58:16 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4E2B06B039C; Sat, 30 Nov 2019 20:58:16 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0104.hostedemail.com [216.40.44.104]) by kanga.kvack.org (Postfix) with ESMTP id 33EA26B0399 for ; Sat, 30 Nov 2019 20:58:16 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id DA0CF40FE for ; Sun, 1 Dec 2019 01:58:15 +0000 (UTC) X-FDA: 76214912550.09.pump79_481e7b7e51c25 X-Spam-Summary: 2,0,0,73078eb1372e22d0,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:haolee.swjtu@gmail.com::mm-commits@vger.kernel.org:richardw.yang@linux.intel.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:3870:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7514:7576:7875:7903:8599:9025:9545:10004:10913:11026:11658:11914:12043:12048:12295:12296:12297:12438:12517:12519:12555:12679:12696:12737:12783:12986:13069:13161:13229:13311:13357:14181:14384:14721:14849:21080:21451:21627:21939:21972:30029:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: pump79_481e7b7e51c25 X-Filterd-Recvd-Size: 2470 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:58:15 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9679F208C3; Sun, 1 Dec 2019 01:58:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165494; bh=rWkuTjIBn+tUg/zhMPzoixOfiWhxOs3ffK5HD7Su2EM=; h=Date:From:To:Subject:From; b=vqhFtUZEA4xoJYAQMNw94lallng+Wq/vaPabpxsyxh+W3CD0YIlkHaw1jV87EzjGM hwomdTIOxZHOZPvcB3RuYXqcpR3Cr4YufWfbIliBxvG91D8QKfRdQCj7JOpVdb6D2a 3g65SXeALhPQYZXhd+7uiCSp1QQbmTyg7zh5wMAc= Date: Sat, 30 Nov 2019 17:58:14 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, haolee.swjtu@gmail.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, richardw.yang@linux.intel.com, torvalds@linux-foundation.org Subject: [patch 153/158] mm: fix struct member name in function comments Message-ID: <20191201015814.UBzEhabPo%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Hao Lee Subject: mm: fix struct member name in function comments The member in struct zonelist is _zonerefs instead of zones. Link: http://lkml.kernel.org/r/20190927144049.GA29622@haolee.github.io Signed-off-by: Hao Lee Reviewed-by: Andrew Morton Reviewed-by: Wei Yang Signed-off-by: Andrew Morton --- include/linux/mmzone.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/include/linux/mmzone.h~mm-fix-struct-member-name-in-function-comments +++ a/include/linux/mmzone.h @@ -1085,7 +1085,7 @@ static inline struct zoneref *first_zone /** * for_each_zone_zonelist_nodemask - helper macro to iterate over valid zones in a zonelist at or below a given zone index and within a nodemask * @zone - The current zone in the iterator - * @z - The current pointer within zonelist->zones being iterated + * @z - The current pointer within zonelist->_zonerefs being iterated * @zlist - The zonelist being iterated * @highidx - The zone index of the highest zone to return * @nodemask - Nodemask allowed by the allocator From patchwork Sun Dec 1 01:58:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268485 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 653E514B7 for ; Sun, 1 Dec 2019 01:58:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 330CF205ED for ; Sun, 1 Dec 2019 01:58:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="qW/sie8U" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 330CF205ED Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 35D066B039B; Sat, 30 Nov 2019 20:58:19 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 30E756B039D; Sat, 30 Nov 2019 20:58:19 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 227286B039E; Sat, 30 Nov 2019 20:58:19 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0207.hostedemail.com [216.40.44.207]) by kanga.kvack.org (Postfix) with ESMTP id 0A29D6B039B for ; Sat, 30 Nov 2019 20:58:19 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id B7396824999B for ; Sun, 1 Dec 2019 01:58:18 +0000 (UTC) X-FDA: 76214912676.20.move85_488b27475ff2f X-Spam-Summary: 2,0,0,f50a5d63a29cf301,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org::mm-commits@vger.kernel.org:richardw.yang@linux.intel.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3867:3870:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4605:5007:6261:6653:7576:8599:8957:9025:9545:10004:10913:11026:11638:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12783:12986:13069:13161:13229:13311:13357:14181:14384:14721:14849:21080:21212:21451:21627:21939:30054,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: move85_488b27475ff2f X-Filterd-Recvd-Size: 2757 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:58:18 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9858B2231B; Sun, 1 Dec 2019 01:58:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165497; bh=9E9dAij0q9QfEpJ1LXg2YO3y9ARcfzAtKub0GvgVjFI=; h=Date:From:To:Subject:From; b=qW/sie8U1jNHUffXfWg+uY8ncf8LOJ6ATkJSiVN2Td6GvlcfEFNg1wv6Wp91T6Rut DbN8+9vcFCYeJuqaf9uKNfNl1lbLF7Wp7xw4F5BZDyzXey0Disvp0QSGvOMNjLC0m/ m6ATutL8KHtWi0acxUIjd76I6sub2sQv0tgrg9u8= Date: Sat, 30 Nov 2019 17:58:17 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, richardw.yang@linux.intel.com, torvalds@linux-foundation.org Subject: [patch 154/158] mm: fix typos in comments when calling __SetPageUptodate() Message-ID: <20191201015817.izJ8MDym6%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wei Yang Subject: mm: fix typos in comments when calling __SetPageUptodate() There are several places emphasise the effect of __SetPageUptodate(), while the comment seems to have a typo in two places. Link: http://lkml.kernel.org/r/20190926023705.7226-1-richardw.yang@linux.intel.com Signed-off-by: Wei Yang Reviewed-by: Andrew Morton Signed-off-by: Andrew Morton --- mm/memory.c | 2 +- mm/userfaultfd.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) --- a/mm/memory.c~mm-fix-typo-in-the-comment-when-calling-function-__setpageuptodate +++ a/mm/memory.c @@ -3105,7 +3105,7 @@ static vm_fault_t do_anonymous_page(stru /* * The memory barrier inside __SetPageUptodate makes sure that - * preceeding stores to the page contents become visible before + * preceding stores to the page contents become visible before * the set_pte_at() write. */ __SetPageUptodate(page); --- a/mm/userfaultfd.c~mm-fix-typo-in-the-comment-when-calling-function-__setpageuptodate +++ a/mm/userfaultfd.c @@ -90,7 +90,7 @@ static int mcopy_atomic_pte(struct mm_st /* * The memory barrier inside __SetPageUptodate makes sure that - * preceeding stores to the page contents become visible before + * preceding stores to the page contents become visible before * the set_pte_at() write. */ __SetPageUptodate(page); From patchwork Sun Dec 1 01:58:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268487 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7406914B7 for ; Sun, 1 Dec 2019 01:58:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 41D9C208E4 for ; Sun, 1 Dec 2019 01:58:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="QjW5PybJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 41D9C208E4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 50A236B039D; Sat, 30 Nov 2019 20:58:22 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4BA816B039F; Sat, 30 Nov 2019 20:58:22 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3D3166B03A1; Sat, 30 Nov 2019 20:58:22 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0046.hostedemail.com [216.40.44.46]) by kanga.kvack.org (Postfix) with ESMTP id 25C286B039D for ; Sat, 30 Nov 2019 20:58:22 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id CF3D2824999B for ; Sun, 1 Dec 2019 01:58:21 +0000 (UTC) X-FDA: 76214912802.03.basin62_48fe23357fd01 X-Spam-Summary: 2,0,0,6710dd54ba11cab2,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:david@redhat.com:jgross@suse.com:jrdr.linux@gmail.com:kirill@shutemov.name::mhocko@suse.com:mm-commits@vger.kernel.org:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:968:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2194:2198:2199:2200:2393:2525:2559:2565:2682:2685:2731:2741:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3868:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4605:5007:6261:6653:7514:7576:7875:8603:8957:9025:9545:9592:10004:11026:11257:11473:11658:11914:12043:12048:12114:12297:12438:12517:12519:12555:12679:12783:12986:13846:14181:14721:14849:21080:21251:21451:21627:21939:30054,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:2,LUA_SUMMARY:none X-HE-Tag: basin62_48fe23357fd01 X-Filterd-Recvd-Size: 4027 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:58:21 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9BF86205ED; Sun, 1 Dec 2019 01:58:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165500; bh=KSs+SH/nQy9mZI7aCyAu5GWu91zMkMha3v8XB7mwPI0=; h=Date:From:To:Subject:From; b=QjW5PybJw0q3JQjvtOsYjxZ+ncvvAqg5DQl0s/L+yO4hWoLngOcR+4nkCMfwpinyd e3W3k+zVjXbTXgMNpHDZerJByvJeLadC5yKlUYFlrDq/kfoi6125sPCUVeyf5hEe71 9w9bOqzMrLu97T1jxZ39ALlK8M/YZDODQhY1GkR4= Date: Sat, 30 Nov 2019 17:58:20 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, david@redhat.com, jgross@suse.com, jrdr.linux@gmail.com, kirill@shutemov.name, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 155/158] mm/memory_hotplug.c: remove __online_page_set_limits() Message-ID: <20191201015820.uOoD9rRC6%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Souptick Joarder Subject: mm/memory_hotplug.c: remove __online_page_set_limits() __online_page_set_limits() is a dummy function - remove it and all callers. Link: http://lkml.kernel.org/r/8e1bc9d3b492f6bde16e95ebc1dee11d6aefabd7.1567889743.git.jrdr.linux@gmail.com Link: http://lkml.kernel.org/r/854db2cf8145d9635249c95584d9a91fd774a229.1567889743.git.jrdr.linux@gmail.com Link: http://lkml.kernel.org/r/9afe6c5a18158f3884a6b302ac2c772f3da49ccc.1567889743.git.jrdr.linux@gmail.com Signed-off-by: Souptick Joarder Reviewed-by: David Hildenbrand Acked-by: Michal Hocko Cc: Juergen Gross Cc: "Kirill A. Shutemov" Signed-off-by: Andrew Morton --- drivers/hv/hv_balloon.c | 1 - drivers/xen/balloon.c | 1 - include/linux/memory_hotplug.h | 2 -- mm/memory_hotplug.c | 5 ----- 4 files changed, 9 deletions(-) --- a/drivers/hv/hv_balloon.c~mm-memory_hotplugc-remove-__online_page_set_limits +++ a/drivers/hv/hv_balloon.c @@ -680,7 +680,6 @@ static void hv_page_online_one(struct hv __ClearPageOffline(pg); /* This frame is currently backed; online the page. */ - __online_page_set_limits(pg); generic_online_page(pg, 0); lockdep_assert_held(&dm_device.ha_lock); --- a/drivers/xen/balloon.c~mm-memory_hotplugc-remove-__online_page_set_limits +++ a/drivers/xen/balloon.c @@ -374,7 +374,6 @@ static void xen_online_page(struct page mutex_lock(&balloon_mutex); for (i = 0; i < size; i++) { p = pfn_to_page(start_pfn + i); - __online_page_set_limits(p); balloon_append(p); } mutex_unlock(&balloon_mutex); --- a/include/linux/memory_hotplug.h~mm-memory_hotplugc-remove-__online_page_set_limits +++ a/include/linux/memory_hotplug.h @@ -106,8 +106,6 @@ extern void generic_online_page(struct p extern int set_online_page_callback(online_page_callback_t callback); extern int restore_online_page_callback(online_page_callback_t callback); -extern void __online_page_set_limits(struct page *page); - extern int try_online_node(int nid); extern int arch_add_memory(int nid, u64 start, u64 size, --- a/mm/memory_hotplug.c~mm-memory_hotplugc-remove-__online_page_set_limits +++ a/mm/memory_hotplug.c @@ -598,11 +598,6 @@ int restore_online_page_callback(online_ } EXPORT_SYMBOL_GPL(restore_online_page_callback); -void __online_page_set_limits(struct page *page) -{ -} -EXPORT_SYMBOL_GPL(__online_page_set_limits); - void generic_online_page(struct page *page, unsigned int order) { kernel_map_pages(page, 1 << order, 1); From patchwork Sun Dec 1 01:58:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268489 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BDF5C15AB for ; Sun, 1 Dec 2019 01:58:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8C23920873 for ; Sun, 1 Dec 2019 01:58:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="Bj/8pvL5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8C23920873 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9EC2A6B039F; Sat, 30 Nov 2019 20:58:25 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 99D966B03AB; Sat, 30 Nov 2019 20:58:25 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 88C016B03B4; Sat, 30 Nov 2019 20:58:25 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0091.hostedemail.com [216.40.44.91]) by kanga.kvack.org (Postfix) with ESMTP id 717086B039F for ; Sat, 30 Nov 2019 20:58:25 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 2E42340FB for ; Sun, 1 Dec 2019 01:58:25 +0000 (UTC) X-FDA: 76214912970.20.jail90_4974e435f7458 X-Spam-Summary: 2,0,0,76ea5a397b46a079,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:david@redhat.com:gregkh@linuxfoundation.org:krzk@kernel.org::mm-commits@vger.kernel.org:torvalds@linux-foundation.org:trivial@kernel.org:yamada.masahiro@socionext.com,RULES_HIT:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1434:1437:1535:1544:1711:1730:1747:1777:1792:2194:2198:2199:2200:2393:2525:2559:2563:2682:2685:2731:2741:2859:2897:2902:2910:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3873:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4605:5007:6119:6261:6653:7576:7903:9025:9391:9545:10004:10913:11026:11232:11233:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13846:14093:14096:14181:14721:14819:14849:21080:21433:21451:21611:21627:21795:21819:21939:30003:30051:30054:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netche ck:none, X-HE-Tag: jail90_4974e435f7458 X-Filterd-Recvd-Size: 5300 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:58:24 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A9B4E2082E; Sun, 1 Dec 2019 01:58:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165504; bh=p8mhqXPe3b4hrGJAxbgQ2gU0ZPf709yUm9NrPc1hmko=; h=Date:From:To:Subject:From; b=Bj/8pvL5xbaV/Z/C076TalT00bY3xDCML4TgBJE2wQunmnI04PIS82Egajz2uKzvl PTUKern01VQMUaGFRFovg+egSS4411ReivxbWZrw/wPgH7n55+B2h6HZUg/3zWRMpY jPRpbP7rHVpkqBloCzyHUDuiRGv0jaSYaRvM1ags= Date: Sat, 30 Nov 2019 17:58:23 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, david@redhat.com, gregkh@linuxfoundation.org, krzk@kernel.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, trivial@kernel.org, yamada.masahiro@socionext.com Subject: [patch 156/158] mm/Kconfig: fix indentation Message-ID: <20191201015823.pxOdngZ5n%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Krzysztof Kozlowski Subject: mm/Kconfig: fix indentation Adjust indentation from spaces to tab (+optional two spaces) as in coding style with command like: $ sed -e 's/^ / /' -i */Kconfig Link: http://lkml.kernel.org/r/1574306437-28837-1-git-send-email-krzk@kernel.org Signed-off-by: Krzysztof Kozlowski Reviewed-by: David Hildenbrand Cc: Greg Kroah-Hartman Cc: Jiri Kosina Cc: Masahiro Yamada Signed-off-by: Andrew Morton --- mm/Kconfig | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) --- a/mm/Kconfig~mm-fix-kconfig-indentation +++ a/mm/Kconfig @@ -122,9 +122,9 @@ config SPARSEMEM_VMEMMAP depends on SPARSEMEM && SPARSEMEM_VMEMMAP_ENABLE default y help - SPARSEMEM_VMEMMAP uses a virtually mapped memmap to optimise - pfn_to_page and page_to_pfn operations. This is the most - efficient option when sufficient kernel resources are available. + SPARSEMEM_VMEMMAP uses a virtually mapped memmap to optimise + pfn_to_page and page_to_pfn operations. This is the most + efficient option when sufficient kernel resources are available. config HAVE_MEMBLOCK_NODE_MAP bool @@ -160,9 +160,9 @@ config MEMORY_HOTPLUG_SPARSE depends on SPARSEMEM && MEMORY_HOTPLUG config MEMORY_HOTPLUG_DEFAULT_ONLINE - bool "Online the newly added memory blocks by default" - depends on MEMORY_HOTPLUG - help + bool "Online the newly added memory blocks by default" + depends on MEMORY_HOTPLUG + help This option sets the default policy setting for memory hotplug onlining policy (/sys/devices/system/memory/auto_online_blocks) which determines what happens to newly added memory regions. Policy setting @@ -227,14 +227,14 @@ config COMPACTION select MIGRATION depends on MMU help - Compaction is the only memory management component to form - high order (larger physically contiguous) memory blocks - reliably. The page allocator relies on compaction heavily and - the lack of the feature can lead to unexpected OOM killer - invocations for high order memory requests. You shouldn't - disable this option unless there really is a strong reason for - it and then we would be really interested to hear about that at - linux-mm@kvack.org. + Compaction is the only memory management component to form + high order (larger physically contiguous) memory blocks + reliably. The page allocator relies on compaction heavily and + the lack of the feature can lead to unexpected OOM killer + invocations for high order memory requests. You shouldn't + disable this option unless there really is a strong reason for + it and then we would be really interested to hear about that at + linux-mm@kvack.org. # # support for page migration @@ -258,7 +258,7 @@ config ARCH_ENABLE_THP_MIGRATION bool config CONTIG_ALLOC - def_bool (MEMORY_ISOLATION && COMPACTION) || CMA + def_bool (MEMORY_ISOLATION && COMPACTION) || CMA config PHYS_ADDR_T_64BIT def_bool 64BIT @@ -302,10 +302,10 @@ config KSM root has set /sys/kernel/mm/ksm/run to 1 (if CONFIG_SYSFS is set). config DEFAULT_MMAP_MIN_ADDR - int "Low address space to protect from user allocation" + int "Low address space to protect from user allocation" depends on MMU - default 4096 - help + default 4096 + help This is the portion of low virtual memory which should be protected from userspace allocation. Keeping a user from writing to low pages can help reduce the impact of kernel NULL pointer bugs. @@ -408,7 +408,7 @@ choice endchoice config ARCH_WANTS_THP_SWAP - def_bool n + def_bool n config THP_SWAP def_bool y From patchwork Sun Dec 1 01:58:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268491 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E91ED15AB for ; Sun, 1 Dec 2019 01:58:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B5BC120873 for ; Sun, 1 Dec 2019 01:58:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="tiux2ll1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B5BC120873 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9EDB76B03AB; Sat, 30 Nov 2019 20:58:28 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 99CF26B03B5; Sat, 30 Nov 2019 20:58:28 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83D8B6B03BD; Sat, 30 Nov 2019 20:58:28 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0102.hostedemail.com [216.40.44.102]) by kanga.kvack.org (Postfix) with ESMTP id 656566B03AB for ; Sat, 30 Nov 2019 20:58:28 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 129A4181AEF0B for ; Sun, 1 Dec 2019 01:58:28 +0000 (UTC) X-FDA: 76214913096.23.angle91_49e423ee19c42 X-Spam-Summary: 2,0,0,bcbc594c8a2e7343,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org::mm-commits@vger.kernel.org:rdunlap@infradead.org:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1540:1711:1714:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3350:3868:3870:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:5007:6261:6653:7576:8784:9025:9545:10004:11257:11658:11914:12043:12048:12294:12296:12297:12517:12519:12679:12783:13069:13311:13357:13846:14094:14181:14384:14721:14849:21080:21433:21451:21627:21939:30003:30054,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: angle91_49e423ee19c42 X-Filterd-Recvd-Size: 2088 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:58:27 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id CE9782082E; Sun, 1 Dec 2019 01:58:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165507; bh=VamCAm/PjRs6RmUhAP6fcMSYHglrK1fu+i24uBYfO7M=; h=Date:From:To:Subject:From; b=tiux2ll17bjKy/gRGKlEEetxphPMrg+glClwmjLjfugXHX/BleRO1n5HybOGmWBAI xh/DDhQIOnaqgHYTLcnrhLh4jTs+1Ov3VEKblc9iTUrzzNl/kPonjTDmatUiGvKrku UYyiUmrqQ6VwTV3Y5+ebHnbCavV0G1J70cwhqJTg= Date: Sat, 30 Nov 2019 17:58:26 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, rdunlap@infradead.org, torvalds@linux-foundation.org Subject: [patch 157/158] mm/Kconfig: fix trivial help text punctuation Message-ID: <20191201015826.oV8pudFgm%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Randy Dunlap Subject: mm/Kconfig: fix trivial help text punctuation End a Kconfig help text sentence with a period (aka full stop). Link: http://lkml.kernel.org/r/c17f2c75-dc2a-42a4-2229-bb6b489addf2@infradead.org Signed-off-by: Randy Dunlap Reviewed-by: Andrew Morton Signed-off-by: Andrew Morton --- mm/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/Kconfig~mm-fix-trivial-help-text-punctuation +++ a/mm/Kconfig @@ -29,7 +29,7 @@ config FLATMEM_MANUAL For systems that have holes in their physical address spaces and for features like NUMA and memory hotplug, - choose "Sparse Memory" + choose "Sparse Memory". If unsure, choose this option (Flat Memory) over any other. From patchwork Sun Dec 1 01:58:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11268493 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F020C14B7 for ; Sun, 1 Dec 2019 01:58:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B350D2082E for ; Sun, 1 Dec 2019 01:58:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="u0nXVymP" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B350D2082E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AD86B6B03B5; Sat, 30 Nov 2019 20:58:31 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A87906B03C1; Sat, 30 Nov 2019 20:58:31 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 99D6C6B0491; Sat, 30 Nov 2019 20:58:31 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0009.hostedemail.com [216.40.44.9]) by kanga.kvack.org (Postfix) with ESMTP id 7DE776B03B5 for ; Sat, 30 Nov 2019 20:58:31 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 2A47B180AD817 for ; Sun, 1 Dec 2019 01:58:31 +0000 (UTC) X-FDA: 76214913222.27.north36_4a562ac883f46 X-Spam-Summary: 2,0,0,d77bb1c6fa2a37a6,d41d8cd98f00b204,akpm@linux-foundation.org,:akpm@linux-foundation.org:ddstreet@ieee.org:hannes@cmpxchg.org::minchan@google.com:mm-commits@vger.kernel.org:shakeelb@google.com:sjenning@redhat.com:torvalds@linux-foundation.org,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2890:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4042:4250:4321:5007:6261:6653:7576:8599:8957:9025:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12783:12986:13161:13229:13846:14096:14181:14721:14849:21080:21450:21451:21627:21740:21939:30001:30054:30070,0,RBL:error,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:3,LUA_SUMMARY:none X-HE-Tag: north36_4a562ac883f46 X-Filterd-Recvd-Size: 3863 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Sun, 1 Dec 2019 01:58:30 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id DEAAB222D9; Sun, 1 Dec 2019 01:58:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1575165510; bh=hTBPmiCmtgFDC4bvIGY/e1TwiV6jlZs7P0ElXXhLqo4=; h=Date:From:To:Subject:From; b=u0nXVymPzpmr2KjqlGaUDvvVbR85JeKGD4y15COMIF2L9GnSYxW01I00s1BNlxal+ iLv46sJDKGDm0epkrf1KnRk8FWXB0Z4h29cqUOjerPl867OETdwaFO1tj1GJr4W3DH eg+v3mJRsdRLSX7oWmio5c8CdsPDeZlKlVKBY6Iw= Date: Sat, 30 Nov 2019 17:58:29 -0800 From: akpm@linux-foundation.org To: akpm@linux-foundation.org, ddstreet@ieee.org, hannes@cmpxchg.org, linux-mm@kvack.org, minchan@google.com, mm-commits@vger.kernel.org, shakeelb@google.com, sjenning@redhat.com, torvalds@linux-foundation.org Subject: [patch 158/158] mm/page_io.c: annotate refault stalls from swap_readpage Message-ID: <20191201015829.n0D0_Bh2c%akpm@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Minchan Kim Subject: mm/page_io.c: annotate refault stalls from swap_readpage If a block device supports rw_page operation, it doesn't submit bios so the annotation in submit_bio() for refault stall doesn't work. It happens with zram in android, especially swap read path which could consume CPU cycle for decompress. It is also a problem for zswap which uses frontswap. Annotate swap_readpage() to account the synchronous IO overhead to prevent underreport memory pressure. [akpm@linux-foundation.org: add comment, per Johannes] Link: http://lkml.kernel.org/r/20191010152134.38545-1-minchan@kernel.org Signed-off-by: Minchan Kim Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Cc: Seth Jennings Cc: Dan Streetman Signed-off-by: Andrew Morton --- mm/page_io.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) --- a/mm/page_io.c~mm-annotate-refault-stalls-from-swap_readpage +++ a/mm/page_io.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -354,10 +355,19 @@ int swap_readpage(struct page *page, boo struct swap_info_struct *sis = page_swap_info(page); blk_qc_t qc; struct gendisk *disk; + unsigned long pflags; VM_BUG_ON_PAGE(!PageSwapCache(page) && !synchronous, page); VM_BUG_ON_PAGE(!PageLocked(page), page); VM_BUG_ON_PAGE(PageUptodate(page), page); + + /* + * Count submission time as memory stall. When the device is congested, + * or the submitting cgroup IO-throttled, submission can be a + * significant part of overall IO time. + */ + psi_memstall_enter(&pflags); + if (frontswap_load(page) == 0) { SetPageUptodate(page); unlock_page(page); @@ -371,7 +381,7 @@ int swap_readpage(struct page *page, boo ret = mapping->a_ops->readpage(swap_file, page); if (!ret) count_vm_event(PSWPIN); - return ret; + goto out; } ret = bdev_read_page(sis->bdev, swap_page_sector(page), page); @@ -382,7 +392,7 @@ int swap_readpage(struct page *page, boo } count_vm_event(PSWPIN); - return 0; + goto out; } ret = 0; @@ -418,6 +428,7 @@ int swap_readpage(struct page *page, boo bio_put(bio); out: + psi_memstall_leave(&pflags); return ret; }