From patchwork Sat Jun 22 00:05:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11011003 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 51830924 for ; Sat, 22 Jun 2019 00:05:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 436512847E for ; Sat, 22 Jun 2019 00:05:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 372632862A; Sat, 22 Jun 2019 00:05:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 165F52847E for ; Sat, 22 Jun 2019 00:05:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 363878E0002; Fri, 21 Jun 2019 20:05:26 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3125C8E0001; Fri, 21 Jun 2019 20:05:26 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 200998E0002; Fri, 21 Jun 2019 20:05:26 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id DF5D38E0001 for ; Fri, 21 Jun 2019 20:05:25 -0400 (EDT) Received: by mail-pf1-f198.google.com with SMTP id y7so5324021pfy.9 for ; Fri, 21 Jun 2019 17:05:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=EqtD9pgjSn3RpeCA8R1A7DD1Wom8eSWCEiXrtl71yV4=; b=RUcitjrliXkm6xyf/tbv2FLktihtaLLHj4zHEfJID2EtMTSSwnP9J9CJolOMVqaM9M t4WkC+fABHSarM2INmcOQwe9LISzyiVSZEALeT/HOd1pn5o9uawYseZCfUL4iGma8bcl 4nWjFF24HFHHUFyVsLHV+QK1qCt5UC9N0mAm3VKom+7yA/80VEhid/8U5Jd2J6sKbc+V ObALMDKU4DJ7z9OBrTXT+hkDvWKbNtYHMYyokQTsc6RJMd/pGxcpvWg9t46pVdW3Izih qaZ1DMHdPTms85yPzCQL8OTf280YGeOXKSeiR48WyTlha3Xd21PytYazr25KnrM+4oR3 DgeA== X-Gm-Message-State: APjAAAV/ayM3ASatConS6hR+oGwJkYqrUw1o4P2uBO+C1DsmiGT12cWo DGUPobhie8OnU2UE2fAEwiDbbrtzGGxl9HFbKIWAepvV/Ks8REMZAZQWiGYpzsRgnpzmei7ysIG ilY/t15rw3p+5b+zdn1brHfNaPdcIj1QAyNbHcNY5E1QXWTtrweg+2jSBgxPTkcULnA== X-Received: by 2002:a63:4105:: with SMTP id o5mr21635880pga.308.1561161925476; Fri, 21 Jun 2019 17:05:25 -0700 (PDT) X-Google-Smtp-Source: APXvYqy1js7B/r8Spj06GD/J0v3rb490c+xaa44/swQ3Hbj3cyOPvuR40i/OZzXjyaX9mfYxUqZC X-Received: by 2002:a63:4105:: with SMTP id o5mr21635786pga.308.1561161924264; Fri, 21 Jun 2019 17:05:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561161924; cv=none; d=google.com; s=arc-20160816; b=Iy1+bhjwptuHgZWMwrDUUjhz/6e1ZdDR5qns6TowHf+j9BOuNUgTNPviffGXMZit+d 2NOMfOgzxtwra2IGNMVS2tHS3pKc7cnMBDlnwJORMNAI+Qz21fOqvG4asWDy8lOOBAyL KWv7VupK1SvS4G84cfTeRu2rrHCr0cpXvX/VY2+E6plPGRP6AtcLfqlQsoTNPoCG/x9n MIMOaLhS4sPmGBlcmR1Cl+YwDzluzDY4unV5uDBPdumzWTrImkyj/aiFHO/Gkl/gvBoJ HbEKzJ5jkQDSxvB/lss38ScMrq4UiKLsudpde1aVWWjqqCrINkdEOX7njCXX5it1xmgq n7Yg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=EqtD9pgjSn3RpeCA8R1A7DD1Wom8eSWCEiXrtl71yV4=; b=BDBxc6fOEn5xrnNeGk32Rdcg0w21SB32Cf1+OhmqfjnZH5YpL9vi5B0ygIFQQXRBGi EW2fc8H5UkabfztLNxsovqHcrGQ1wWXu51XHGW+3KN2jFw9aXqlUdFVZrPKWhd4oTeVY Gkzu+hFWY7TniRc3bFQZ7XdCipiEfEGttEtsjbULvNBJM7UCJY8F8iaVG+bQSoG4wIV8 vMVClmZcI5592IO9aju11z9PL9laSpl9bwZ/K0ojIdyf41AuPaisu6u7hV/+2moLtjgv XfYUpDE3x18PHzrDegsSTmH/R0PC/x/R/1PwiPx6NDjOTNvtZopK/4yyzWx61xBngMNG qN8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Ot34M4pG; spf=pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1076a8f7d5=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com. [67.231.145.42]) by mx.google.com with ESMTPS id j14si3955772pfe.183.2019.06.21.17.05.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Jun 2019 17:05:24 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.145.42 as permitted sender) client-ip=67.231.145.42; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Ot34M4pG; spf=pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1076a8f7d5=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5LNsfcH018841 for ; Fri, 21 Jun 2019 17:05:23 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=EqtD9pgjSn3RpeCA8R1A7DD1Wom8eSWCEiXrtl71yV4=; b=Ot34M4pGIBjEv7uH8XSL69Ax226lIXalnxy1je4+lnAicyxhR2+EdWYHGIdnEAtC5s46 70XsPwKD2rAcTTXHA+RZpFxs8hRZcfhoAp2yyNFxIEEqWk/znsTaB3VKL/naG5gS8/7q C7ojZeLOTssgN8avSzUbFrCij9EdvTmyDAs= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2t91rg1qm5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Fri, 21 Jun 2019 17:05:23 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::127) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Fri, 21 Jun 2019 17:05:22 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 8299662E2D56; Fri, 21 Jun 2019 17:05:20 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v6 1/6] filemap: check compound_head(page)->mapping in filemap_fault() Date: Fri, 21 Jun 2019 17:05:07 -0700 Message-ID: <20190622000512.923867-2-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190622000512.923867-1-songliubraving@fb.com> References: <20190622000512.923867-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-21_16:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=922 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906210182 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Currently, filemap_fault() avoids trace condition with truncate by checking page->mapping == mapping. This does not work for compound pages. This patch let it check compound_head(page)->mapping instead. Acked-by: Rik van Riel Signed-off-by: Song Liu --- mm/filemap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/filemap.c b/mm/filemap.c index df2006ba0cfa..f5b79a43946d 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2517,7 +2517,7 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) goto out_retry; /* Did it get truncated? */ - if (unlikely(page->mapping != mapping)) { + if (unlikely(compound_head(page)->mapping != mapping)) { unlock_page(page); put_page(page); goto retry_find; From patchwork Sat Jun 22 00:05:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11011013 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C6465924 for ; Sat, 22 Jun 2019 00:05:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B853820008 for ; Sat, 22 Jun 2019 00:05:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AC78C2862A; Sat, 22 Jun 2019 00:05:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 38EBE20008 for ; Sat, 22 Jun 2019 00:05:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 229868E0007; Fri, 21 Jun 2019 20:05:41 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1DCC98E0001; Fri, 21 Jun 2019 20:05:41 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 054EF8E0007; Fri, 21 Jun 2019 20:05:40 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id BC9658E0001 for ; Fri, 21 Jun 2019 20:05:40 -0400 (EDT) Received: by mail-pf1-f200.google.com with SMTP id e25so4177648pfn.5 for ; Fri, 21 Jun 2019 17:05:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=z7orUW1w2AFawDH57ZTLvYALJSDsMI3HPUuppHQ+jCM=; b=o4/yqFlv32zlBu3St4mGRnQupfTx4yogeyDXgxYIDCwTgdWGNTrgP8VxEKbtYAVokW RLXJU7sKQRM9gu/Zi7og3es5p/vfwMH2btth6mhTozsPTyMhyeYrUlAiABFCLgOhKXAs I2TWH/SW0DwkrR9HCzHLC8d/VnDRJ8lqH36uSZnjLS6aLiS2VC26aBgJKw/jjh0mLdeQ ocnIIlWplizB5/Bpa1b3QCIxVh2EjIQdhESpb3L1BgSIup8ureBXDBtjZRJG6BQVVCEI OydPp0OtixMi6hK+OCb14b98kEde/juL++F2g1ZDbSEFnnoTxhyeqlir08zXF47gJbgH aE4g== X-Gm-Message-State: APjAAAXG/PdYrqm0x/AN3aKk94w3VsEm3cSY6YWNxR59obnko92LTRbS +ojJdkYCC08vo3DnSc5eL0wN5XRvFvRAruqCxIQHgyexK5pR6t2k4X7UBMpRSn9DfJOBIylkQBf 0/vb8HTw353NuC9lyPlraL9vIZifSDS7o5W188zvuFYWygrKawc/6CDZxeDrp74YDXg== X-Received: by 2002:a17:902:7793:: with SMTP id o19mr73236252pll.110.1561161940453; Fri, 21 Jun 2019 17:05:40 -0700 (PDT) X-Google-Smtp-Source: APXvYqzH/SQbt3DXf+EqMApEAJwR1YtcK07J/8Eyw8xv666cwY6WFt/sqhGqij8fAf5kgB4YRqu7 X-Received: by 2002:a17:902:7793:: with SMTP id o19mr73236151pll.110.1561161938949; Fri, 21 Jun 2019 17:05:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561161938; cv=none; d=google.com; s=arc-20160816; b=alIeIV/XYNqOoBTqyWyq8jA+DQ7FUf8QQhswgl23TsrAK5a2c6Qu956h2+f95gPobW 4KVG/od4VgyrMf8aTstOpPcTf8u4/cwjPvd3zETCIlo3PzkqhE+5/BqTxl9svLWLyuwD bu7K8tUMJbJv1TnjCeKlPNFRVVwSsFFCe08oij/uIaVssHxPMaEljKeNIixarv9pkc1I Ihf1eILtq2GpnLKZoGR4ud/jOXQ7dtM7VDCZwrD73JGsuD+wVFLbTFlTwiDCNYn5oxsT 7Knix2c+Cnk8W2t5RMv5Y02/b3/VIs7dxz05+IIJ8iyMJgQNWGGHlGqlw5j0jGKHE7Hi TNUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=z7orUW1w2AFawDH57ZTLvYALJSDsMI3HPUuppHQ+jCM=; b=azyD7SMr9i7nXD0hngViSSS4C2Dq9/qkPvnHXS8P3Gpqu47QYX6Cgx6s5SoWHmZp5m Z42yizruikfyWQfTV0w607hHgBc7eb1Rp/SEQh46tRrGJN3lFxYt3Ca92QXYSnvevCv6 PxHjZSkXx02tf1gQUGgV3EdG5LFa8wWV3XrDBVeXhUOYneQaqZWsJeREyG83x6Jo00Y9 Uwc5JQhO027GHJ19pkk7NtcnNQLSjoQh9eUp8x2BSi/T3r8FzEl0jl+ziDvtoY1NYBd2 rrvXWuIwqoS3/Roe57CcwP+qQ1DIkJqCLdRqGhRQa/J3McnAwtInJzJTMZ9+GAyaRYYd gbvQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=DdeLBro7; spf=pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1076a8f7d5=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com. [67.231.145.42]) by mx.google.com with ESMTPS id o4si3749190pgi.160.2019.06.21.17.05.38 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Jun 2019 17:05:38 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.145.42 as permitted sender) client-ip=67.231.145.42; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=DdeLBro7; spf=pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1076a8f7d5=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0044008.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5LNubfo026471 for ; Fri, 21 Jun 2019 17:05:38 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=z7orUW1w2AFawDH57ZTLvYALJSDsMI3HPUuppHQ+jCM=; b=DdeLBro7s+f3+XblTN5J2RqMc3lasE1XneaauOlUUJsvJ+FD5czrEBrNSIlbbBbJqB0C Xvbqhjd+VA6KrWHpNpNqJVVrAzGovUeRD7emQV4nfjOF7BrUvA7MljybdOirg2QNVR+G cFzngT+bC7hG7ORgGbaPjxyd0LOMQE4p5cg= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2t99btr0ps-16 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Fri, 21 Jun 2019 17:05:38 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::126) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Fri, 21 Jun 2019 17:05:23 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id A946E62E2D56; Fri, 21 Jun 2019 17:05:22 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v6 2/6] filemap: update offset check in filemap_fault() Date: Fri, 21 Jun 2019 17:05:08 -0700 Message-ID: <20190622000512.923867-3-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190622000512.923867-1-songliubraving@fb.com> References: <20190622000512.923867-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-21_16:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=801 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906210182 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP With THP, current check of offset: VM_BUG_ON_PAGE(page->index != offset, page); is no longer accurate. Update it to: VM_BUG_ON_PAGE(page_to_pgoff(page) != offset, page); Acked-by: Rik van Riel Signed-off-by: Song Liu --- mm/filemap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/filemap.c b/mm/filemap.c index f5b79a43946d..5f072a113535 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2522,7 +2522,7 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) put_page(page); goto retry_find; } - VM_BUG_ON_PAGE(page->index != offset, page); + VM_BUG_ON_PAGE(page_to_pgoff(page) != offset, page); /* * We have a locked page in the page cache, now we need to check From patchwork Sat Jun 22 00:05:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11011005 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4EF04924 for ; Sat, 22 Jun 2019 00:05:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E70A20008 for ; Sat, 22 Jun 2019 00:05:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 30FD02862A; Sat, 22 Jun 2019 00:05:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 84A2620008 for ; Sat, 22 Jun 2019 00:05:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5D2DC8E0003; Fri, 21 Jun 2019 20:05:29 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 583A78E0001; Fri, 21 Jun 2019 20:05:29 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 473248E0003; Fri, 21 Jun 2019 20:05:29 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by kanga.kvack.org (Postfix) with ESMTP id 13FD58E0001 for ; Fri, 21 Jun 2019 20:05:29 -0400 (EDT) Received: by mail-pl1-f199.google.com with SMTP id bb9so4489391plb.2 for ; Fri, 21 Jun 2019 17:05:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=DNX0Ai841ZgyA6yG9XH0uRWq8DS0N63a6b+fvROsjjM=; b=ciDdRXFNVhMFCMFlFDL0IcaQ8hYiCLoGRPGfJVSp+AcC53dFQjOOwvA+EZFSXUankC +CmV7p9x8IaEObSk2HUqLmC5R5OFnurGLTaZ5JHuVnKRlqSQMH65H0QvOa5Y6s9otPYQ DqwRWN9JykXQPElFar2CUDuaj+f9j5qwsevqP4CLQj3Jp0SMlCbREzFyQpflMByVzqqk nba63XU8IN+T7B7wbNDtg353s/FOFS+uKs5BdKc41dlb3BpZcXLpQ35U/TMZiuGRCP7+ rueb2v1+7dyHZGax3VuSEvTuJB5vPuv2Q4zTYzY8/+yWaSJMK4m19wK+hacIujg3osn7 oh2A== X-Gm-Message-State: APjAAAX+BhQ2OrpJKfjokC2TT126+ctgE7An2xufyc0do8YbtjygLkU0 P3z7dieKfQ8v0BHIS91JRVCyn2lXrurjZcuKywqRieQX1zH1n7znd+HBpnDevcB8HX0+b0LGUC5 ipc8xvLoX5dnmZ+t1nqhvRXDAW9jSDArIAa2M8sr45MItSHkY5Kh3IYScoVqsThcz0w== X-Received: by 2002:a63:a046:: with SMTP id u6mr5318022pgn.122.1561161928684; Fri, 21 Jun 2019 17:05:28 -0700 (PDT) X-Google-Smtp-Source: APXvYqybA9LUzm25jN2mrlmDInqCmERJVt20oiU/1A0km+84EXHu/Uax20tLxCYZClALeEIlzc8W X-Received: by 2002:a63:a046:: with SMTP id u6mr5317956pgn.122.1561161927828; Fri, 21 Jun 2019 17:05:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561161927; cv=none; d=google.com; s=arc-20160816; b=nPxxWc5d2fKs/Prc9ZS2siLzW2C3jlMwi00f67iw8OS0f/HRv+XWX2d9dP2vdWN93f 8D5jswEJDQXQheSPzgcEtPdVL1wNJ+ArDtTJWvr8KkJewFicjqnLZbaN4FkA757OEz3z lCowqTVUkZPwPZd3uMJVqAjIl1xf5KOCQwXCzrGpOWOg7O8pGv589KqpQi7vK9tmfzuW Z0+yVnSyMWpwa8AqutxCEWeaNbhJQvKLtqtUvPvfQwWku4HrENd7EjmzTqFZpnYd5Jjr F+4ZhXWFbm8NTBzoTE8Pmpp2evf4zlcyLboGzj6STTRx4V0NXLBC1DP0SWRirag0j9qm SBXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=DNX0Ai841ZgyA6yG9XH0uRWq8DS0N63a6b+fvROsjjM=; b=l5LJjvnyLKmZanBe0rB7VoPGS4WvGNsjGUqKArSfiZWdWHQXnZ9M6vcBlTCZ7rg761 DX5gKHvllDihrG3o3IQlIFlRxLnBaRNkHJFKnp+S+SRNbjd1iwMGWtTId/L3405yRlXp vIrMTyJnyiciOp5yE/aRfSeePJe5EMV7v4w8PvxJBGLnx1JslUW3Q2dXTeTZxL535VjL P0zf5SsGh4zPOmBjsIVPcTq/Rg5DFeR4rnR6f+MMp3tgwTw7lLXV75pSssZAYsYMmr6y FJGJ0F9sAYQRCwKYdeRjWzOh+6pzr4O66AslLyQ9o1M0TzyM+kykfz2xx18jobMgKjSd dzwQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b="YCqTrT/J"; spf=pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1076a8f7d5=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com. [67.231.145.42]) by mx.google.com with ESMTPS id s36si3977093pld.164.2019.06.21.17.05.27 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Jun 2019 17:05:27 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.145.42 as permitted sender) client-ip=67.231.145.42; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b="YCqTrT/J"; spf=pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1076a8f7d5=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0044008.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5LNudUt026475 for ; Fri, 21 Jun 2019 17:05:27 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=DNX0Ai841ZgyA6yG9XH0uRWq8DS0N63a6b+fvROsjjM=; b=YCqTrT/JYASHpQEOOYkGYg92c16QN3f691f37DMGfg7VNFrKhjAdtGuCBrvVxwNr09w6 tohmnJ9WfVpElA6TQNfZEgq+itS38qXPwRjZ2zVNgQqSCnX5WEYxVtIX2JEJz15kdQua U1qRg8bLRHjdvvworvkgyXymApb4yBziE8Y= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 2t99btr0pc-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 21 Jun 2019 17:05:27 -0700 Received: from mx-out.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Fri, 21 Jun 2019 17:05:25 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id DCC9D62E2D56; Fri, 21 Jun 2019 17:05:24 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v6 3/6] mm,thp: stats for file backed THP Date: Fri, 21 Jun 2019 17:05:09 -0700 Message-ID: <20190622000512.923867-4-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190622000512.923867-1-songliubraving@fb.com> References: <20190622000512.923867-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-21_16:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906210182 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP In preparation for non-shmem THP, this patch adds a few stats and exposes them in /proc/meminfo, /sys/bus/node/devices//meminfo, and /proc//task//smaps. This patch is mostly a rewrite of Kirill A. Shutemov's earlier version: https://lkml.org/lkml/2017/1/26/284. Acked-by: Rik van Riel Signed-off-by: Song Liu --- drivers/base/node.c | 6 ++++++ fs/proc/meminfo.c | 4 ++++ fs/proc/task_mmu.c | 4 +++- include/linux/mmzone.h | 2 ++ mm/vmstat.c | 2 ++ 5 files changed, 17 insertions(+), 1 deletion(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 8598fcbd2a17..71ae2dc93489 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -426,6 +426,8 @@ static ssize_t node_read_meminfo(struct device *dev, "Node %d AnonHugePages: %8lu kB\n" "Node %d ShmemHugePages: %8lu kB\n" "Node %d ShmemPmdMapped: %8lu kB\n" + "Node %d FileHugePages: %8lu kB\n" + "Node %d FilePmdMapped: %8lu kB\n" #endif , nid, K(node_page_state(pgdat, NR_FILE_DIRTY)), @@ -451,6 +453,10 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * + HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_FILE_THPS) * + HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) #endif ); diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 568d90e17c17..bac395fc11f9 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -136,6 +136,10 @@ static int meminfo_proc_show(struct seq_file *m, void *v) global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); + show_val_kb(m, "FileHugePages: ", + global_node_page_state(NR_FILE_THPS) * HPAGE_PMD_NR); + show_val_kb(m, "FilePmdMapped: ", + global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); #endif #ifdef CONFIG_CMA diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 01d4eb0e6bd1..0360e3b2ba89 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -413,6 +413,7 @@ struct mem_size_stats { unsigned long lazyfree; unsigned long anonymous_thp; unsigned long shmem_thp; + unsigned long file_thp; unsigned long swap; unsigned long shared_hugetlb; unsigned long private_hugetlb; @@ -563,7 +564,7 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr, else if (is_zone_device_page(page)) /* pass */; else - VM_BUG_ON_PAGE(1, page); + mss->file_thp += HPAGE_PMD_SIZE; smaps_account(mss, page, true, pmd_young(*pmd), pmd_dirty(*pmd), locked); } #else @@ -767,6 +768,7 @@ static void __show_smap(struct seq_file *m, const struct mem_size_stats *mss) SEQ_PUT_DEC(" kB\nLazyFree: ", mss->lazyfree); SEQ_PUT_DEC(" kB\nAnonHugePages: ", mss->anonymous_thp); SEQ_PUT_DEC(" kB\nShmemPmdMapped: ", mss->shmem_thp); + SEQ_PUT_DEC(" kB\nFilePmdMapped: ", mss->file_thp); SEQ_PUT_DEC(" kB\nShared_Hugetlb: ", mss->shared_hugetlb); seq_put_decimal_ull_width(m, " kB\nPrivate_Hugetlb: ", mss->private_hugetlb >> 10, 7); diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 70394cabaf4e..827f9b777938 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -234,6 +234,8 @@ enum node_stat_item { NR_SHMEM, /* shmem pages (included tmpfs/GEM pages) */ NR_SHMEM_THPS, NR_SHMEM_PMDMAPPED, + NR_FILE_THPS, + NR_FILE_PMDMAPPED, NR_ANON_THPS, NR_UNSTABLE_NFS, /* NFS unstable pages */ NR_VMSCAN_WRITE, diff --git a/mm/vmstat.c b/mm/vmstat.c index fd7e16ca6996..6afc892a148a 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1158,6 +1158,8 @@ const char * const vmstat_text[] = { "nr_shmem", "nr_shmem_hugepages", "nr_shmem_pmdmapped", + "nr_file_hugepages", + "nr_file_pmdmapped", "nr_anon_transparent_hugepages", "nr_unstable", "nr_vmscan_write", From patchwork Sat Jun 22 00:05:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11011037 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F3E8276 for ; Sat, 22 Jun 2019 00:06:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E76E728B7B for ; Sat, 22 Jun 2019 00:06:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D8F1C28BB1; Sat, 22 Jun 2019 00:06:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6C26B28B7B for ; Sat, 22 Jun 2019 00:06:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 81DCB8E0008; Fri, 21 Jun 2019 20:06:05 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 77EFC8E0001; Fri, 21 Jun 2019 20:06:05 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 66E7D8E0008; Fri, 21 Jun 2019 20:06:05 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id 2EB248E0001 for ; Fri, 21 Jun 2019 20:06:05 -0400 (EDT) Received: by mail-pf1-f200.google.com with SMTP id a125so5318168pfa.13 for ; Fri, 21 Jun 2019 17:06:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=n2snFM7QmmPTPpShlBhEnsGKC2i2/uKCmCMTseCu52k=; b=QksPa2C9reR3caIS1v1fiXIJDHiL4/jBvVnRW0wPxN5xY8amzWoAbGUKt2QzRaB961 zK+8RjSxoKudi0J0PP942T0rT5eIv5muhm0VllbPz4KXMP+AJ4MHwBIZ/p/R17P31L7s G7ZJhHMSc4BVbCoeR+1zF1qJfd9nEfWdiYElXGKSlEgi0TJ8m98dkSLdgref4Zs0FHrN KnhXIWggPn91C6/mv2DI1Lq9aRnJHhnHyHwFHLT6FOfWAEaZWZi7gepfjaDAbXsBivcz wsS5PeLlj3C91iIScF/+2nstH+8A+8ef5WVwqckAQS9yohHiTnir1Bv0OUw7v27GZNQt zwZA== X-Gm-Message-State: APjAAAXS1gt+ke+oKfd+t8Ky6icMFYZajShExGmGsyWf+91dswJ/1RXg 3crwzAh/lj46/fEXExPJPuvooGSHkwriqCW9ihxZe+ZZfOJqk9KDKXSnme5cR9v9PqJeEEJsVgZ imoNlDvCPL8evbZaEBzJXR6axmyhFLBQm5rQUWluzSuRLb6wSrKzdobkl+Ik6pU+XnQ== X-Received: by 2002:a17:90a:9b8a:: with SMTP id g10mr9745439pjp.66.1561161964846; Fri, 21 Jun 2019 17:06:04 -0700 (PDT) X-Google-Smtp-Source: APXvYqxrVy1yCyj8WZrvovOhgQLga3HuVLAcj/CqSlSWYXGlhbPJ4vtEO2VdIskEGalWM4ulvxWW X-Received: by 2002:a17:90a:9b8a:: with SMTP id g10mr9745384pjp.66.1561161964218; Fri, 21 Jun 2019 17:06:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561161964; cv=none; d=google.com; s=arc-20160816; b=zUPYZiM8x4jo6cSFZgGOU51aXPiaWYKUJAMBqVie+DqkuNMlJMgAR6jaeuPW5mvdp7 9R0EZsK8H7frHhYb2qry7FMnQ5v38nP3ymuNPJ6KARq0JUEUqTErpGdKA48JAucz0sTW 7sNTAlJGEdT2rhPaujH1tronv+zqOKmRTf1flX0hYcccicLYk5vfVobESwGWSK4A1nTu qqIt0eDfeRwCoKFHDqMZe/Www959slJssZZQ1wVcYq7DOp/kepUIIzEufe2JTBTl8Ohb NphKGEXR8rHYldmyGmSOXNjqcwjsBT9TD5yLXPCARx0KGze+fDNfHGC2SWpnPWZv9MON farQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=n2snFM7QmmPTPpShlBhEnsGKC2i2/uKCmCMTseCu52k=; b=AgHlEw9TGW1gRHxoQb7sET9sXltk7GDSwh8PqQYBsQvkernpYaLIyLgUhU8Xv2SYJi 9yRZUHVb+CEK7ask7IsBF2jN7pRTwjYbTyl/wFNXCql7yyVavuFB/qhMg1/qQeHGaiRh e/pFwA/2mHzFTVfbYBuqSpMUY8M5g72r8JxKO4voo5tK8U0woWwyxnyl2/LauCkqn0yi v67zk+EckvoRoR8g3rNu9BzHXDbGp610QdJ5k8tZ5SsMn9ic2OFeTR3FHvU9Nn1DoELi XXoWVgZ4DbTofkvH56bKjvpDmgq5GUk6w+X9rza6kkTFf4PuT0fMoRvXyPhKWzc6qnGJ qexQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=TkB3WEMk; spf=pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1076a8f7d5=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com. [67.231.145.42]) by mx.google.com with ESMTPS id f4si3881584pgo.216.2019.06.21.17.06.04 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Jun 2019 17:06:04 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.145.42 as permitted sender) client-ip=67.231.145.42; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=TkB3WEMk; spf=pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=1076a8f7d5=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0044012.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5LNt1fZ005696 for ; Fri, 21 Jun 2019 17:06:03 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=n2snFM7QmmPTPpShlBhEnsGKC2i2/uKCmCMTseCu52k=; b=TkB3WEMksGPkiSgJMtbOi5nd60Vn9LkBFGWBLqllbfRHKqjMvyN9xgpjLMSBxiNsT59u xXSjznVWMZNCtWyTtBnc2LUl/YYmfYW56N4xJiuIsFr+y6Qe5ZFFAOb3Jro0mEberPQs hJx7W5AQaf95gqEbevnn2c0lR2ZmiRAx+Bc= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2t90mjj1eg-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Fri, 21 Jun 2019 17:06:03 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::129) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Fri, 21 Jun 2019 17:05:27 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 1BB8B62E2D56; Fri, 21 Jun 2019 17:05:27 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v6 4/6] khugepaged: rename collapse_shmem() and khugepaged_scan_shmem() Date: Fri, 21 Jun 2019 17:05:10 -0700 Message-ID: <20190622000512.923867-5-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190622000512.923867-1-songliubraving@fb.com> References: <20190622000512.923867-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-21_16:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=563 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906210182 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Next patch will add khugepaged support of non-shmem files. This patch renames these two functions to reflect the new functionality: collapse_shmem() => collapse_file() khugepaged_scan_shmem() => khugepaged_scan_file() Acked-by: Rik van Riel Signed-off-by: Song Liu --- mm/khugepaged.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 0f7419938008..dde8e45552b3 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1287,7 +1287,7 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) } /** - * collapse_shmem - collapse small tmpfs/shmem pages into huge one. + * collapse_file - collapse small tmpfs/shmem pages into huge one. * * Basic scheme is simple, details are more complex: * - allocate and lock a new huge page; @@ -1304,10 +1304,11 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) * + restore gaps in the page cache; * + unlock and free huge page; */ -static void collapse_shmem(struct mm_struct *mm, +static void collapse_file(struct vm_area_struct *vma, struct address_space *mapping, pgoff_t start, struct page **hpage, int node) { + struct mm_struct *mm = vma->vm_mm; gfp_t gfp; struct page *new_page; struct mem_cgroup *memcg; @@ -1563,7 +1564,7 @@ static void collapse_shmem(struct mm_struct *mm, /* TODO: tracepoints */ } -static void khugepaged_scan_shmem(struct mm_struct *mm, +static void khugepaged_scan_file(struct vm_area_struct *vma, struct address_space *mapping, pgoff_t start, struct page **hpage) { @@ -1631,14 +1632,14 @@ static void khugepaged_scan_shmem(struct mm_struct *mm, result = SCAN_EXCEED_NONE_PTE; } else { node = khugepaged_find_target_node(); - collapse_shmem(mm, mapping, start, hpage, node); + collapse_file(vma, mapping, start, hpage, node); } } /* TODO: tracepoints */ } #else -static void khugepaged_scan_shmem(struct mm_struct *mm, +static void khugepaged_scan_file(struct vm_area_struct *vma, struct address_space *mapping, pgoff_t start, struct page **hpage) { @@ -1722,7 +1723,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, file = get_file(vma->vm_file); up_read(&mm->mmap_sem); ret = 1; - khugepaged_scan_shmem(mm, file->f_mapping, + khugepaged_scan_file(vma, file->f_mapping, pgoff, hpage); fput(file); } else { From patchwork Sat Jun 22 00:05:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11011007 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C83B5924 for ; Sat, 22 Jun 2019 00:05:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B94F620008 for ; Sat, 22 Jun 2019 00:05:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AB9C42862A; Sat, 22 Jun 2019 00:05:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C427120008 for ; Sat, 22 Jun 2019 00:05:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 843F88E0005; Fri, 21 Jun 2019 20:05:32 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7F5F68E0001; Fri, 21 Jun 2019 20:05:32 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70AFA8E0005; Fri, 21 Jun 2019 20:05:32 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yw1-f69.google.com (mail-yw1-f69.google.com [209.85.161.69]) by kanga.kvack.org (Postfix) with ESMTP id 507838E0001 for ; Fri, 21 Jun 2019 20:05:32 -0400 (EDT) Received: by mail-yw1-f69.google.com with SMTP id p18so8040008ywe.17 for ; Fri, 21 Jun 2019 17:05:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=IWV65r5U1l3yQ4pVTe+jKPIu04iDbUqXRbQ4VzDCPdU=; b=K9hSRhlHFY8E8XNRosyUeE5dreAGyAi7ZIc5gotYWGlzUh3fGYw6ZEbZqZm5fgj1nL rWNqTQ9wIml7x2gbV13a9gnl6LWBRp7dwcnTZQEFGYox0adNn+wQkPXt0zv4olBaTaXI PFktBcMy/a/Bj+xi03bca5KkmlQJtqDWNzL6+0a1NFIXoHyCcNReVxH3j3q4kOLkWCYm JQlcNJpPyXBGZQC+eNhC7Nqa1K8zeAAo9UTrorqAXHWXFm7aWSdvhsoSJ7MXpJqBosFT ioinVFokwGFSNTMLxQ60xeCqJrvry+VU373Yv1Om1poCM0shpo1KROLj0z/4vYVwUpzt AQkw== X-Gm-Message-State: APjAAAW5gLkm8IshoUGV9+4QHebAeyxGOZ5m//ZAC+ii5FjzUMPP9G9e oD4mE9hukKlGeSQSCOP9ZvLanxlE+ODNpcCMAgewHUqg5y64I1lEOWKD6TMW8mmw9CCczGNHwza TB+jX5whm3Mn3vy/WKCe102v0ih4TwnU+qUwFzKm6rz4subD2d/vWUkELGvswUc3Bvg== X-Received: by 2002:a81:1153:: with SMTP id 80mr17267387ywr.39.1561161932057; Fri, 21 Jun 2019 17:05:32 -0700 (PDT) X-Google-Smtp-Source: APXvYqzN5cedPvqgeYD4PhM9Rh9+aQqdRrq7/loAfy7oUQ+4E4xFLrQj5dSKXeSsE6rY6G8vfA9X X-Received: by 2002:a81:1153:: with SMTP id 80mr17267341ywr.39.1561161931176; Fri, 21 Jun 2019 17:05:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561161931; cv=none; d=google.com; s=arc-20160816; b=bTGd/p30lDB+LHz/gNH2e8coz52DXT6/bLji7qBDhD4Mbs3Ci+2NRwd93i+zAhK+Qg kgGhem1BlN+Ol1euM7ZFRzswFvk1Q50XBDWpgFKfcBWdLc0E95XSU/oy9X7ou+P2CN9J SUP9+O6r2vukUz4QCcnbN1U5y7PPsWsJ8CHxWu9vLCCwlXv8tezxOup/aAhyxZsWJdwz oQ4iLqkjTcTk8qqLCRS3u/m6Wxb/U7PpPY39kYjQYVJTZX5YBRVLW+vjpQUdLwddYaRj 7HI632tfqXq/6ILpDh6mEXVA0K9itg1XP4eNWLZQOi2OUCuTze/jaWlBRLN075Q1g+tK G//g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=IWV65r5U1l3yQ4pVTe+jKPIu04iDbUqXRbQ4VzDCPdU=; b=rsiVyd8DzQzqjQlCoQSRTx7xwZClq5HK9l0LNtbKZol0IHszAYtL7mymqoZ3S+8May fimhGgJ/mbQDMaS/M6Kk/qrY5x1OXXv+WIEikvtzRXvtLZ/WdxpJd1EmzyRY+N2LW478 gbch81R7Zs3MIygz3zWxAmdaO2RodMuHjFg8qjBSgkA91VQ6ytL5aUctSxiobJcBKJ7K VEjzAilg4x9HPmo042h9n33TBeO5GtYh+SurhjdVY4LJouxiKA4QL6Ljw3uttCE0MO8N FcOT32wxhqLxriGAktWDNeOszPALrKSNKnupNpH+Ay5aDBDn/hkljiOZsKyEClWaylPi AM6A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=kxf8MJM9; spf=pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1076a8f7d5=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id o6si1484421ybk.138.2019.06.21.17.05.31 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Jun 2019 17:05:31 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=kxf8MJM9; spf=pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1076a8f7d5=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.16.0.27/8.16.0.27) with SMTP id x5LNqVlf029137 for ; Fri, 21 Jun 2019 17:05:31 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=IWV65r5U1l3yQ4pVTe+jKPIu04iDbUqXRbQ4VzDCPdU=; b=kxf8MJM9a4RH8AW/c2NAHkpyDFMaiI5wyyll6HN4NpTI/rFcowgVM8Ntx5pmkOA3HA7h gdDMqg6elkrpPdhnmZsSOYtmut/7JeewSYDBL8Csk7GZgD0HuqEdB6x50foKxYNxN5tX cr2YN8BBnLhlXOoCCgmrVt2f1NQRdE2tBB4= Received: from maileast.thefacebook.com ([163.114.130.16]) by m0001303.ppops.net with ESMTP id 2t97gegbwx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 21 Jun 2019 17:05:30 -0700 Received: from mx-out.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Fri, 21 Jun 2019 17:05:30 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 5593B62E2D56; Fri, 21 Jun 2019 17:05:29 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v6 5/6] mm,thp: add read-only THP support for (non-shmem) FS Date: Fri, 21 Jun 2019 17:05:11 -0700 Message-ID: <20190622000512.923867-6-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190622000512.923867-1-songliubraving@fb.com> References: <20190622000512.923867-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-21_16:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906210182 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch is (hopefully) the first step to enable THP for non-shmem filesystems. This patch enables an application to put part of its text sections to THP via madvise, for example: madvise((void *)0x600000, 0x200000, MADV_HUGEPAGE); We tried to reuse the logic for THP on tmpfs. Currently, write is not supported for non-shmem THP. khugepaged will only process vma with VM_DENYWRITE. The next patch will handle writes, which would only happen when the vma with VM_DENYWRITE is unmapped. An EXPERIMENTAL config, READ_ONLY_THP_FOR_FS, is added to gate this feature. Acked-by: Rik van Riel Signed-off-by: Song Liu --- mm/Kconfig | 11 ++++++ mm/filemap.c | 4 +-- mm/khugepaged.c | 91 +++++++++++++++++++++++++++++++++++++++++-------- mm/rmap.c | 12 ++++--- 4 files changed, 97 insertions(+), 21 deletions(-) diff --git a/mm/Kconfig b/mm/Kconfig index f0c76ba47695..0a8fd589406d 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -762,6 +762,17 @@ config GUP_BENCHMARK See tools/testing/selftests/vm/gup_benchmark.c +config READ_ONLY_THP_FOR_FS + bool "Read-only THP for filesystems (EXPERIMENTAL)" + depends on TRANSPARENT_HUGE_PAGECACHE && SHMEM + + help + Allow khugepaged to put read-only file-backed pages in THP. + + This is marked experimental because it is a new feature. Write + support of file THPs will be developed in the next few release + cycles. + config ARCH_HAS_PTE_SPECIAL bool diff --git a/mm/filemap.c b/mm/filemap.c index 5f072a113535..e79ceccdc6df 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -203,8 +203,8 @@ static void unaccount_page_cache_page(struct address_space *mapping, __mod_node_page_state(page_pgdat(page), NR_SHMEM, -nr); if (PageTransHuge(page)) __dec_node_page_state(page, NR_SHMEM_THPS); - } else { - VM_BUG_ON_PAGE(PageTransHuge(page), page); + } else if (PageTransHuge(page)) { + __dec_node_page_state(page, NR_FILE_THPS); } /* diff --git a/mm/khugepaged.c b/mm/khugepaged.c index dde8e45552b3..fbcff5a1d65a 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -48,6 +48,7 @@ enum scan_result { SCAN_CGROUP_CHARGE_FAIL, SCAN_EXCEED_SWAP_PTE, SCAN_TRUNCATED, + SCAN_PAGE_HAS_PRIVATE, }; #define CREATE_TRACE_POINTS @@ -404,7 +405,11 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, (vm_flags & VM_NOHUGEPAGE) || test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) return false; - if (shmem_file(vma->vm_file)) { + + if (shmem_file(vma->vm_file) || + (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && + vma->vm_file && + (vm_flags & VM_DENYWRITE))) { if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGE_PAGECACHE)) return false; return IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, @@ -456,8 +461,9 @@ int khugepaged_enter_vma_merge(struct vm_area_struct *vma, unsigned long hstart, hend; /* - * khugepaged does not yet work on non-shmem files or special - * mappings. And file-private shmem THP is not supported. + * khugepaged only supports read-only files for non-shmem files. + * khugepaged does not yet work on special mappings. And + * file-private shmem THP is not supported. */ if (!hugepage_vma_check(vma, vm_flags)) return 0; @@ -1287,12 +1293,12 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) } /** - * collapse_file - collapse small tmpfs/shmem pages into huge one. + * collapse_file - collapse filemap/tmpfs/shmem pages into huge one. * * Basic scheme is simple, details are more complex: * - allocate and lock a new huge page; * - scan page cache replacing old pages with the new one - * + swap in pages if necessary; + * + swap/gup in pages if necessary; * + fill in gaps; * + keep old pages around in case rollback is required; * - if replacing succeeds: @@ -1316,7 +1322,11 @@ static void collapse_file(struct vm_area_struct *vma, LIST_HEAD(pagelist); XA_STATE_ORDER(xas, &mapping->i_pages, start, HPAGE_PMD_ORDER); int nr_none = 0, result = SCAN_SUCCEED; + bool is_shmem = shmem_file(vma->vm_file); +#ifndef CONFIG_READ_ONLY_THP_FOR_FS + VM_BUG_ON(!is_shmem); +#endif VM_BUG_ON(start & (HPAGE_PMD_NR - 1)); /* Only allocate from the target node */ @@ -1348,7 +1358,8 @@ static void collapse_file(struct vm_area_struct *vma, } while (1); __SetPageLocked(new_page); - __SetPageSwapBacked(new_page); + if (is_shmem) + __SetPageSwapBacked(new_page); new_page->index = start; new_page->mapping = mapping; @@ -1363,7 +1374,7 @@ static void collapse_file(struct vm_area_struct *vma, struct page *page = xas_next(&xas); VM_BUG_ON(index != xas.xa_index); - if (!page) { + if (is_shmem && !page) { /* * Stop if extent has been truncated or hole-punched, * and is now completely empty. @@ -1384,7 +1395,7 @@ static void collapse_file(struct vm_area_struct *vma, continue; } - if (xa_is_value(page) || !PageUptodate(page)) { + if (is_shmem && (xa_is_value(page) || !PageUptodate(page))) { xas_unlock_irq(&xas); /* swap in or instantiate fallocated page */ if (shmem_getpage(mapping->host, index, &page, @@ -1392,6 +1403,24 @@ static void collapse_file(struct vm_area_struct *vma, result = SCAN_FAIL; goto xa_unlocked; } + } else if (!page || xa_is_value(page)) { + unsigned long vaddr; + + VM_BUG_ON(is_shmem); + + vaddr = vma->vm_start + + ((index - vma->vm_pgoff) << PAGE_SHIFT); + xas_unlock_irq(&xas); + if (get_user_pages_remote(NULL, mm, vaddr, 1, + FOLL_FORCE, &page, NULL, NULL) != 1) { + result = SCAN_FAIL; + goto xa_unlocked; + } + lru_add_drain(); + lock_page(page); + } else if (!PageUptodate(page) || PageDirty(page)) { + result = SCAN_FAIL; + goto xa_locked; } else if (trylock_page(page)) { get_page(page); xas_unlock_irq(&xas); @@ -1426,6 +1455,12 @@ static void collapse_file(struct vm_area_struct *vma, goto out_unlock; } + if (page_has_private(page) && + !try_to_release_page(page, GFP_KERNEL)) { + result = SCAN_PAGE_HAS_PRIVATE; + break; + } + if (page_mapped(page)) unmap_mapping_pages(mapping, index, 1, false); @@ -1463,12 +1498,18 @@ static void collapse_file(struct vm_area_struct *vma, goto xa_unlocked; } - __inc_node_page_state(new_page, NR_SHMEM_THPS); + if (is_shmem) + __inc_node_page_state(new_page, NR_SHMEM_THPS); + else + __inc_node_page_state(new_page, NR_FILE_THPS); + if (nr_none) { struct zone *zone = page_zone(new_page); __mod_node_page_state(zone->zone_pgdat, NR_FILE_PAGES, nr_none); - __mod_node_page_state(zone->zone_pgdat, NR_SHMEM, nr_none); + if (is_shmem) + __mod_node_page_state(zone->zone_pgdat, + NR_SHMEM, nr_none); } xa_locked: @@ -1506,10 +1547,15 @@ static void collapse_file(struct vm_area_struct *vma, SetPageUptodate(new_page); page_ref_add(new_page, HPAGE_PMD_NR - 1); - set_page_dirty(new_page); mem_cgroup_commit_charge(new_page, memcg, false, true); + + if (is_shmem) { + set_page_dirty(new_page); + lru_cache_add_anon(new_page); + } else { + lru_cache_add_file(new_page); + } count_memcg_events(memcg, THP_COLLAPSE_ALLOC, 1); - lru_cache_add_anon(new_page); /* * Remove pte page tables, so we can re-fault the page as huge. @@ -1524,7 +1570,9 @@ static void collapse_file(struct vm_area_struct *vma, /* Something went wrong: roll back page cache changes */ xas_lock_irq(&xas); mapping->nrpages -= nr_none; - shmem_uncharge(mapping->host, nr_none); + + if (is_shmem) + shmem_uncharge(mapping->host, nr_none); xas_set(&xas, start); xas_for_each(&xas, page, end - 1) { @@ -1607,6 +1655,17 @@ static void khugepaged_scan_file(struct vm_area_struct *vma, break; } + if (page_has_private(page) && trylock_page(page)) { + int ret; + + ret = try_to_release_page(page, GFP_KERNEL); + unlock_page(page); + if (!ret) { + result = SCAN_PAGE_HAS_PRIVATE; + break; + } + } + if (page_count(page) != 1 + page_mapcount(page)) { result = SCAN_PAGE_COUNT; break; @@ -1714,11 +1773,13 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, VM_BUG_ON(khugepaged_scan.address < hstart || khugepaged_scan.address + HPAGE_PMD_SIZE > hend); - if (shmem_file(vma->vm_file)) { + if (vma->vm_file) { struct file *file; pgoff_t pgoff = linear_page_index(vma, khugepaged_scan.address); - if (!shmem_huge_enabled(vma)) + + if (shmem_file(vma->vm_file) + && !shmem_huge_enabled(vma)) goto skip; file = get_file(vma->vm_file); up_read(&mm->mmap_sem); diff --git a/mm/rmap.c b/mm/rmap.c index e5dfe2ae6b0d..87cfa2c19eda 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1192,8 +1192,10 @@ void page_add_file_rmap(struct page *page, bool compound) } if (!atomic_inc_and_test(compound_mapcount_ptr(page))) goto out; - VM_BUG_ON_PAGE(!PageSwapBacked(page), page); - __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + if (PageSwapBacked(page)) + __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + else + __inc_node_page_state(page, NR_FILE_PMDMAPPED); } else { if (PageTransCompound(page) && page_mapping(page)) { VM_WARN_ON_ONCE(!PageLocked(page)); @@ -1232,8 +1234,10 @@ static void page_remove_file_rmap(struct page *page, bool compound) } if (!atomic_add_negative(-1, compound_mapcount_ptr(page))) goto out; - VM_BUG_ON_PAGE(!PageSwapBacked(page), page); - __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + if (PageSwapBacked(page)) + __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + else + __dec_node_page_state(page, NR_FILE_PMDMAPPED); } else { if (!atomic_add_negative(-1, &page->_mapcount)) goto out; From patchwork Sat Jun 22 00:05:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11011041 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7F66076 for ; Sat, 22 Jun 2019 00:06:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6F96B28B7B for ; Sat, 22 Jun 2019 00:06:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 63F4128BB1; Sat, 22 Jun 2019 00:06:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C69E528B7B for ; Sat, 22 Jun 2019 00:06:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B9F1B8E000A; Fri, 21 Jun 2019 20:06:11 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B013A8E0001; Fri, 21 Jun 2019 20:06:11 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 97AB48E000A; Fri, 21 Jun 2019 20:06:11 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yw1-f70.google.com (mail-yw1-f70.google.com [209.85.161.70]) by kanga.kvack.org (Postfix) with ESMTP id 6FC278E0001 for ; Fri, 21 Jun 2019 20:06:11 -0400 (EDT) Received: by mail-yw1-f70.google.com with SMTP id f11so8125010ywc.4 for ; Fri, 21 Jun 2019 17:06:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=G4U2JA+ITVRUPcGtQR0tfoOVR3mjydybAs8QYLDNQp4=; b=MYDH0eR0p6pcOZIHSEP/9Gt3mseSfsYqjTBn35D6nyMVi80h5xP4metzM2qMkW1SI9 QDdJwbPxrrrBO8WTf4yi1vFZ5lT7RRRu4N08/+JdKh9655GzXEmHRYEmxixZtGpNScqx iwlG6IA77y3hWkpF7/c8z8qEmjfjPDeZpa/JFkD4Qf152Ak8jSVUJJ3JpMrpSGtQwkob 1s9E+oxiENGpzSZ7k3B2p/5j6vUNtCD/YJm949dtobfXhiLn163iSR9AT7FRaAKxVrO6 Hc0inwHSiKgKgNXJfinjaV2y8mHhwjqcPz7qYD0orm6gjDyBy2pUWVGEEjil9+uOGK/f D8VQ== X-Gm-Message-State: APjAAAXdhgWROME2k2nej3Vmoh63DKpzvsqH48sWbd3e71vOmV2YhftL 5iHo2bEXW95ohTwqAtjUkNh3OiI5sZ8Bt4ooaF+TRsVVec4xtRnjIfycrM/RbBeqzE0W4/DGo2O 2y32iu7XCMycL+Iny86mdxXYz18wuJSi2a4tUGMNcTdVF0F4vkBlvkhoszyB6NH50BA== X-Received: by 2002:a81:708d:: with SMTP id l135mr74952048ywc.225.1561161971202; Fri, 21 Jun 2019 17:06:11 -0700 (PDT) X-Google-Smtp-Source: APXvYqwB58sm0m+VXsxkWgbT2HbL7daLhVEMCcEGEtG2Dm7p4h54MeqDFlvBA4myoUrpMc8d4i5J X-Received: by 2002:a81:708d:: with SMTP id l135mr74952012ywc.225.1561161970510; Fri, 21 Jun 2019 17:06:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561161970; cv=none; d=google.com; s=arc-20160816; b=oN/OmmZdbjlcLy4lUuuEN26BeUP7iZ3sqDYVveD2I04LlI+qKrpBIZvs8QH8w/iWg/ pS8PEVm77byl/16Kl+Dw/6I0VGpg71oTc/vFjTUVY5eCMWRTdC7yg0yUba31X3i4pcKP iHfJ9tTcRhG1JuBMyYSavjGNq/7SEG9oH+gGuYcsdFckFhYmKgYD1eOskNYqwHsFNVJ+ Ggio7q4kyXPdDm/cS5eVOxK/0Dyjkqb4rGUkR9zQXT7oe6WfEWWMHpWozai1zVkHwM7W qaAktc1TPxKeHqKaetb1m3USKktJF0aJQVX6vyLCevkjmkOY01ncD50PLqOVaIl1DP8y OhLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=G4U2JA+ITVRUPcGtQR0tfoOVR3mjydybAs8QYLDNQp4=; b=eWmaYjswqF8Sh99oTqBJLAZ/IGd1kPXKfRu2U7yT4HZGL3wsVYWg597WM02Ayg5QWw 8znlsSvR5TTm55BJiwY6XkbZtxe9q33kmG9TLt+K/atuXN90igBzOfdRVgkt+PHJ9Xyt 6gUbODgakhPoMeiq88ygxiV7EhO4SH7zYwbxzufr2lbuxLmEZ+EuKQB0acWqO8ztbQ1U JCmCHCJ115dEUd8kV7SayIulOSTAI9Czi5btvupOfLgVDiChScUrYU2y/UXDLIh1dU+w 0Zo2RXcVbAuFhANeylrtanQ9P4ARqPi6TcywcdO82rZNrYiFFLcuYMobQ3t+49i0rmzH 785w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=ov9Wys35; spf=pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1076a8f7d5=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id 5si1605540ybb.383.2019.06.21.17.06.10 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Jun 2019 17:06:10 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=ov9Wys35; spf=pass (google.com: domain of prvs=1076a8f7d5=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=1076a8f7d5=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0089730.ppops.net [127.0.0.1]) by m0089730.ppops.net (8.16.0.27/8.16.0.27) with SMTP id x5LNsBEA018724 for ; Fri, 21 Jun 2019 17:06:10 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=G4U2JA+ITVRUPcGtQR0tfoOVR3mjydybAs8QYLDNQp4=; b=ov9Wys356HeJMTOhjJyvf09iU61MVwV9DlIOXWcRwc4uMSJ71zbAfKOLG/Y1zklxUweo toEHE/vDQB4xFKuqzRAs96BzcTLaH1DHXQLBuiFZ/ppm7PZuf0IvT1BG/maN2sll9JN/ pIq6XLGME0LZezrk6NQv3iOiSQSDN0oQNlQ= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by m0089730.ppops.net with ESMTP id 2t936x1a2f-14 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Fri, 21 Jun 2019 17:06:10 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::129) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Fri, 21 Jun 2019 17:05:32 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id A698162E2D56; Fri, 21 Jun 2019 17:05:31 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v6 6/6] mm,thp: avoid writes to file with THP in pagecache Date: Fri, 21 Jun 2019 17:05:12 -0700 Message-ID: <20190622000512.923867-7-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190622000512.923867-1-songliubraving@fb.com> References: <20190622000512.923867-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-06-21_16:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=608 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1906210182 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP In previous patch, an application could put part of its text section in THP via madvise(). These THPs will be protected from writes when the application is still running (TXTBSY). However, after the application exits, the file is available for writes. This patch avoids writes to file THP by dropping page cache for the file when the file is open for write. A new counter nr_thps is added to struct address_space. In do_last(), if the file is open for write and nr_thps is non-zero, we drop page cache for the whole file. Signed-off-by: Song Liu --- fs/inode.c | 3 +++ fs/namei.c | 22 +++++++++++++++++++++- include/linux/fs.h | 31 +++++++++++++++++++++++++++++++ mm/filemap.c | 1 + mm/khugepaged.c | 4 +++- 5 files changed, 59 insertions(+), 2 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index df6542ec3b88..518113a4e219 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -181,6 +181,9 @@ int inode_init_always(struct super_block *sb, struct inode *inode) mapping->flags = 0; mapping->wb_err = 0; atomic_set(&mapping->i_mmap_writable, 0); +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + atomic_set(&mapping->nr_thps, 0); +#endif mapping_set_gfp_mask(mapping, GFP_HIGHUSER_MOVABLE); mapping->private_data = NULL; mapping->writeback_index = 0; diff --git a/fs/namei.c b/fs/namei.c index 20831c2fbb34..de64f24b58e9 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -3249,6 +3249,22 @@ static int lookup_open(struct nameidata *nd, struct path *path, return error; } +/* + * The file is open for write, so it is not mmapped with VM_DENYWRITE. If + * it still has THP in page cache, drop the whole file from pagecache + * before processing writes. This helps us avoid handling write back of + * THP for now. + */ +static inline void release_file_thp(struct file *file) +{ +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + struct inode *inode = file_inode(file); + + if (inode_is_open_for_write(inode) && filemap_nr_thps(inode->i_mapping)) + truncate_pagecache(inode, 0); +#endif +} + /* * Handle the last step of open() */ @@ -3418,7 +3434,11 @@ static int do_last(struct nameidata *nd, goto out; opened: error = ima_file_check(file, op->acc_mode); - if (!error && will_truncate) + if (error) + goto out; + + release_file_thp(file); + if (will_truncate) error = handle_truncate(file); out: if (unlikely(error > 0)) { diff --git a/include/linux/fs.h b/include/linux/fs.h index f7fdfe93e25d..3edf4ee42eee 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -444,6 +444,10 @@ struct address_space { struct xarray i_pages; gfp_t gfp_mask; atomic_t i_mmap_writable; +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + /* number of thp, only for non-shmem files */ + atomic_t nr_thps; +#endif struct rb_root_cached i_mmap; struct rw_semaphore i_mmap_rwsem; unsigned long nrpages; @@ -2790,6 +2794,33 @@ static inline errseq_t filemap_sample_wb_err(struct address_space *mapping) return errseq_sample(&mapping->wb_err); } +static inline int filemap_nr_thps(struct address_space *mapping) +{ +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + return atomic_read(&mapping->nr_thps); +#else + return 0; +#endif +} + +static inline void filemap_nr_thps_inc(struct address_space *mapping) +{ +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + atomic_inc(&mapping->nr_thps); +#else + WARN_ON_ONCE(1); +#endif +} + +static inline void filemap_nr_thps_dec(struct address_space *mapping) +{ +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + atomic_dec(&mapping->nr_thps); +#else + WARN_ON_ONCE(1); +#endif +} + extern int vfs_fsync_range(struct file *file, loff_t start, loff_t end, int datasync); extern int vfs_fsync(struct file *file, int datasync); diff --git a/mm/filemap.c b/mm/filemap.c index e79ceccdc6df..a8e86c136381 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -205,6 +205,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, __dec_node_page_state(page, NR_SHMEM_THPS); } else if (PageTransHuge(page)) { __dec_node_page_state(page, NR_FILE_THPS); + filemap_nr_thps_dec(mapping); } /* diff --git a/mm/khugepaged.c b/mm/khugepaged.c index fbcff5a1d65a..17ebe9da56ce 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1500,8 +1500,10 @@ static void collapse_file(struct vm_area_struct *vma, if (is_shmem) __inc_node_page_state(new_page, NR_SHMEM_THPS); - else + else { __inc_node_page_state(new_page, NR_FILE_THPS); + filemap_nr_thps_inc(mapping); + } if (nr_none) { struct zone *zone = page_zone(new_page);