From patchwork Thu Aug 1 18:42:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11071249 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 666F2112C for ; Thu, 1 Aug 2019 18:43:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 579942873E for ; Thu, 1 Aug 2019 18:43:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4BADC28740; Thu, 1 Aug 2019 18:43:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1BB062873E for ; Thu, 1 Aug 2019 18:43:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 020AA6B0006; Thu, 1 Aug 2019 14:43:04 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F14216B0007; Thu, 1 Aug 2019 14:43:03 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E01AF6B0008; Thu, 1 Aug 2019 14:43:03 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yw1-f69.google.com (mail-yw1-f69.google.com [209.85.161.69]) by kanga.kvack.org (Postfix) with ESMTP id C14E96B0006 for ; Thu, 1 Aug 2019 14:43:03 -0400 (EDT) Received: by mail-yw1-f69.google.com with SMTP id e12so53216623ywe.6 for ; Thu, 01 Aug 2019 11:43:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=U9VgRs98GbU87xGfDbgwnxYoI5hIm/KB7jgPU3pbj0k=; b=dHkpAsaK4PRByflNztpatigRtLYKPx5+yhkDlZLKQMZwybffCNAdSXaTZQq5ERZB/Y 8sc+oPCSMPYORCj+HnzRz5RJskQO/W3phn31yFCUIg2XYlY9a8Bv2ggctmbP5VPrHlpG 9Oi2wDmiudlMNgO2O5/vt9VeDGQqLoeOlVxXLR+iuTwT74Pxats+ClGnEHXPh2FuqzXr duJn1jCH3nzMZa6zThNsRI8IYXBIn6ogUdn7lvyBkvAenAt4qA0Mtv0yio2ZIMghtV30 01xbIjSMByAXyAYFFzgWT23YdKbCYwnYdwSx1A1Thm0swNNeBE5xYzZTWzdJaefwnZTX 8inw== X-Gm-Message-State: APjAAAUTm3XNTdIYx1VLSAqXkmdUttz5+0hG9WRucxYYUORHmrD+MLR9 C89jirDCeF1pissw/Xu7ZG6UDK4lKCGdYhvT+Z2KF7BQualSwKK3pnoJ/TTCshPrC1Zt6YqmUPu y4KuCTWmghNWiaqm3e+71N2uOEV9iURAtmQSuZ4AW9j0ny4QEbBASAH/lK0uYD9Cx3A== X-Received: by 2002:a25:8392:: with SMTP id t18mr65366418ybk.161.1564684983547; Thu, 01 Aug 2019 11:43:03 -0700 (PDT) X-Google-Smtp-Source: APXvYqyxONNHGyYEZnJYNIf3xbrU45rQMmpdW9K3M53MkDaNE+i4Bn/uDKrC/Lb6gOmT0uSECpQV X-Received: by 2002:a25:8392:: with SMTP id t18mr65366392ybk.161.1564684983046; Thu, 01 Aug 2019 11:43:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1564684983; cv=none; d=google.com; s=arc-20160816; b=J4Gdo8JLIgO9cI3BmC+MJR9ZKm9Xp9U1LIFa6kZwkOBNfSPJBD80/ImroF5hokXjHQ L9g7vwD4PIz8q/GlzA2SWJmXWsasWnvU5emOX7d5r2JzZBFWDEVfBXDurRTt3ItYYngY NjJMJfu8VgKl90yWKNEV77wb5sLTrH9rdGgvOoWISgj/0pKK/zmTOESR45mQjankGHn8 1T5w3ChgAAScVeTQDxShyne8dIFaDRQeuVbJlRYBn23Awq1xbRgyjYNQG7C6PUmuEKoF NhVc6ayOeXCi8mBl1rf6ADqsJ6/39DuMIGijJ5PL87Ntm2bST8VFXGW4VaBToL12z/vQ 6F2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=U9VgRs98GbU87xGfDbgwnxYoI5hIm/KB7jgPU3pbj0k=; b=CcQkofqFmId8brUikkMwSa4ZF6ndmrG4fGGg87RuvtXkgDbd1UqlezcKbKi1qrhQKZ krQ5wknpjqnC22kbK1VwwEblHUSEI4AtoGJxvb6UBEwmvVGuev+WgKeQ6hS5IT47p/46 qHCwVKePIoQN8SLP9Abjs7ygVdR+TsSkYwMe/MNN0RMqcn+AhtVbtLh7hUgegs5FtlHr yKb2qFvatyqYG5QlJUi2abhY1sRFnZ7DkYLQuf6ZdPhJ5KQmCzz6Hzn2lSFdCT1nBz9D VrR0dcy0omDShnQKxQwk+kT1dCjZbVZMJrTcC6lQd/gkowZzWTdoxjna3Oln+XAnt5po hQZw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=h+0IcRmz; spf=pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=3116992784=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id p188si26681923ywg.204.2019.08.01.11.43.02 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Aug 2019 11:43:03 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=h+0IcRmz; spf=pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=3116992784=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.16.0.27/8.16.0.27) with SMTP id x71IfrtN019367 for ; Thu, 1 Aug 2019 11:43:02 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=U9VgRs98GbU87xGfDbgwnxYoI5hIm/KB7jgPU3pbj0k=; b=h+0IcRmzWSwWGwzzn7mE9a8id0EgPb+kaehHVQVuXTNWvHk7/bt94DL0hmDrKCw9wNzS r6KkPql6FPRNoVfypIR1Xb3b5z6qtpg3uKka+aiKZNoabIlKI0BOEVIWtZ29tBYmu6z5 F88sGWzHcdUkPwHSA2r0ms/6KdFB1uCCbpM= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by m0001303.ppops.net with ESMTP id 2u449ggauj-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 01 Aug 2019 11:43:02 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::127) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Thu, 1 Aug 2019 11:42:59 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id B26D962E1E18; Thu, 1 Aug 2019 11:42:56 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v10 1/7] filemap: check compound_head(page)->mapping in filemap_fault() Date: Thu, 1 Aug 2019 11:42:38 -0700 Message-ID: <20190801184244.3169074-2-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190801184244.3169074-1-songliubraving@fb.com> References: <20190801184244.3169074-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-08-01_08:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=884 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908010194 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Currently, filemap_fault() avoids race condition with truncate by checking page->mapping == mapping. This does not work for compound pages. This patch let it check compound_head(page)->mapping instead. Acked-by: Rik van Riel Acked-by: Kirill A. Shutemov Acked-by: Johannes Weiner Signed-off-by: Song Liu --- mm/filemap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/filemap.c b/mm/filemap.c index 7161fb937e78..d0bd9e585c2f 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2537,7 +2537,7 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) goto out_retry; /* Did it get truncated? */ - if (unlikely(page->mapping != mapping)) { + if (unlikely(compound_head(page)->mapping != mapping)) { unlock_page(page); put_page(page); goto retry_find; From patchwork Thu Aug 1 18:42:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11071253 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D242113B1 for ; Thu, 1 Aug 2019 18:43:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C4C0C2873E for ; Thu, 1 Aug 2019 18:43:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B909428740; Thu, 1 Aug 2019 18:43:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5A0462873E for ; Thu, 1 Aug 2019 18:43:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AA66A6B0007; Thu, 1 Aug 2019 14:43:04 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A591E6B0008; Thu, 1 Aug 2019 14:43:04 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F93C6B000A; Thu, 1 Aug 2019 14:43:04 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by kanga.kvack.org (Postfix) with ESMTP id 582366B0007 for ; Thu, 1 Aug 2019 14:43:04 -0400 (EDT) Received: by mail-pf1-f197.google.com with SMTP id q14so46347918pff.8 for ; Thu, 01 Aug 2019 11:43:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=lq7eKJkDsLmHUV4qu7iZTnjT46Y88VB+lwzmGkYti6U=; b=R16Kcfbaeu56iJJBsvYWsPQ/8CHOrGaYR0TIiO5jwcujn+Svu+KV6vo8OELCDKo4mz 8lgKVZduGzGxyI8EshsN6XxAFjRcz+VSgEgOPTfKEH4TCZvMwlgyrAP/4+d+381yMw+q pUQEfIWm69ZUIEnJDkoMKER67Cr2HzwPMW1GAfyM3CtXLoxHPnoCPttll3NLlADF8lLY 8UBD5GpV31rLw/ttFiT7UJOYd6sj39oI/rwiNABgCENBWsV2xhqM32kesFa361AwB1zv Lm89EZ/HQ06KJe9ykHPpnTnn5bQqlO6mPFHLtKJV0+28Q6LLE+VQ4wmU2vIFiPepM1WJ 748w== X-Gm-Message-State: APjAAAU3ba7OV6/kys1kErqLuIb4YXBNEJSlE+LU1Au22GvHMEKXqm6H eVkqTjpGb8ulBG8qt7FPtSWNvB65vX6gV/ayS1iuujfTeFMT/Scvct4x2+kFse20ejn0gGp/SWd TtKOM6msjxINlWDOh3q5aEZGCXUdxhdw58W4RNYlfWiJxNd+pMiKIL+NXBY8lXUv+MA== X-Received: by 2002:a62:8745:: with SMTP id i66mr54165889pfe.259.1564684983997; Thu, 01 Aug 2019 11:43:03 -0700 (PDT) X-Google-Smtp-Source: APXvYqzWl0u/op1TRLINOqRhuPhLhYFnuixW/FdKjAhUjIhFhjalpcM0cZH1zsu22DUplWrcuHLL X-Received: by 2002:a62:8745:: with SMTP id i66mr54165848pfe.259.1564684983414; Thu, 01 Aug 2019 11:43:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1564684983; cv=none; d=google.com; s=arc-20160816; b=S9vqf4eWTVDKx6Sf1jtWSRsmdmDNIuHdG4tkUvpE1i8bc7178qav4X7Jh4WVfzHj15 CypFRSxIOI0ztcmBx54wvZe8yG6UlsBek4M5KRZopWE7dATNV5QTO3i8Gfo3RsrNTsXP raR7SLeTE8dzLYFXJQPEfHdVwokI1T8Ya1+ZG3eIVFA55Xs1ubtZK9bbP2+r7NHymISg iYq+Pem2eFXxWPjpvHzsGTQ4SPYPGFAnEsMREgUs78IsFXUNyevPs/7dneEi5pgb8jqf yamsKx1fcJowXvNTNrH/f1q+WCX18bLXR0CMzBFbd4CCllzlEoB1Z9eUtCNZokPTPNVK A4uA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=lq7eKJkDsLmHUV4qu7iZTnjT46Y88VB+lwzmGkYti6U=; b=DtJhEQ/mIDA0ws4aeNHhuSoTnMzMO2Ra3DZ8HCsewiI4VymYBlAhOJSL/6vA5ZSpR0 Hf8Uu699BJ/mIQT2Y2PnY7dUDo7Q7kfXw75bI/Oru04sxMLpL4NjSaWx7tzuZhXbLxFW SVaGcPOzXZk5fbXJ2/T8khO5NEBKt42x5my5a416/TEw69WcGHAMhZCRb4BfuifSZDI4 5KdiMcb9LYxPvCz05GCa8NYJv/hf2q9HRH9gXJKvIxtFpr7W8e9KBkNGEWG9jP7jWTu9 OGRtVxoBU8gy61iuyZ9K0sQgB1MeTmmOzbJrsdy+4ciUxHTxvgIpMzRiVXZS2jXC7hFZ x0wQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=lpapXaVx; spf=pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=3116992784=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com. [67.231.145.42]) by mx.google.com with ESMTPS id v186si35417782pgd.358.2019.08.01.11.43.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Aug 2019 11:43:03 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.145.42 as permitted sender) client-ip=67.231.145.42; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=lpapXaVx; spf=pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=3116992784=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x71IgjR8026770 for ; Thu, 1 Aug 2019 11:43:02 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=lq7eKJkDsLmHUV4qu7iZTnjT46Y88VB+lwzmGkYti6U=; b=lpapXaVxfIKHC/ZsfavKbczUogVos6FhHhByK2rkJHhxRup9QMEqFYJsHh7DbKfuyc36 2vQssz0CJR1mpxFJ4VdxcLs884xwsE9DxoTuyHN9aqw5CzaB5pmyPJ+HQ/iMvR9ls3a/ qy7zJyo/6NAkNfntRYgboDezLa9r7q7eri8= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2u438rgmnr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 01 Aug 2019 11:43:02 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::125) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Thu, 1 Aug 2019 11:43:01 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 1961F62E1E18; Thu, 1 Aug 2019 11:43:01 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v10 2/7] filemap: check compound_head(page)->mapping in pagecache_get_page() Date: Thu, 1 Aug 2019 11:42:39 -0700 Message-ID: <20190801184244.3169074-3-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190801184244.3169074-1-songliubraving@fb.com> References: <20190801184244.3169074-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-08-01_08:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=750 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908010194 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Similar to previous patch, pagecache_get_page() avoids race condition with truncate by checking page->mapping == mapping. This does not work for compound pages. This patch let it check compound_head(page)->mapping instead. Suggested-by: Johannes Weiner Signed-off-by: Song Liu Acked-by: Johannes Weiner --- mm/filemap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/filemap.c b/mm/filemap.c index d0bd9e585c2f..aaee1ef96f6d 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1644,7 +1644,7 @@ struct page *pagecache_get_page(struct address_space *mapping, pgoff_t offset, } /* Has the page been truncated? */ - if (unlikely(page->mapping != mapping)) { + if (unlikely(compound_head(page)->mapping != mapping)) { unlock_page(page); put_page(page); goto repeat; From patchwork Thu Aug 1 18:42:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11071261 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2257313A4 for ; Thu, 1 Aug 2019 18:43:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1485D2873E for ; Thu, 1 Aug 2019 18:43:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 088EA28740; Thu, 1 Aug 2019 18:43:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 883D92873E for ; Thu, 1 Aug 2019 18:43:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 724CF6B000C; Thu, 1 Aug 2019 14:43:18 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6D4A06B000D; Thu, 1 Aug 2019 14:43:18 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5EB7F6B000E; Thu, 1 Aug 2019 14:43:18 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id 239BA6B000C for ; Thu, 1 Aug 2019 14:43:18 -0400 (EDT) Received: by mail-pf1-f200.google.com with SMTP id e20so46351151pfd.3 for ; Thu, 01 Aug 2019 11:43:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=C54ZWbJvrFTMnXF4mH3WgvCiOakGl9oWpo4pI3ucJzc=; b=D2DK18/5Hj73Ty7WqPWrJ5wLuzSW8qUitpu+yPyebXDLYaAYnstRr5/Lua1C6xOuEq qeRjPeAIU3Lk75XXa9qO7HW0tX29WIBaAN2zDYjdThyKO5mWcm39PBSoavSaceGNntlt jfJWrdu7IKYmjUdttCzvw9/dt4OEzxk00p/4VmnLiZLl5co72f4Rnt+srLGFhhLRjec/ p7H2nt4uBntAG5EteGoitZlkBSP/yK87HLei1Id3elznVZjm4z5D5YPQhHSev7yOy2hQ gQunA26WN/4Bkl0YGpaCerZ7TB4UYyXba4m/jbNmFK+9FVFQq+IJzUbue1ViXnILacSq j7ng== X-Gm-Message-State: APjAAAVMU4i81nzBFYJcic/PAE9EHghmc6bWHKlGR9TwRVFBUfzljcRT NMVQz4rY/akvEDNuNlE1Ncbo1H0WktOGtvQkt6ex0HsnAyMhYTE/j9tyK2ueLXZ6a6xL+ugV9mI SXA1Gfl38/VS7/Gr6ZxaIIw+qSFxmfn5SDweqZYENlzakUz/b3NHKwEEWJcrxvJqn7w== X-Received: by 2002:a17:902:5ac4:: with SMTP id g4mr130475159plm.80.1564684997810; Thu, 01 Aug 2019 11:43:17 -0700 (PDT) X-Google-Smtp-Source: APXvYqxy8VLAHQBdTaEN/fr/qVzJoo5uu/Z4ral8CsQz+6qO883eJTBqTLGutppjMZb20bziAQSS X-Received: by 2002:a17:902:5ac4:: with SMTP id g4mr130475120plm.80.1564684997235; Thu, 01 Aug 2019 11:43:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1564684997; cv=none; d=google.com; s=arc-20160816; b=IDhXZgVv4qPYtY0Ae3Qt2LZYGUS4clbmEaqsGHM6WT1Gwx3LqV2afC1poAXZv6EYz+ lWdHCGQiKMWM+MfyjhSSRmtKt4TNMEJrZ4kGbBseS/S+3zYAQSeSYnQD+lS4a8kRlUH8 xHeQX5nFLhytlUytejsOV/2bBGLLeZu3Y+aex3UsWv16j+e39DrLI4f+qjn1rz4RpLZ5 vpXQGrfucImO+KKvMuqYdb2nIDEWDkqaClqBXi6sit3tO+NRYn/UJkat5J6JnietoKh0 DKuZhRLb7R44dW2kFRN8IkWGcEWiN3OnKTAKWOuU/3bm/WfgwunaXz9Tyx6j4gibj+8R POnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=C54ZWbJvrFTMnXF4mH3WgvCiOakGl9oWpo4pI3ucJzc=; b=H+OeU3yjVSjgSeFSG4P9u9K2CWYdspDAxXN2YmtRR5IzYE3VGRyC04nz091smJHepT afCNg+avzh+X00q0Tl1EfWrQYXsC0ZlTbF6Fv3GSpFcBWOUe2GYsEQFdQLLu9F26R1n0 4ZVCU/oyBHT9NowQ2M73JpR/V7qXbyfPiTxO5MWmhDnx5zcY2C71yEK8eKAZlwE7ZtXa bWO0PPxCpNOvO1CYbgzU69kRk8gOTIJAnJNz+dKiG8JiAAM4tH3mbHFcj9oJdvBi4UQj Rf08PsUrLNLt3Q2C/LtqNFEN2M6Kjo6yy21BbwLxGRHTqbtLkv5WA2PH1l76OLmCjfol JAfw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Es8L2Bvd; spf=pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=3116992784=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com. [67.231.145.42]) by mx.google.com with ESMTPS id b6si37167889pfa.76.2019.08.01.11.43.17 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Aug 2019 11:43:17 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.145.42 as permitted sender) client-ip=67.231.145.42; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Es8L2Bvd; spf=pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=3116992784=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x71IgjRR026770 for ; Thu, 1 Aug 2019 11:43:16 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=C54ZWbJvrFTMnXF4mH3WgvCiOakGl9oWpo4pI3ucJzc=; b=Es8L2BvdNjJH76WMCKEX0o5taV5UXqTKbZYAEpMS3FMusm5Ka57nVshXyXfZY5S4ItGI 0GVqHnJ0s42u4dyx7NbsC5B3BD8nM6uKP5q3dCJthvZgPRsScfKMuCIxsgqEBQo/phAR W9DROEPiHIe4vup/dCGrxEq9JvwdW0ed5po= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2u438rgmnr-20 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 01 Aug 2019 11:43:16 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::125) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Thu, 1 Aug 2019 11:43:08 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 4683062E1E18; Thu, 1 Aug 2019 11:43:05 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v10 3/7] filemap: update offset check in filemap_fault() Date: Thu, 1 Aug 2019 11:42:40 -0700 Message-ID: <20190801184244.3169074-4-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190801184244.3169074-1-songliubraving@fb.com> References: <20190801184244.3169074-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-08-01_08:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=863 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908010194 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP With THP, current check of offset: VM_BUG_ON_PAGE(page->index != offset, page); is no longer accurate. Update it to: VM_BUG_ON_PAGE(page_to_pgoff(page) != offset, page); Acked-by: Rik van Riel Acked-by: Kirill A. Shutemov Acked-by: Johannes Weiner Signed-off-by: Song Liu --- mm/filemap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/filemap.c b/mm/filemap.c index aaee1ef96f6d..97c7b7b92c20 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2542,7 +2542,7 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) put_page(page); goto retry_find; } - VM_BUG_ON_PAGE(page->index != offset, page); + VM_BUG_ON_PAGE(page_to_pgoff(page) != offset, page); /* * We have a locked page in the page cache, now we need to check From patchwork Thu Aug 1 18:42:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11071259 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 91841112C for ; Thu, 1 Aug 2019 18:43:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 845A72873E for ; Thu, 1 Aug 2019 18:43:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 786B028740; Thu, 1 Aug 2019 18:43:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 77D8B2873E for ; Thu, 1 Aug 2019 18:43:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 59E156B0008; Thu, 1 Aug 2019 14:43:13 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 54E866B000A; Thu, 1 Aug 2019 14:43:13 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 465486B000C; Thu, 1 Aug 2019 14:43:13 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yw1-f71.google.com (mail-yw1-f71.google.com [209.85.161.71]) by kanga.kvack.org (Postfix) with ESMTP id 262046B0008 for ; Thu, 1 Aug 2019 14:43:13 -0400 (EDT) Received: by mail-yw1-f71.google.com with SMTP id i73so53627069ywa.18 for ; Thu, 01 Aug 2019 11:43:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=LfQYj2OmrExzc7P2TypmvcyFMxGAqWp8dejToxaSLG4=; b=tVdGEF4QMofKQI90yY9pvlYxA4+9sWJvEyKwKYaSzxqW0YYjRVaMGm2srtWBfu0ixZ jReTh0gryUuGSj0ukI042c+1cfj9PeXhmBLvUpGpWaBNKrN7KtWodMp0zVzQXO6bHhl+ 8gd5ibNQ1v6F9namcH1VStzWXAFPVeV6DhincGa8d77S3dUoNuL76yWKf0Mn9iJR4LiE vTLnG1KUPivewkwwzX+3+y0cXAaC1HO1/daPFB74ZlSC79v6KpRHkRgLQGJ4KXEISe2M zQjiTY7yKYw/C+nMS8h2Mm3EbAZqwU2YT/PawXsjyUbWZA8/j2/VZe6JVmGs0b8j0Xzr 95Qg== X-Gm-Message-State: APjAAAWgsglB6pt6YuRiQ7Vl3OkfxvJvto32nlzcKmRWllj2WsCt/mDu f3/pAN6tlR9douvIBatgnLhOdGSJHYjsmqho/TNuydNKiqWKS7U99fXei65vupyPfJTpgRI/cO8 GAesqc05nvj7TWQ8vI5EKKdkBbnNMT04NxygVLf2zAhemqVHk+ROHsWzUd2qAsMcbxg== X-Received: by 2002:a25:8602:: with SMTP id y2mr82279881ybk.483.1564684992867; Thu, 01 Aug 2019 11:43:12 -0700 (PDT) X-Google-Smtp-Source: APXvYqwGbxYPm82ATccbXoOT2XwjMOjGY/xR9vGlBKw55KeST3JfSYktFfMWDKPzNSyUudyD6J3z X-Received: by 2002:a25:8602:: with SMTP id y2mr82279851ybk.483.1564684992240; Thu, 01 Aug 2019 11:43:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1564684992; cv=none; d=google.com; s=arc-20160816; b=rvVf5ZmKSxVt9jco99jjm2/HMPRLxuoUhKVx69T+gAvf1USXrT+Rw0RET0ozIGYhcE 2j+gOceNCwAhLrP6aVs+jH64GZhi4hWEWGnQDzvbA+94AzWKTYXg1ZBzkvE+77XeATEQ LqeXa8fjOE4kIHVYmCx1OMAEukvsElGgHunGK6cdLCxtzic1N/p7sgtnfwumTm1iRS61 s4+kToTtVNzgwtSFwZ8GKRJF9XpewBOS9hxNW7FCDs7X0shwxe55wEUBZYXNFrwdE9+A 6sCyybwq+dNSu/UduvEodtMCsnArzZq44efNTR64seO1xG3r/ClboGCocq+ud3IyW1xs Y2eQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=LfQYj2OmrExzc7P2TypmvcyFMxGAqWp8dejToxaSLG4=; b=hvDWHY+O6maQ/eJe1AzN/ufBFqt70U/tdDl1iDdGhEkjRmZcNTh4ZJDlD1sGDwU6dH hyIUXe+qi4YXCTZZxIWCfQsn3EOWQJULReOMtfuG/5cOu5s1kFUU0JSGZ0SpcCIfxYUr /Gjueb6+Do2eX/eBygHo1iDC5S7N1hL9b63tasrwLY+7VeL/i8Mq311ZEdIwMKYc7JKb P8GI5wct1O/lozvBNVkZQOnwl6JH5p2qsXp7UOBZ0fge/6IsWoa+LDdWs1hhmSSCIQeN O/29T+7ky101tmbLQztjiJZCb65cyphnOLK8R0P4aFl6WlFtPRaRsd/EApYeg0Svok3d UVpA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=bTxYL4j2; spf=pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=3116992784=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id s63si8299050ybb.108.2019.08.01.11.43.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Aug 2019 11:43:12 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=bTxYL4j2; spf=pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=3116992784=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0109331.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x71IbsNR023547 for ; Thu, 1 Aug 2019 11:43:12 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=LfQYj2OmrExzc7P2TypmvcyFMxGAqWp8dejToxaSLG4=; b=bTxYL4j2Z5s+bEHcpcypRHjABvPo0Jc+onbph55Rtptzyb5U0Vp6QMeZKHCG2US+4RgA krB5WdEHYjHv+Yww7rfbrGIXSTTcaEVqE2k2TDUo68HgbeXRunY4JXmPRR4dwi5pcj24 JQYT3boP+Q4AFIK44WNPme7z10Tj1Ps+vVA= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 2u435b8nw8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Thu, 01 Aug 2019 11:43:11 -0700 Received: from mx-out.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::d) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 1 Aug 2019 11:43:11 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 5FCE362E1E18; Thu, 1 Aug 2019 11:43:10 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v10 4/7] mm,thp: stats for file backed THP Date: Thu, 1 Aug 2019 11:42:41 -0700 Message-ID: <20190801184244.3169074-5-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190801184244.3169074-1-songliubraving@fb.com> References: <20190801184244.3169074-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-08-01_08:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908010193 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP In preparation for non-shmem THP, this patch adds a few stats and exposes them in /proc/meminfo, /sys/bus/node/devices//meminfo, and /proc//task//smaps. This patch is mostly a rewrite of Kirill A. Shutemov's earlier version: https://lkml.kernel.org/r/20170126115819.58875-5-kirill.shutemov@linux.intel.com/ Acked-by: Rik van Riel Acked-by: Kirill A. Shutemov Acked-by: Johannes Weiner Signed-off-by: Song Liu --- drivers/base/node.c | 6 ++++++ fs/proc/meminfo.c | 4 ++++ fs/proc/task_mmu.c | 4 +++- include/linux/mmzone.h | 2 ++ mm/vmstat.c | 2 ++ 5 files changed, 17 insertions(+), 1 deletion(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 75b7e6f6535b..4f2714ee819b 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -427,6 +427,8 @@ static ssize_t node_read_meminfo(struct device *dev, "Node %d AnonHugePages: %8lu kB\n" "Node %d ShmemHugePages: %8lu kB\n" "Node %d ShmemPmdMapped: %8lu kB\n" + "Node %d FileHugePages: %8lu kB\n" + "Node %d FilePmdMapped: %8lu kB\n" #endif , nid, K(node_page_state(pgdat, NR_FILE_DIRTY)), @@ -452,6 +454,10 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * + HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_FILE_THPS) * + HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) #endif ); diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index 465ea0153b2a..82673470dde7 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -136,6 +136,10 @@ static int meminfo_proc_show(struct seq_file *m, void *v) global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); + show_val_kb(m, "FileHugePages: ", + global_node_page_state(NR_FILE_THPS) * HPAGE_PMD_NR); + show_val_kb(m, "FilePmdMapped: ", + global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); #endif #ifdef CONFIG_CMA diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 731642e0f5a0..1ea7d730774c 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -417,6 +417,7 @@ struct mem_size_stats { unsigned long lazyfree; unsigned long anonymous_thp; unsigned long shmem_thp; + unsigned long file_thp; unsigned long swap; unsigned long shared_hugetlb; unsigned long private_hugetlb; @@ -586,7 +587,7 @@ static void smaps_pmd_entry(pmd_t *pmd, unsigned long addr, else if (is_zone_device_page(page)) /* pass */; else - VM_BUG_ON_PAGE(1, page); + mss->file_thp += HPAGE_PMD_SIZE; smaps_account(mss, page, true, pmd_young(*pmd), pmd_dirty(*pmd), locked); } #else @@ -803,6 +804,7 @@ static void __show_smap(struct seq_file *m, const struct mem_size_stats *mss, SEQ_PUT_DEC(" kB\nLazyFree: ", mss->lazyfree); SEQ_PUT_DEC(" kB\nAnonHugePages: ", mss->anonymous_thp); SEQ_PUT_DEC(" kB\nShmemPmdMapped: ", mss->shmem_thp); + SEQ_PUT_DEC(" kB\nFilePmdMapped: ", mss->file_thp); SEQ_PUT_DEC(" kB\nShared_Hugetlb: ", mss->shared_hugetlb); seq_put_decimal_ull_width(m, " kB\nPrivate_Hugetlb: ", mss->private_hugetlb >> 10, 7); diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index d77d717c620c..aa0dd8ca36c8 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -234,6 +234,8 @@ enum node_stat_item { NR_SHMEM, /* shmem pages (included tmpfs/GEM pages) */ NR_SHMEM_THPS, NR_SHMEM_PMDMAPPED, + NR_FILE_THPS, + NR_FILE_PMDMAPPED, NR_ANON_THPS, NR_UNSTABLE_NFS, /* NFS unstable pages */ NR_VMSCAN_WRITE, diff --git a/mm/vmstat.c b/mm/vmstat.c index fd7e16ca6996..6afc892a148a 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1158,6 +1158,8 @@ const char * const vmstat_text[] = { "nr_shmem", "nr_shmem_hugepages", "nr_shmem_pmdmapped", + "nr_file_hugepages", + "nr_file_pmdmapped", "nr_anon_transparent_hugepages", "nr_unstable", "nr_vmscan_write", From patchwork Thu Aug 1 18:42:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11071265 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EE54E13A4 for ; Thu, 1 Aug 2019 18:43:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DE6A32873E for ; Thu, 1 Aug 2019 18:43:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D283128740; Thu, 1 Aug 2019 18:43:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4B7CA2873E for ; Thu, 1 Aug 2019 18:43:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4B2F86B000D; Thu, 1 Aug 2019 14:43:19 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 466386B000E; Thu, 1 Aug 2019 14:43:19 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F3F76B0010; Thu, 1 Aug 2019 14:43:19 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yw1-f69.google.com (mail-yw1-f69.google.com [209.85.161.69]) by kanga.kvack.org (Postfix) with ESMTP id 004066B000D for ; Thu, 1 Aug 2019 14:43:19 -0400 (EDT) Received: by mail-yw1-f69.google.com with SMTP id b63so53623642ywc.12 for ; Thu, 01 Aug 2019 11:43:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=AzYai9sTcIKTsjMHtj4n9MazMkcrW4fTxycCVLF20+Q=; b=WhLr94DjcANv0A2EV8PueooqwoXoWxSzeLDCoeLwdzKEqPyErRY6C94vserwqP8FZL l9OeV0acUx2wm5Jc4F5i1mj2OR7/GYK/+gfagQmgMw7wMlvRj+wI/ShKhQdKcQilnBhs Gt2d5Z8ORLupijP1OxFINSbt8GG4foSIkwhmFwp0CChuWxhq7Ras79okaSRWCICcHKDz f8KSXQt/0pmypXVEP7HaWo7epie4sDJttFlpB5lPSK1/21F3Av6ygNWefgRrVV22T7vb FxSb45IczP+jmAoYIHUiMUbseBHn+uv4YitixspiyJJRhuXOhsAg5/+ok68QY0CyK8i1 eA+g== X-Gm-Message-State: APjAAAWusbt9RYakxbnMQ6Ss0DXXI03kICzSTMBZtsXTTkhIKhdGHMhT c9xp6/yEAPjc+iWSoMKa/oBCmxT0Jf7Qz6b58GJiML1HH1ToIc3U66QYPfgc2gUsSoJ6kKc1oPb VUA2JsSsuJNSbgLuuKEeWcS+x7YDpKuT5Bg2leNI58qrG4228mJJYJFbCS4lCtNU5xQ== X-Received: by 2002:a25:8248:: with SMTP id d8mr76023367ybn.393.1564684998741; Thu, 01 Aug 2019 11:43:18 -0700 (PDT) X-Google-Smtp-Source: APXvYqw9rjHS8pfaRR1S/Z1mzttgW0dm1B8M1cXDvxG/V/ixiDElpJoBeByTu0B1jduvMk3mejD5 X-Received: by 2002:a25:8248:: with SMTP id d8mr76023350ybn.393.1564684998230; Thu, 01 Aug 2019 11:43:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1564684998; cv=none; d=google.com; s=arc-20160816; b=U0Vv05wJQwJsRfsWkBzkJxuZWYTnKHCQWMMfNrEV/82OStutFWTpqs0YVGvGV2j2OC kg1Tgv2X7Ud+IO1iTqUaz5HoN1Lq7/24mpfHwaWQoBK7j3huZv2KdOkc7zkBnwkvNiTQ FtmEBUh3nO5N9uUSi+zjc+9wnHxTIUQvp9mFPGX9s+bLv5VwAK9mS8119Bsyzs0zLpMq bMvalNO9jA5c3CEsQSMqWHM/uSzHdsq9v5CqErYDispBrj7iWLY2zpzPMZWh6x5APmFp 1nTDz0yhMv+l3MX1rAwL+1mvdco9GGpJeqMkmszcjuBUiuYEjyTEB9SlpdABIJiKDa0A 7BkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=AzYai9sTcIKTsjMHtj4n9MazMkcrW4fTxycCVLF20+Q=; b=figtip1lU5eWZS7MwMVB/DV5BUus3mpv+gpFQwOLU4QTd4XTQnF+qoHT4GLQ0npxsN ik2Qv9+6iDgKhxFOzE8gXev4leex6ptmNIFbWefU27YkkzTAPDZ7MqQToK2+pvNGDZdK rIrWr6u6/7CcXw8Nqn+/8l9kdoqC+SZD32PdOWL/MrP6aHmmZfZkVn+0uNIb1IPS2r9L 9hLa1PCrdETpy8VGU3Oi5s9Bgis7FO5HM6qlI0ZdJUmV2cpoOP9hcEj/tsMLtd+sBchD XVeBncS8ZHtHPiyXNqQiNA56vahOe7N3dNjBBQQEVn4cQ7Hj9u9jym54Pi8K0y8KDRDP zE8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=PFEjq2jj; spf=pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=3116992784=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id y13si23549206ybk.460.2019.08.01.11.43.18 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Aug 2019 11:43:18 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=PFEjq2jj; spf=pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=3116992784=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0001255.ppops.net [127.0.0.1]) by mx0b-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x71IgBIW001277 for ; Thu, 1 Aug 2019 11:43:18 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=AzYai9sTcIKTsjMHtj4n9MazMkcrW4fTxycCVLF20+Q=; b=PFEjq2jjC9LTyDyDwNC7HQFn1A/r25tzdX3bUfGIUvc2iC20XaQqeApxuSuSKTxL8YcM /zXquoKzDH5QrHVenNuHrklRnI5oTJkTte6rI6x8OEAZ/I8kO9wd8R6Iw57+LTZjwMnV lTblVg9TuIpzjqqzAfZzPj+BkLVDW3XFkvI= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by mx0b-00082601.pphosted.com with ESMTP id 2u427wrwcg-8 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 01 Aug 2019 11:43:17 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::126) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Thu, 1 Aug 2019 11:43:15 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 527C562E1E18; Thu, 1 Aug 2019 11:43:13 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v10 5/7] khugepaged: rename collapse_shmem() and khugepaged_scan_shmem() Date: Thu, 1 Aug 2019 11:42:42 -0700 Message-ID: <20190801184244.3169074-6-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190801184244.3169074-1-songliubraving@fb.com> References: <20190801184244.3169074-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-08-01_08:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=644 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908010194 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Next patch will add khugepaged support of non-shmem files. This patch renames these two functions to reflect the new functionality: collapse_shmem() => collapse_file() khugepaged_scan_shmem() => khugepaged_scan_file() Acked-by: Rik van Riel Acked-by: Kirill A. Shutemov Acked-by: Johannes Weiner Signed-off-by: Song Liu --- mm/khugepaged.c | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index b9949014346b..9d3cc2061960 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1426,7 +1426,7 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) } /** - * collapse_shmem - collapse small tmpfs/shmem pages into huge one. + * collapse_file - collapse small tmpfs/shmem pages into huge one. * * Basic scheme is simple, details are more complex: * - allocate and lock a new huge page; @@ -1443,10 +1443,11 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) * + restore gaps in the page cache; * + unlock and free huge page; */ -static void collapse_shmem(struct mm_struct *mm, - struct address_space *mapping, pgoff_t start, +static void collapse_file(struct mm_struct *mm, + struct file *file, pgoff_t start, struct page **hpage, int node) { + struct address_space *mapping = file->f_mapping; gfp_t gfp; struct page *new_page; struct mem_cgroup *memcg; @@ -1702,11 +1703,11 @@ static void collapse_shmem(struct mm_struct *mm, /* TODO: tracepoints */ } -static void khugepaged_scan_shmem(struct mm_struct *mm, - struct address_space *mapping, - pgoff_t start, struct page **hpage) +static void khugepaged_scan_file(struct mm_struct *mm, + struct file *file, pgoff_t start, struct page **hpage) { struct page *page = NULL; + struct address_space *mapping = file->f_mapping; XA_STATE(xas, &mapping->i_pages, start); int present, swap; int node = NUMA_NO_NODE; @@ -1770,16 +1771,15 @@ static void khugepaged_scan_shmem(struct mm_struct *mm, result = SCAN_EXCEED_NONE_PTE; } else { node = khugepaged_find_target_node(); - collapse_shmem(mm, mapping, start, hpage, node); + collapse_file(mm, file, start, hpage, node); } } /* TODO: tracepoints */ } #else -static void khugepaged_scan_shmem(struct mm_struct *mm, - struct address_space *mapping, - pgoff_t start, struct page **hpage) +static void khugepaged_scan_file(struct mm_struct *mm, + struct file *file, pgoff_t start, struct page **hpage) { BUILD_BUG(); } @@ -1862,8 +1862,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, file = get_file(vma->vm_file); up_read(&mm->mmap_sem); ret = 1; - khugepaged_scan_shmem(mm, file->f_mapping, - pgoff, hpage); + khugepaged_scan_file(mm, file, pgoff, hpage); fput(file); } else { ret = khugepaged_scan_pmd(mm, vma, From patchwork Thu Aug 1 18:42:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11071279 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 26EC6112C for ; Thu, 1 Aug 2019 18:46:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 15FC92873F for ; Thu, 1 Aug 2019 18:46:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0808B28748; Thu, 1 Aug 2019 18:46:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 28E092873F for ; Thu, 1 Aug 2019 18:46:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4B0776B0005; Thu, 1 Aug 2019 14:46:08 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 43C216B0006; Thu, 1 Aug 2019 14:46:08 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 302596B0007; Thu, 1 Aug 2019 14:46:08 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) by kanga.kvack.org (Postfix) with ESMTP id E138D6B0005 for ; Thu, 1 Aug 2019 14:46:07 -0400 (EDT) Received: by mail-pf1-f199.google.com with SMTP id a20so46327085pfn.19 for ; Thu, 01 Aug 2019 11:46:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=3of5E40BZCvrFLhVG73nHh78NNgN1wW5I01U6khZ8sM=; b=g2jtKUfzFvxDWtLfGab1pYg9bxSxUahoKooOg7BD6NXpzXnm5/lbwMjkqcLR9EqPVs Nw3vmiUGYS2qGU4gKxIhl0KFCATThBxzqqsePsmxWo8uLbXnXyUD6/Cp5ctVntbtyqsl yLMIh0vESo7VwJc++Y4fh2uTOfuvB1J18kkzqZ6sbkS2V+hEaQ6tLOgMqlOu7dZNKRLm 9GHrf6o5v97K48usQ6SDDAKlOmyoklW96m59nR/KJOjod/frpqLqf8Uv/HkCWC48X1z+ TQSkHgqgR7ytBaPa0/XbZvCaSY+UiUKpV1+yso2BVHLTv1wF0NGi6WpfTkMnIYTqul0H gbQw== X-Gm-Message-State: APjAAAVVeA3GVkYIed+w+Z4z/bbA6BmDRzmWfUkADLMAPpL1272t08wO EcKlwfrMEqVJcXz/Ootbzt9UcazipQkQ3DcgamaxqtvWYw7FzftPuStZy3N5CTChrRWILbS5EsB +JV/CTvCORG/290W0bB7uxkf/3ImCm4KbMfCLRRktsLlNSh0v5yFXSa0uK1qQDmcXzQ== X-Received: by 2002:a17:902:1003:: with SMTP id b3mr68240083pla.172.1564685167482; Thu, 01 Aug 2019 11:46:07 -0700 (PDT) X-Google-Smtp-Source: APXvYqzixSrOu07pl8zIn9MZLbR51fdNU11caLdgs6lhdblimpnBGlFEyWoOrPN4fbC7fs4pG0xO X-Received: by 2002:a17:902:1003:: with SMTP id b3mr68240032pla.172.1564685166348; Thu, 01 Aug 2019 11:46:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1564685166; cv=none; d=google.com; s=arc-20160816; b=uR/PiDoPKzq0hus/abqDq2+XIvenalgGzMZNnP0nfaNj3qc8pmhuPd4RsI/Tj+Wape 6AJAaxePpAfk8yo5sR3lfpRqn3y709HLzaNeGvZPVmlszhTHiA02WE+fZY1zpxEdOZPU n+l5sk2GwQd85CkBBx1WZjdyNF4vT5cSLr/fG/1rr37StgCn+7yrJ0HykkpJtQAs9cOB xEuRCs9Xg2caqxJcLGw1IyEH7XuTbf+832Irsjs2IiUkoLh7x8nF5aaxYsP1vIM8YgKh dy3yBxLiLCNssIsx68YIspowLCZG71y232tZuOh0ZomGjvihHd01qykjuj07wGJhpyny Rsig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=3of5E40BZCvrFLhVG73nHh78NNgN1wW5I01U6khZ8sM=; b=g7hivAU7sXBTCbKNKoXoCKADn94fUCP8PRbhrH1udsAbvFX8S0zgrrJEH+eSz9dqZT IHlXQKPlH6e4JXGfSso2dw8NWeYlDsqTACK5k1mjnBkYBRAHu/UMPFI/e+iCMNmOvrSb TixzUw/qVlH/pHFIbD8GOpFpvSetUVZ0phAecziMyo6DUMtIi9zzPNJTsdLOHw836XCQ yzAto8LmqJXbrdNEm/rRPHW/zM2TVVC6mCR7eydE6tcVCAMNFPIeC+xNjc3jjjAnvUdj UsRPuNT1WLzOhP4B+a/0+Sk4Fisaet+xJ4apxTnXviNtDsvbuQTD2lHOES/jrgCg8Imb sBbA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b="MN/0bsf9"; spf=pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=3116992784=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com. [67.231.145.42]) by mx.google.com with ESMTPS id n123si19087081pgn.151.2019.08.01.11.46.06 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Aug 2019 11:46:06 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.145.42 as permitted sender) client-ip=67.231.145.42; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b="MN/0bsf9"; spf=pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=3116992784=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0044008.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x71IhIEX002020 for ; Thu, 1 Aug 2019 11:46:05 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=3of5E40BZCvrFLhVG73nHh78NNgN1wW5I01U6khZ8sM=; b=MN/0bsf9Xlg6LME/eVypAzy3Ytc/TPzqNZU0w6xsPr/OSHiEqVz3cSM6ZcO+7rsYAz7e Jm/UUfTzWpo0g4AnZTsY9jcsEj//O2bD1wdCe3dfhu2Tzu7Tbd/CpPbS46DfLscUZX6u TpcmyUuTdcxrrREd7SYvmp9IPk0962fYXZE= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2u430v0qsy-12 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 01 Aug 2019 11:46:05 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::128) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Thu, 1 Aug 2019 11:45:53 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 66F6262E1E18; Thu, 1 Aug 2019 11:43:17 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu , Johannes Weiner Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v10 6/7] mm,thp: add read-only THP support for (non-shmem) FS Date: Thu, 1 Aug 2019 11:42:43 -0700 Message-ID: <20190801184244.3169074-7-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190801184244.3169074-1-songliubraving@fb.com> References: <20190801184244.3169074-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-08-01_08:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908010194 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch is (hopefully) the first step to enable THP for non-shmem filesystems. This patch enables an application to put part of its text sections to THP via madvise, for example: madvise((void *)0x600000, 0x200000, MADV_HUGEPAGE); We tried to reuse the logic for THP on tmpfs. Currently, write is not supported for non-shmem THP. khugepaged will only process vma with VM_DENYWRITE. sys_mmap() ignores VM_DENYWRITE requests (see ksys_mmap_pgoff). The only way to create vma with VM_DENYWRITE is execve(). This requirement limits non-shmem THP to text sections. The next patch will handle writes, which would only happen when the all the vmas with VM_DENYWRITE are unmapped. An EXPERIMENTAL config, READ_ONLY_THP_FOR_FS, is added to gate this feature. Cc: Johannes Weiner Acked-by: Rik van Riel Acked-by: Kirill A. Shutemov Signed-off-by: Song Liu Acked-by: Johannes Weiner --- mm/Kconfig | 11 ++++ mm/filemap.c | 4 +- mm/khugepaged.c | 149 ++++++++++++++++++++++++++++++++++-------------- mm/rmap.c | 12 ++-- 4 files changed, 128 insertions(+), 48 deletions(-) diff --git a/mm/Kconfig b/mm/Kconfig index 56cec636a1fc..06f758786e4a 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -723,6 +723,17 @@ config GUP_BENCHMARK config GUP_GET_PTE_LOW_HIGH bool +config READ_ONLY_THP_FOR_FS + bool "Read-only THP for filesystems (EXPERIMENTAL)" + depends on TRANSPARENT_HUGE_PAGECACHE && SHMEM + + help + Allow khugepaged to put read-only file-backed pages in THP. + + This is marked experimental because it is a new feature. Write + support of file THPs will be developed in the next few release + cycles. + config ARCH_HAS_PTE_SPECIAL bool diff --git a/mm/filemap.c b/mm/filemap.c index 97c7b7b92c20..0b8b117dbed6 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -203,8 +203,8 @@ static void unaccount_page_cache_page(struct address_space *mapping, __mod_node_page_state(page_pgdat(page), NR_SHMEM, -nr); if (PageTransHuge(page)) __dec_node_page_state(page, NR_SHMEM_THPS); - } else { - VM_BUG_ON_PAGE(PageTransHuge(page), page); + } else if (PageTransHuge(page)) { + __dec_node_page_state(page, NR_FILE_THPS); } /* diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 9d3cc2061960..8fb26856a7e9 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -48,6 +48,7 @@ enum scan_result { SCAN_CGROUP_CHARGE_FAIL, SCAN_EXCEED_SWAP_PTE, SCAN_TRUNCATED, + SCAN_PAGE_HAS_PRIVATE, }; #define CREATE_TRACE_POINTS @@ -410,7 +411,11 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, (vm_flags & VM_NOHUGEPAGE) || test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) return false; - if (shmem_file(vma->vm_file)) { + + if (shmem_file(vma->vm_file) || + (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && + vma->vm_file && + (vm_flags & VM_DENYWRITE))) { if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGE_PAGECACHE)) return false; return IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, @@ -462,8 +467,9 @@ int khugepaged_enter_vma_merge(struct vm_area_struct *vma, unsigned long hstart, hend; /* - * khugepaged does not yet work on non-shmem files or special - * mappings. And file-private shmem THP is not supported. + * khugepaged only supports read-only files for non-shmem files. + * khugepaged does not yet work on special mappings. And + * file-private shmem THP is not supported. */ if (!hugepage_vma_check(vma, vm_flags)) return 0; @@ -1426,12 +1432,12 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) } /** - * collapse_file - collapse small tmpfs/shmem pages into huge one. + * collapse_file - collapse filemap/tmpfs/shmem pages into huge one. * * Basic scheme is simple, details are more complex: * - allocate and lock a new huge page; * - scan page cache replacing old pages with the new one - * + swap in pages if necessary; + * + swap/gup in pages if necessary; * + fill in gaps; * + keep old pages around in case rollback is required; * - if replacing succeeds: @@ -1455,7 +1461,9 @@ static void collapse_file(struct mm_struct *mm, LIST_HEAD(pagelist); XA_STATE_ORDER(xas, &mapping->i_pages, start, HPAGE_PMD_ORDER); int nr_none = 0, result = SCAN_SUCCEED; + bool is_shmem = shmem_file(file); + VM_BUG_ON(!IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && !is_shmem); VM_BUG_ON(start & (HPAGE_PMD_NR - 1)); /* Only allocate from the target node */ @@ -1487,7 +1495,8 @@ static void collapse_file(struct mm_struct *mm, } while (1); __SetPageLocked(new_page); - __SetPageSwapBacked(new_page); + if (is_shmem) + __SetPageSwapBacked(new_page); new_page->index = start; new_page->mapping = mapping; @@ -1502,41 +1511,75 @@ static void collapse_file(struct mm_struct *mm, struct page *page = xas_next(&xas); VM_BUG_ON(index != xas.xa_index); - if (!page) { - /* - * Stop if extent has been truncated or hole-punched, - * and is now completely empty. - */ - if (index == start) { - if (!xas_next_entry(&xas, end - 1)) { - result = SCAN_TRUNCATED; + if (is_shmem) { + if (!page) { + /* + * Stop if extent has been truncated or + * hole-punched, and is now completely + * empty. + */ + if (index == start) { + if (!xas_next_entry(&xas, end - 1)) { + result = SCAN_TRUNCATED; + goto xa_locked; + } + xas_set(&xas, index); + } + if (!shmem_charge(mapping->host, 1)) { + result = SCAN_FAIL; goto xa_locked; } - xas_set(&xas, index); + xas_store(&xas, new_page); + nr_none++; + continue; } - if (!shmem_charge(mapping->host, 1)) { - result = SCAN_FAIL; + + if (xa_is_value(page) || !PageUptodate(page)) { + xas_unlock_irq(&xas); + /* swap in or instantiate fallocated page */ + if (shmem_getpage(mapping->host, index, &page, + SGP_NOHUGE)) { + result = SCAN_FAIL; + goto xa_unlocked; + } + } else if (trylock_page(page)) { + get_page(page); + xas_unlock_irq(&xas); + } else { + result = SCAN_PAGE_LOCK; goto xa_locked; } - xas_store(&xas, new_page); - nr_none++; - continue; - } - - if (xa_is_value(page) || !PageUptodate(page)) { - xas_unlock_irq(&xas); - /* swap in or instantiate fallocated page */ - if (shmem_getpage(mapping->host, index, &page, - SGP_NOHUGE)) { + } else { /* !is_shmem */ + if (!page || xa_is_value(page)) { + xas_unlock_irq(&xas); + page_cache_sync_readahead(mapping, &file->f_ra, + file, index, + PAGE_SIZE); + /* drain pagevecs to help isolate_lru_page() */ + lru_add_drain(); + page = find_lock_page(mapping, index); + if (unlikely(page == NULL)) { + result = SCAN_FAIL; + goto xa_unlocked; + } + } else if (!PageUptodate(page)) { + xas_unlock_irq(&xas); + wait_on_page_locked(page); + if (!trylock_page(page)) { + result = SCAN_PAGE_LOCK; + goto xa_unlocked; + } + get_page(page); + } else if (PageDirty(page)) { result = SCAN_FAIL; - goto xa_unlocked; + goto xa_locked; + } else if (trylock_page(page)) { + get_page(page); + xas_unlock_irq(&xas); + } else { + result = SCAN_PAGE_LOCK; + goto xa_locked; } - } else if (trylock_page(page)) { - get_page(page); - xas_unlock_irq(&xas); - } else { - result = SCAN_PAGE_LOCK; - goto xa_locked; } /* @@ -1565,6 +1608,12 @@ static void collapse_file(struct mm_struct *mm, goto out_unlock; } + if (page_has_private(page) && + !try_to_release_page(page, GFP_KERNEL)) { + result = SCAN_PAGE_HAS_PRIVATE; + break; + } + if (page_mapped(page)) unmap_mapping_pages(mapping, index, 1, false); @@ -1602,12 +1651,18 @@ static void collapse_file(struct mm_struct *mm, goto xa_unlocked; } - __inc_node_page_state(new_page, NR_SHMEM_THPS); + if (is_shmem) + __inc_node_page_state(new_page, NR_SHMEM_THPS); + else + __inc_node_page_state(new_page, NR_FILE_THPS); + if (nr_none) { struct zone *zone = page_zone(new_page); __mod_node_page_state(zone->zone_pgdat, NR_FILE_PAGES, nr_none); - __mod_node_page_state(zone->zone_pgdat, NR_SHMEM, nr_none); + if (is_shmem) + __mod_node_page_state(zone->zone_pgdat, + NR_SHMEM, nr_none); } xa_locked: @@ -1645,10 +1700,15 @@ static void collapse_file(struct mm_struct *mm, SetPageUptodate(new_page); page_ref_add(new_page, HPAGE_PMD_NR - 1); - set_page_dirty(new_page); mem_cgroup_commit_charge(new_page, memcg, false, true); + + if (is_shmem) { + set_page_dirty(new_page); + lru_cache_add_anon(new_page); + } else { + lru_cache_add_file(new_page); + } count_memcg_events(memcg, THP_COLLAPSE_ALLOC, 1); - lru_cache_add_anon(new_page); /* * Remove pte page tables, so we can re-fault the page as huge. @@ -1663,7 +1723,9 @@ static void collapse_file(struct mm_struct *mm, /* Something went wrong: roll back page cache changes */ xas_lock_irq(&xas); mapping->nrpages -= nr_none; - shmem_uncharge(mapping->host, nr_none); + + if (is_shmem) + shmem_uncharge(mapping->host, nr_none); xas_set(&xas, start); xas_for_each(&xas, page, end - 1) { @@ -1746,7 +1808,8 @@ static void khugepaged_scan_file(struct mm_struct *mm, break; } - if (page_count(page) != 1 + page_mapcount(page)) { + if (page_count(page) != + 1 + page_mapcount(page) + page_has_private(page)) { result = SCAN_PAGE_COUNT; break; } @@ -1853,11 +1916,13 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, VM_BUG_ON(khugepaged_scan.address < hstart || khugepaged_scan.address + HPAGE_PMD_SIZE > hend); - if (shmem_file(vma->vm_file)) { + if (vma->vm_file) { struct file *file; pgoff_t pgoff = linear_page_index(vma, khugepaged_scan.address); - if (!shmem_huge_enabled(vma)) + + if (shmem_file(vma->vm_file) + && !shmem_huge_enabled(vma)) goto skip; file = get_file(vma->vm_file); up_read(&mm->mmap_sem); diff --git a/mm/rmap.c b/mm/rmap.c index e5dfe2ae6b0d..87cfa2c19eda 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1192,8 +1192,10 @@ void page_add_file_rmap(struct page *page, bool compound) } if (!atomic_inc_and_test(compound_mapcount_ptr(page))) goto out; - VM_BUG_ON_PAGE(!PageSwapBacked(page), page); - __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + if (PageSwapBacked(page)) + __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + else + __inc_node_page_state(page, NR_FILE_PMDMAPPED); } else { if (PageTransCompound(page) && page_mapping(page)) { VM_WARN_ON_ONCE(!PageLocked(page)); @@ -1232,8 +1234,10 @@ static void page_remove_file_rmap(struct page *page, bool compound) } if (!atomic_add_negative(-1, compound_mapcount_ptr(page))) goto out; - VM_BUG_ON_PAGE(!PageSwapBacked(page), page); - __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + if (PageSwapBacked(page)) + __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + else + __dec_node_page_state(page, NR_FILE_PMDMAPPED); } else { if (!atomic_add_negative(-1, &page->_mapcount)) goto out; From patchwork Thu Aug 1 18:42:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 11071281 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A4EB1399 for ; Thu, 1 Aug 2019 18:46:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4C4A928748 for ; Thu, 1 Aug 2019 18:46:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3FAC12874F; Thu, 1 Aug 2019 18:46:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8F46028748 for ; Thu, 1 Aug 2019 18:46:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 748F66B000C; Thu, 1 Aug 2019 14:46:17 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6F9146B0010; Thu, 1 Aug 2019 14:46:17 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5C1246B0266; Thu, 1 Aug 2019 14:46:17 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yw1-f72.google.com (mail-yw1-f72.google.com [209.85.161.72]) by kanga.kvack.org (Postfix) with ESMTP id 34B586B000C for ; Thu, 1 Aug 2019 14:46:17 -0400 (EDT) Received: by mail-yw1-f72.google.com with SMTP id 77so53822468ywp.14 for ; Thu, 01 Aug 2019 11:46:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:smtp-origin-hostprefix:from :smtp-origin-hostname:to:cc:smtp-origin-cluster:subject:date :message-id:in-reply-to:references:mime-version; bh=DSUSc38wHM5RkmzSyeUjxuPKYIC6pKxjmKbqWbRefe4=; b=CG2pnciswqv6ubHIl6qDunsdjMZu8ufXlZ845z1UIc+vjD5fkIE53NL+EKe77lduRF 2LNhUQNc6M/jfQ5GXBc8rXud+iYwB31vSDBjzhO2KOIy9j9gXr5R4KFBKgtbmZVHcxdU Gv2GZoV5jFCILOo6sC9L5j9HrcEKwjeyLHyk6oRSdUviToTpOLkvmi7AZjr5J/0S6+2f gJlUOTYU3bxTBQL41YLx47W2sq79Vqq9mlss8w8A0QvnMGVz8DQr2bOTHAFvK8b52T89 PzPUh/VWFEYLleVKGGiNn38/lQpnjDlRR6f80PYANJ2xUyoeiwO3CZqWvMFKRWIZ/bsx bO7Q== X-Gm-Message-State: APjAAAVtNnVSaEVK/FxMYNY3dfIo0zqzvENlGtXnhaJoSChjnziCrNN5 H3STCYnSAL7odAfkw2iwgWqmqivy+5YCunkQ3/ZtFOf82LRP7250qxuuWLh7EPlE8NB5TTA6J+7 yThsaaD2XkcQvgCF4xwbKMBGRdS0QTotlRbmehZIGdTALFedn1I864DA4ZOqFnR6a5w== X-Received: by 2002:a25:6c2:: with SMTP id 185mr37421612ybg.68.1564685176962; Thu, 01 Aug 2019 11:46:16 -0700 (PDT) X-Google-Smtp-Source: APXvYqwOQhnwdEFKJl6a2AV2I/saIXxk4W8A0xVQeeQkmtwZO6jECg0455vnHPFSfdHq6LXA0onu X-Received: by 2002:a25:6c2:: with SMTP id 185mr37421547ybg.68.1564685176025; Thu, 01 Aug 2019 11:46:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1564685176; cv=none; d=google.com; s=arc-20160816; b=vTByJaiIZBPcPfdCjIwobbYmJsXtJ3YqitIc+ycqfKydU+thPx6zRBkE/1SAHUIcJ5 6tSqJ62CSwyyaj9lQMmrhluaKM/7w/kR7s2YX4ZF7knoq67rLlqLJnUxUP13NLhTjZOk y0Mt5fn9TfLsh01XxCQ/0p+5fwd5KIOi3DjaLwukQa+hzwa8wvOMwHilhOKm3thQ77fX 5d4UB3IpfravhPl55HjhLEUW1tsw5gvhA8RoAQVuTpSXkMBfgp1zvfAPUE1zxN9R312w AVt4F5hrLWflRNOvot1MCBhPy/gkgudoqDvIVshwPh/N2073j2Xawjqfe07EAzWeTUgS Ilvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject :smtp-origin-cluster:cc:to:smtp-origin-hostname:from :smtp-origin-hostprefix:dkim-signature; bh=DSUSc38wHM5RkmzSyeUjxuPKYIC6pKxjmKbqWbRefe4=; b=jG2ikB1o5G+SXj2M/eYCu/eGMVYPbz/N19moBDEAee7JEnvnT1y+j6gdCWV/4wHhfq K+qJXIhGTzab4lRjPneNBXNhIu2GXvumvGJ6rIl5CqolMBmAW3hM7LkDm7dFq6tTe7b/ pJ2bewLKLP3R4cSXVQyqSc94Vy7ftIOf/zN52pJbgla+S5uZLqqDJnObgQvEytmA5CJK R/0lH3rVc2VdNUFZWfeB8v26GLNzzSlCGzSWccwU4zTbu1xzlBRGUHH6Q+hNEjrOPWcP V3CDLVvCpTWmCKN5CDerfkCB7lvhv7lSqmNfqtszufBn1yBv8fv/+Lmv/ESv7kKEkYew +qNg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=YQzgCtlG; spf=pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=3116992784=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id 191si24810556ybe.19.2019.08.01.11.46.15 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Aug 2019 11:46:16 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=YQzgCtlG; spf=pass (google.com: domain of prvs=3116992784=songliubraving@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=3116992784=songliubraving@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0109332.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x71IhoFJ006842 for ; Thu, 1 Aug 2019 11:46:15 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=DSUSc38wHM5RkmzSyeUjxuPKYIC6pKxjmKbqWbRefe4=; b=YQzgCtlGKcw3cZAFdT3dY+Q0nIn0gOdjHOIyt0NnuKjMY2KwdwRYvCCdJneWssmu1TXf dc0TQ6UKsM4GRJgEsQP3AXyQpRY/XM7eDEo6RbFSQKs4lCR2fV46VMS4r6yR7fwhnw+u FJNidSnfXfFpoh6Y+ZmN7aWHX2n87/4WWlY= Received: from mail.thefacebook.com (mailout.thefacebook.com [199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2u41gys3xy-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 01 Aug 2019 11:46:15 -0700 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::128) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1713.5; Thu, 1 Aug 2019 11:45:53 -0700 Received: by devbig006.ftw2.facebook.com (Postfix, from userid 4523) id 835FE62E225C; Thu, 1 Aug 2019 11:43:20 -0700 (PDT) Smtp-Origin-Hostprefix: devbig From: Song Liu Smtp-Origin-Hostname: devbig006.ftw2.facebook.com To: , , CC: , , , , , , Song Liu , Johannes Weiner Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v10 7/7] mm,thp: avoid writes to file with THP in pagecache Date: Thu, 1 Aug 2019 11:42:44 -0700 Message-ID: <20190801184244.3169074-8-songliubraving@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190801184244.3169074-1-songliubraving@fb.com> References: <20190801184244.3169074-1-songliubraving@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-08-01_08:,, signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=876 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1906280000 definitions=main-1908010194 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP In previous patch, an application could put part of its text section in THP via madvise(). These THPs will be protected from writes when the application is still running (TXTBSY). However, after the application exits, the file is available for writes. This patch avoids writes to file THP by dropping page cache for the file when the file is open for write. A new counter nr_thps is added to struct address_space. In do_dentry_open(), if the file is open for write and nr_thps is non-zero, we drop page cache for the whole file. Cc: Johannes Weiner Reported-by: kbuild test robot Acked-by: Rik van Riel Acked-by: Kirill A. Shutemov Signed-off-by: Song Liu Acked-by: Johannes Weiner --- fs/inode.c | 3 +++ fs/open.c | 8 ++++++++ include/linux/fs.h | 32 ++++++++++++++++++++++++++++++++ mm/filemap.c | 1 + mm/khugepaged.c | 4 +++- 5 files changed, 47 insertions(+), 1 deletion(-) diff --git a/fs/inode.c b/fs/inode.c index 0f1e3b563c47..ef33fdf0105f 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -181,6 +181,9 @@ int inode_init_always(struct super_block *sb, struct inode *inode) mapping->flags = 0; mapping->wb_err = 0; atomic_set(&mapping->i_mmap_writable, 0); +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + atomic_set(&mapping->nr_thps, 0); +#endif mapping_set_gfp_mask(mapping, GFP_HIGHUSER_MOVABLE); mapping->private_data = NULL; mapping->writeback_index = 0; diff --git a/fs/open.c b/fs/open.c index a59abe3c669a..c60cd22cc052 100644 --- a/fs/open.c +++ b/fs/open.c @@ -818,6 +818,14 @@ static int do_dentry_open(struct file *f, if (!f->f_mapping->a_ops || !f->f_mapping->a_ops->direct_IO) return -EINVAL; } + + /* + * XXX: Huge page cache doesn't support writing yet. Drop all page + * cache for this file before processing writes. + */ + if ((f->f_mode & FMODE_WRITE) && filemap_nr_thps(inode->i_mapping)) + truncate_pagecache(inode, 0); + return 0; cleanup_all: diff --git a/include/linux/fs.h b/include/linux/fs.h index 56b8e358af5c..29b2729bd48a 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -427,6 +427,7 @@ int pagecache_write_end(struct file *, struct address_space *mapping, * @i_pages: Cached pages. * @gfp_mask: Memory allocation flags to use for allocating pages. * @i_mmap_writable: Number of VM_SHARED mappings. + * @nr_thps: Number of THPs in the pagecache (non-shmem only). * @i_mmap: Tree of private and shared mappings. * @i_mmap_rwsem: Protects @i_mmap and @i_mmap_writable. * @nrpages: Number of page entries, protected by the i_pages lock. @@ -444,6 +445,10 @@ struct address_space { struct xarray i_pages; gfp_t gfp_mask; atomic_t i_mmap_writable; +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + /* number of thp, only for non-shmem files */ + atomic_t nr_thps; +#endif struct rb_root_cached i_mmap; struct rw_semaphore i_mmap_rwsem; unsigned long nrpages; @@ -2775,6 +2780,33 @@ static inline errseq_t filemap_sample_wb_err(struct address_space *mapping) return errseq_sample(&mapping->wb_err); } +static inline int filemap_nr_thps(struct address_space *mapping) +{ +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + return atomic_read(&mapping->nr_thps); +#else + return 0; +#endif +} + +static inline void filemap_nr_thps_inc(struct address_space *mapping) +{ +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + atomic_inc(&mapping->nr_thps); +#else + WARN_ON_ONCE(1); +#endif +} + +static inline void filemap_nr_thps_dec(struct address_space *mapping) +{ +#ifdef CONFIG_READ_ONLY_THP_FOR_FS + atomic_dec(&mapping->nr_thps); +#else + WARN_ON_ONCE(1); +#endif +} + extern int vfs_fsync_range(struct file *file, loff_t start, loff_t end, int datasync); extern int vfs_fsync(struct file *file, int datasync); diff --git a/mm/filemap.c b/mm/filemap.c index 0b8b117dbed6..45940d1f59db 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -205,6 +205,7 @@ static void unaccount_page_cache_page(struct address_space *mapping, __dec_node_page_state(page, NR_SHMEM_THPS); } else if (PageTransHuge(page)) { __dec_node_page_state(page, NR_FILE_THPS); + filemap_nr_thps_dec(mapping); } /* diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 8fb26856a7e9..80735bf2a70e 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1653,8 +1653,10 @@ static void collapse_file(struct mm_struct *mm, if (is_shmem) __inc_node_page_state(new_page, NR_SHMEM_THPS); - else + else { __inc_node_page_state(new_page, NR_FILE_THPS); + filemap_nr_thps_inc(mapping); + } if (nr_none) { struct zone *zone = page_zone(new_page);