From patchwork Mon Nov 26 23:16:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 10699325 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3710713AD for ; Mon, 26 Nov 2018 23:16:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1C3E52A63D for ; Mon, 26 Nov 2018 23:16:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0E9122A63F; Mon, 26 Nov 2018 23:16:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D3C202A63D for ; Mon, 26 Nov 2018 23:16:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E4CF66B4450; Mon, 26 Nov 2018 18:16:32 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DFBCA6B4451; Mon, 26 Nov 2018 18:16:32 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D126C6B4452; Mon, 26 Nov 2018 18:16:32 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f198.google.com (mail-pl1-f198.google.com [209.85.214.198]) by kanga.kvack.org (Postfix) with ESMTP id 8D3FE6B4450 for ; Mon, 26 Nov 2018 18:16:32 -0500 (EST) Received: by mail-pl1-f198.google.com with SMTP id l9so21851267plt.7 for ; Mon, 26 Nov 2018 15:16:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :message-id:user-agent:mime-version; bh=rGPpra3RUdchVXXSTigsvpJv4NP8t4KkKUk8/ihis1I=; b=ELK4xtMdn3A6BSll9jOtmBXWGZ0EBCzCAIOiMUcmULmO3NYvIaoZxQUk1WJWt8+BCJ EniUFopECkJA4d0QFQexr6bd3iqWtY+EHn7ZV5hMM2xpljRAy1qyZk5eOzVdOdy7hQtT ab3ji/EnffR42gDcI1Br3cIofzNzEhjJ/3PvLCQpLj/jB5t49eSwH9xTslSwyAyi5J4e EMv7CH9kmKsGxf2+CW1p0Iz+2hrd+e4QyO4fo8NdAqJ3bada56Xmx46M77nTqhOz6IjN ZLgofokvOkmKbS7CbdYeBkTeGirynn/O2kt7rzGvZT8i5VkcBzD+7epQB6GtG+o6AdhP 2+BA== X-Gm-Message-State: AA+aEWbdqLNh4ya1ainj0K6YFQyARHf/buUmOTAEdd3A6S+hXdqu7b3U uI2it1YaUaucSv0rcDBp8xWVQwyVQekhtkcoWzXrEU9yI4QGTh0FuasDVT23kWeq+CFdfAL1dMj Z0OMqz92r6xmm+fKBd6fMWQ6MvZcEQpbBKaTVPnG7nuNZP9R1eOlWmHVlFgFoAPs1tRZiic2kWa LHCV62i4Jgkn47ohK1KliPliloPvdrNu6566yuSO/602BO0vya25Mn4sXRy5C4SuoAwmIZpmbko S2IVleS33sz4vMCE8zZWhEOpsyKHTRIpJGk+UiyESVc/Pg/hi+rDC/XgLbu1CJBPejwj/YpCxxU p/wHQZnlDqafCmseemw++kEHZAvyR8xRMqkA0V02oRtrk5AMJjfiOl8YFfQX8iwDGGOfUGpwsx+ 4 X-Received: by 2002:a17:902:1d4a:: with SMTP id u10mr27541386plu.122.1543274192071; Mon, 26 Nov 2018 15:16:32 -0800 (PST) X-Received: by 2002:a17:902:1d4a:: with SMTP id u10mr27541345plu.122.1543274191316; Mon, 26 Nov 2018 15:16:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543274191; cv=none; d=google.com; s=arc-20160816; b=jVrcuECadkiAazb5JydiFCPyXdKxN9IS1O65lVRUrj+nwIkxhvfFX/eDqWSQU47yE/ wxw2OA+70PgOOuQ5seW9rM4GJmD3t+QzmhabHQ4dFCy35YS1DdQt7LOzX581sdInxYxo zv5hStqkZ4Q5kgLt5z1dmI+M5ifFUXbmAGN63asWMlDAUyqLlzdUH8lLPHHD8ziPN8ns FtatWHUDGsmJtx0W17Q44e4eURvKD9NKzjIMeNyQ1rf1xNKDsFmhW0uZ2dKGtquNITHs Beei0KSdi8GGZe64/0pX6fQezEQQkEWJem4g2TT8a6xMj4NRVPl1k9V9kwsgtRpEppM+ 4Abg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:message-id:subject:cc:to:from:date :dkim-signature; bh=rGPpra3RUdchVXXSTigsvpJv4NP8t4KkKUk8/ihis1I=; b=EmP1QQS4ZA6F7cBRhR2/uyUBKbRK86vdbww+eyUwkGfcQemYGxCnnc7hZoaro/leLV yZYbuK9m7h96fLkt0QKb5lMF0dC0ca8NAE+5cIvGMN7rAUQeO9uRGmKDOjB9b1jmRfV5 +y7/ukdqElR9bCwtPO7BnAmKDp8QwHJkn1EIsejGOEwsDfomgOYDNcjhkoNIgGBRpHIm qVCdqbr+83EILOGo+UsfT3prjwa0OLBFN2T8OZOQFWmjLK9UcEdVuCBLmYSA85FnLWJB xJ0bdMsUwlSXg3zby3RqJ9wxRi8evjpEx7IalFMS3vCGv+4jjdED/0LBa2JDtWJfPjZH bS3w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=FbNe40Ki; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id b13sor2463908pls.16.2018.11.26.15.16.31 for (Google Transport Security); Mon, 26 Nov 2018 15:16:31 -0800 (PST) Received-SPF: pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=FbNe40Ki; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:user-agent:mime-version; bh=rGPpra3RUdchVXXSTigsvpJv4NP8t4KkKUk8/ihis1I=; b=FbNe40KiPhIzrDnOald8UymMf5PXD3KU9mANbO/aChyAxRoURMsdWbTV6+MitU4c40 Bk3ryNwoW3EDN225NRbHLp+zVyJFyivUtRkbKbooR+nL2oBdpMdyfxV/u+FJPGbNEoBF eOkralF2RSaLT8JikaM9vGu7gxwPi6qSRUm9ZhVxoRJLwyl6Ng1VMtoCwJfNJvLHIbCl IiAo7Wmv7Em8SXkodUA4azgfVRyR8LHcdSm1xRoaE83Bq7CCAbjI+BspeZHEwGTsljfm sLbb17KjLQVWUfdQR7LXRS9OVmLjCbP3pK7mN2+lvv9MEAEW8Eglp2Wa6Nlnn6+pD1+F Em3g== X-Google-Smtp-Source: AFSGD/VKU0bctJI+lmzailmyZ2EBHMkvF5udps/y++mxDIvNQLcUMAiDEFxDtnADtaokfFN64+Waew== X-Received: by 2002:a17:902:161:: with SMTP id 88mr14895455plb.306.1543274190544; Mon, 26 Nov 2018 15:16:30 -0800 (PST) Received: from [100.112.89.103] ([104.133.8.103]) by smtp.gmail.com with ESMTPSA id p2sm1414716pgc.94.2018.11.26.15.16.29 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Nov 2018 15:16:29 -0800 (PST) Date: Mon, 26 Nov 2018 15:16:28 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Andrew Morton cc: "Kirill A. Shutemov" , "Kirill A. Shutemov" , Matthew Wilcox , Jerome Glisse , linux-mm@kvack.org Subject: [PATCH 01/10] mm/huge_memory: rename freeze_page() to unmap_page() Message-ID: User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The term "freeze" is used in several ways in the kernel, and in mm it has the particular meaning of forcing page refcount temporarily to 0. freeze_page() is just too confusing a name for a function that unmaps a page: rename it unmap_page(), and rename unfreeze_page() remap_page(). Went to change the mention of freeze_page() added later in mm/rmap.c, but found it to be incorrect: ordinary page reclaim reaches there too; but the substance of the comment still seems correct, so edit it down. Fixes: e9b61f19858a5 ("thp: reintroduce split_huge_page()") Signed-off-by: Hugh Dickins Cc: Kirill A. Shutemov Cc: Jerome Glisse Cc: stable@vger.kernel.org # 4.8+ Acked-by: Kirill A. Shutemov --- mm/huge_memory.c | 12 ++++++------ mm/rmap.c | 13 +++---------- 2 files changed, 9 insertions(+), 16 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 55478ab3c83b..30100fac2341 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2350,7 +2350,7 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma, } } -static void freeze_page(struct page *page) +static void unmap_page(struct page *page) { enum ttu_flags ttu_flags = TTU_IGNORE_MLOCK | TTU_IGNORE_ACCESS | TTU_RMAP_LOCKED | TTU_SPLIT_HUGE_PMD; @@ -2365,7 +2365,7 @@ static void freeze_page(struct page *page) VM_BUG_ON_PAGE(!unmap_success, page); } -static void unfreeze_page(struct page *page) +static void remap_page(struct page *page) { int i; if (PageTransHuge(page)) { @@ -2483,7 +2483,7 @@ static void __split_huge_page(struct page *page, struct list_head *list, spin_unlock_irqrestore(zone_lru_lock(page_zone(head)), flags); - unfreeze_page(head); + remap_page(head); for (i = 0; i < HPAGE_PMD_NR; i++) { struct page *subpage = head + i; @@ -2664,7 +2664,7 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) } /* - * Racy check if we can split the page, before freeze_page() will + * Racy check if we can split the page, before unmap_page() will * split PMDs */ if (!can_split_huge_page(head, &extra_pins)) { @@ -2673,7 +2673,7 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) } mlocked = PageMlocked(page); - freeze_page(head); + unmap_page(head); VM_BUG_ON_PAGE(compound_mapcount(head), head); /* Make sure the page is not on per-CPU pagevec as it takes pin */ @@ -2727,7 +2727,7 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) fail: if (mapping) xa_unlock(&mapping->i_pages); spin_unlock_irqrestore(zone_lru_lock(page_zone(head)), flags); - unfreeze_page(head); + remap_page(head); ret = -EBUSY; } diff --git a/mm/rmap.c b/mm/rmap.c index 1e79fac3186b..85b7f9423352 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1627,16 +1627,9 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, address + PAGE_SIZE); } else { /* - * We should not need to notify here as we reach this - * case only from freeze_page() itself only call from - * split_huge_page_to_list() so everything below must - * be true: - * - page is not anonymous - * - page is locked - * - * So as it is a locked file back page thus it can not - * be remove from the page cache and replace by a new - * page before mmu_notifier_invalidate_range_end so no + * This is a locked file-backed page, thus it cannot + * be removed from the page cache and replaced by a new + * page before mmu_notifier_invalidate_range_end, so no * concurrent thread might update its page table to * point at new page while a device still is using this * page. From patchwork Mon Nov 26 23:19:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 10699327 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9780C13BB for ; Mon, 26 Nov 2018 23:20:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 842382A50E for ; Mon, 26 Nov 2018 23:20:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 715BC2A51A; Mon, 26 Nov 2018 23:20:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 810CD2A50E for ; Mon, 26 Nov 2018 23:20:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8C8896B4455; Mon, 26 Nov 2018 18:20:01 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 853146B4457; Mon, 26 Nov 2018 18:20:01 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F3E06B4458; Mon, 26 Nov 2018 18:20:01 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) by kanga.kvack.org (Postfix) with ESMTP id 245186B4455 for ; Mon, 26 Nov 2018 18:20:01 -0500 (EST) Received: by mail-pg1-f198.google.com with SMTP id q62so8793654pgq.9 for ; Mon, 26 Nov 2018 15:20:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :in-reply-to:message-id:references:user-agent:mime-version; bh=khDn23SWH+VCa5WnDaGtQ9G0p+QCH9yXW3GVZzGGRA8=; b=d/iIXqN3y0wx225xa5ZN0x5TnTa1KlLlntcvRHCFJA/TSLtQD2thnqnLqbFk+p8Uwt QpyPlG2hNHKnhir/KonQfnKxmZAc+LQ4rn5mc0+S3KwkpPcdKEZyKJaVW4ZTGYF597t5 JpqFBkh6CAMKy8BH+ZYeNhRRo8b/4URWU6uMGYJoKIKUT+bTwmksAgxMmVAAdTsr72Es wW8UgqptdVBXwNJBxjdnkCHNRjr8gKA+mk8wgGTenL8APRgsDCmrRF/H0FjtUtsDOksB mS4rpYFJdqTqGSo2AmPIw+kMKGsIinxlSFMlJo1TrpcSuJXoXSec2EMICC290WqkmlDx Oq0Q== X-Gm-Message-State: AA+aEWYUjUSpTe2NIU/IkLjk1FxV16sNgX5FhvcQtHc25tK+gAb7ZFjP MqhVS7TMOYOO8kmbGVun2iZheIBe7arLZrxLUtBJDVVs0sMmTybAlM0gOAJrn4k7ujCqwNjnWWP 4oR5tvL1vDFd1vfLtKWgVq4G2AhiZEeFlzajR4br6IeqCZz104YF+DIzSe3B6k5YjxOY8C5S31o F88Ns7f/ILaEkUhi1k/Y8XTbnAvIN569gyQifer204cRxfgw5G/uGFUqumy/pbM2ALtgyxIA4ja XbDnsZJPExDNtoMsMVewms//mG+qcLTEeiYvwNgNSf7Xr+1RFC5FKY+d2R1yX0WWNUYVcbXv/si 9dZUFISNG7m7PZ33EUfm26JWPW9JV4TZ0iIETCJVctbIzvjo8eyzMJIuyFddyYwrrV7aVnBvgm+ S X-Received: by 2002:a17:902:b60a:: with SMTP id b10mr28284607pls.303.1543274400783; Mon, 26 Nov 2018 15:20:00 -0800 (PST) X-Received: by 2002:a17:902:b60a:: with SMTP id b10mr28284582pls.303.1543274400147; Mon, 26 Nov 2018 15:20:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543274400; cv=none; d=google.com; s=arc-20160816; b=xD1gsGob+mow7BFJa8gSm7XNWTSBAjarb1PbH1g+XPUpLied55bu0B/2LQFek4w30L GZ/PKbtqYAiGKUXmvoimoMkc1+u6FIg3ucxOS9iAR7QcctzuI/6wMUZeucOPsOrkj5dy Q6hp6YoxdU3XpF8DKBJaK9tFDN956sZ7B7be1TKwwdgr76Uo5rPqzkJE1dqG+nFKsCEm ennU1ViXWSD0yvKloRSIRVfd+yatz81ax2EDIpRK+4vjrzp1GMFuicJODlhvYIW/Qn+P 4NKiQF17EUFsFX9t8n/UDufbh1wwzYDK+TUwUqq3bUQD8sBO/h2d53snn5CB9SQ80VZE v3Sw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:message-id:in-reply-to:subject :cc:to:from:date:dkim-signature; bh=khDn23SWH+VCa5WnDaGtQ9G0p+QCH9yXW3GVZzGGRA8=; b=r7ki3svttMhQUnzgU+4Ifr7iRWR9X9/56+lP03SNVs3AuGnZNt+C/TO4W8vyXB6QaW 1UfmI1sOtkMONrlpPrYWACg0q75TXO+CGsmlT3GmfQbXftfr0P9EujT0yQ85+Owd0lS7 6O5R5G+L+DXO0VA1XfWCHaPhIw/EQM9XHxVMWM712BGoSjj/wJUA5QKqlkRsvdSeTh12 H/p7yu/E3mj+xcVGcl3f7Lpk9W/jvJcnD2a+TB0Ln6X1w3LRseiHxqgF9RfOaXXqbTF5 y700+cEBqziBtCVq2NDYCusEpkBDX1Z9CLGaaauAHBvgychGcVpNpNzriLZSNfxz+X6A Or1Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=OKedTm5U; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id n13sor2812115pfj.12.2018.11.26.15.20.00 for (Google Transport Security); Mon, 26 Nov 2018 15:20:00 -0800 (PST) Received-SPF: pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=OKedTm5U; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=khDn23SWH+VCa5WnDaGtQ9G0p+QCH9yXW3GVZzGGRA8=; b=OKedTm5UXv0BNAxfE+pLCb7261S0fsMpfSMQl9R2MbYMvd/xjMLSD7zEpowCyvENqo h9YJlbHRgkJAZ3QmcUhR8aLtM/DrIAU7oJ1j1fDaQ++7BAD95sPDunJiMx4hB7JaB5V7 bKMY+JtkszjJnHVLj7Zsyl7HQJP8OEMGZ+7OWxA1EQ5Hd5Bfc6i2vEzmG2VTS8va+g7T 2zDICEWeLTPZUA4OusZ3C139AehglM4HPXtl3ilBdLXh7msSLoiLFNFP5FdYoriRIdMt iAiySEZFNE8yWzZMoIPe8iyzrKIxLhElfZ2Pt0rZIwVu1q1TsFDEUicS8QtXOn9Re9vM K7dg== X-Google-Smtp-Source: AJdET5dE8PEPcQo4WzICFBQ3PNPT+VfmhlobQluDLeVMuPdjvAcs17I+LLUfRL+mDeURYK09tE58sg== X-Received: by 2002:a62:c21c:: with SMTP id l28mr30018943pfg.74.1543274399402; Mon, 26 Nov 2018 15:19:59 -0800 (PST) Received: from [100.112.89.103] ([104.133.8.103]) by smtp.gmail.com with ESMTPSA id x27sm2492472pfe.178.2018.11.26.15.19.58 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Nov 2018 15:19:58 -0800 (PST) Date: Mon, 26 Nov 2018 15:19:57 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Andrew Morton cc: "Kirill A. Shutemov" , "Kirill A. Shutemov" , Matthew Wilcox , Konstantin Khlebnikov , linux-mm@kvack.org Subject: [PATCH 02/10] mm/huge_memory: splitting set mapping+index before unfreeze In-Reply-To: Message-ID: References: User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Huge tmpfs stress testing has occasionally hit shmem_undo_range()'s VM_BUG_ON_PAGE(page_to_pgoff(page) != index, page). Move the setting of mapping and index up before the page_ref_unfreeze() in __split_huge_page_tail() to fix this: so that a page cache lookup cannot get a reference while the tail's mapping and index are unstable. In fact, might as well move them up before the smp_wmb(): I don't see an actual need for that, but if I'm missing something, this way round is safer than the other, and no less efficient. You might argue that VM_BUG_ON_PAGE(page_to_pgoff(page) != index, page) is misplaced, and should be left until after the trylock_page(); but left as is has not crashed since, and gives more stringent assurance. Fixes: e9b61f19858a5 ("thp: reintroduce split_huge_page()") Requires: 605ca5ede764 ("mm/huge_memory.c: reorder operations in __split_huge_page_tail()") Signed-off-by: Hugh Dickins Cc: Kirill A. Shutemov Cc: Konstantin Khlebnikov Cc: stable@vger.kernel.org # 4.8+ Acked-by: Kirill A. Shutemov --- mm/huge_memory.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 30100fac2341..cef2c256e7c4 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2402,6 +2402,12 @@ static void __split_huge_page_tail(struct page *head, int tail, (1L << PG_unevictable) | (1L << PG_dirty))); + /* ->mapping in first tail page is compound_mapcount */ + VM_BUG_ON_PAGE(tail > 2 && page_tail->mapping != TAIL_MAPPING, + page_tail); + page_tail->mapping = head->mapping; + page_tail->index = head->index + tail; + /* Page flags must be visible before we make the page non-compound. */ smp_wmb(); @@ -2422,12 +2428,6 @@ static void __split_huge_page_tail(struct page *head, int tail, if (page_is_idle(head)) set_page_idle(page_tail); - /* ->mapping in first tail page is compound_mapcount */ - VM_BUG_ON_PAGE(tail > 2 && page_tail->mapping != TAIL_MAPPING, - page_tail); - page_tail->mapping = head->mapping; - - page_tail->index = head->index + tail; page_cpupid_xchg_last(page_tail, page_cpupid_last(head)); /* From patchwork Mon Nov 26 23:21:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 10699329 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 501BC13AD for ; Mon, 26 Nov 2018 23:22:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 328E82A591 for ; Mon, 26 Nov 2018 23:22:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2358A2A63A; Mon, 26 Nov 2018 23:22:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D62DF2A591 for ; Mon, 26 Nov 2018 23:22:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 03EEC6B445A; Mon, 26 Nov 2018 18:22:03 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F2FDD6B445B; Mon, 26 Nov 2018 18:22:02 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD1276B445C; Mon, 26 Nov 2018 18:22:02 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) by kanga.kvack.org (Postfix) with ESMTP id 975AF6B445A for ; Mon, 26 Nov 2018 18:22:02 -0500 (EST) Received: by mail-pl1-f197.google.com with SMTP id p3so2415831plk.9 for ; Mon, 26 Nov 2018 15:22:02 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :in-reply-to:message-id:references:user-agent:mime-version; bh=Sd48llQOR4D/qyL1g9Jnl5MHdkb3/UlhYavmB9jv1dY=; b=CAVcEt6IhttbJQu1If2OSmG3mmV55g9G7T+jDDOLfQ4kuTzQxGvi1dRyJjle2OTL3N VJYYpbyStRNMv29LiKzPWhGJlHkAJtr/2/F1EHNLal6VwHJVWoKy2Koxeb7frLbEOIoH xzdU4c9ly3t6cW1dT/y0Eg9WpfyWhwXKa75myon7hau9MzfZsu+JSyuYCLRTGgYVE+BL RQrvfgxbvbrfSKqXOkZ9MlHNqDdOFFPeAkNMYlPmdWwrhvg1JgW6jYBwxldSZhfGJC0i zzXzcyeT1QBeMVsGMhlqoNmBUebE9adsJIoSKWQLVQKsv+pxnx/ioZiquLlgFPbdZZXW uZNA== X-Gm-Message-State: AA+aEWYOAxO65Ep57qt7R9varZbY6lMrUemFWfWmPGx4XJvBZIXeF/kc +QejAFT8eEH8rtFPMkoyefzmfXJYyQBQ9lFxyKpL48sQAAvYu4D9eV2hJLTza3/7Z1d7GeLDymX gVh9cU6ZqLEl/X4p7kUJ/yo0Dufa3+HRmaboWSS2CrkWcvKslp11RsBbbMFx+xqF7tYXQABTOSW FgSf3Lhyc6BpmroIrpUHGOQjX8NsGqk9hVccJfG91L7s4uxwLiLTvTNJ/H9RtZ/QMo4H2JxYGZi Kn4T6jNRsELNwwmmN7M03eWOFwv95sKYXRATNY+kF1R9XBrFn8q8Gn08neDQ86uQpgQu9DLju7b EKCVrqmmuHPl+u7snzc8d1HMqsA99X4Dc1Sglq648pAOUTcCqztuFN2AGLpN+of45G1h+0zwmBa n X-Received: by 2002:a63:f615:: with SMTP id m21mr27574257pgh.428.1543274522269; Mon, 26 Nov 2018 15:22:02 -0800 (PST) X-Received: by 2002:a63:f615:: with SMTP id m21mr27574223pgh.428.1543274521504; Mon, 26 Nov 2018 15:22:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543274521; cv=none; d=google.com; s=arc-20160816; b=IMBFdfZkUn145ApafN26QkLCik3xEKiy9tOJUGTVLNhLJKwgEdSgDfonzuTpp++PpK VAt+Cy0ERSH7IVbzYHTBnc5UxOhBtBd4Y+M1mJrf7yFc573a1yjbjSSsLU8XlrMkMTEi VI+2SE1mgdaxb+7jxayDlW0wUQYIrW4Jmr6Gduas+1BJZDJd8EVGMBEJFcYjQa1jd07V T3AmNk4TM3U9et/dE9PnP4qMTNlddaChucSJYHElPdnDLz4L7U9demM1ZnH4hdKkGlA3 fVNQ2njz2dPtoN1BeO6HCryPRm+7+6e+fJDqyzB3OW54tkXtO9An79lcFD2SlpVHqFAY 2jzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:message-id:in-reply-to:subject :cc:to:from:date:dkim-signature; bh=Sd48llQOR4D/qyL1g9Jnl5MHdkb3/UlhYavmB9jv1dY=; b=xCxj08AssJjt+9MkHvqmx799hzUNp/RUybZXjga+ospoU9Ct1LDcjKzmQSq8RFCIlh 6sN3rpPVfnH2Mv8R6DwtXhipmgoK1smXXk+j/LhMZiJmpQvfSHTBqIKgTaHkp6dVHpg3 jOJpIBUAjkVXSM/4mdN6rJ84Ed1L7wKcS+Nud2nyrwoCGtM0zYyodjc+Tx5L7V5JP9hz UcrnpTy4I8e0NAHVvpnyYHKRN9hFOgSXhdyWQDg9r4DNyNqLA4FP3RQYBxe6P8h8YT/N 8/KAyWQzZWoruJd1xRHsx/OD7S5bS0lD/JEuFvyz/IYXExN/cNwuy/0knekOtCKAO3uP QHxw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=l32BeqjW; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y73sor2781442pfd.54.2018.11.26.15.22.01 for (Google Transport Security); Mon, 26 Nov 2018 15:22:01 -0800 (PST) Received-SPF: pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=l32BeqjW; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=Sd48llQOR4D/qyL1g9Jnl5MHdkb3/UlhYavmB9jv1dY=; b=l32BeqjW/6g8fnU+g/0mZEfadbn2Fmd3vxypVj/uXzB2jVkSHupm3+53sy9mKGo1zP 2u7F9sXIJsHzwsl4+a4QYNHv026yTjo6LE5sB0npBN/efc1YNNPcTVDgLnehYg60U+54 KKweQLAV3dOpuFO13GJ5RvxpjbViPFQX5HM+HkWEORSChqaNSfMoGRpD3wZ8S628bpfP 0LBqXH/JSF7HnA5jH05k4CyBE2S1dmanjJ420xyuVhNIkD9tB7X/PMQfaZcW1S02nRQB qjvGV5xrwSqPNbzxGFqOYE2NsfN7kGQe/tpKSgvDbmpyURnazLPrdKpFyVD4dXgdRjpc qokg== X-Google-Smtp-Source: AJdET5f94N7DQav0rdhXNHF8c8uVfneBZ69jde1doHH2ayYG/EVeSeiQb0YsRsxV5b6iVqf1qYGLDg== X-Received: by 2002:a62:798f:: with SMTP id u137mr30481824pfc.168.1543274520560; Mon, 26 Nov 2018 15:22:00 -0800 (PST) Received: from [100.112.89.103] ([104.133.8.103]) by smtp.gmail.com with ESMTPSA id v76sm2094359pfj.145.2018.11.26.15.21.59 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Nov 2018 15:21:59 -0800 (PST) Date: Mon, 26 Nov 2018 15:21:58 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Andrew Morton cc: "Kirill A. Shutemov" , "Kirill A. Shutemov" , Matthew Wilcox , linux-mm@kvack.org Subject: [PATCH 03/10] mm/huge_memory: fix lockdep complaint on 32-bit i_size_read() In-Reply-To: Message-ID: References: User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Huge tmpfs testing, on 32-bit kernel with lockdep enabled, showed that __split_huge_page() was using i_size_read() while holding the irq-safe lru_lock and page tree lock, but the 32-bit i_size_read() uses an irq-unsafe seqlock which should not be nested inside them. Instead, read the i_size earlier in split_huge_page_to_list(), and pass the end offset down to __split_huge_page(): all while holding head page lock, which is enough to prevent truncation of that extent before the page tree lock has been taken. Fixes: baa355fd33142 ("thp: file pages support for split_huge_page()") Signed-off-by: Hugh Dickins Cc: Kirill A. Shutemov Cc: stable@vger.kernel.org # 4.8+ Acked-by: Kirill A. Shutemov --- mm/huge_memory.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index cef2c256e7c4..622cced74fd9 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2439,12 +2439,11 @@ static void __split_huge_page_tail(struct page *head, int tail, } static void __split_huge_page(struct page *page, struct list_head *list, - unsigned long flags) + pgoff_t end, unsigned long flags) { struct page *head = compound_head(page); struct zone *zone = page_zone(head); struct lruvec *lruvec; - pgoff_t end = -1; int i; lruvec = mem_cgroup_page_lruvec(head, zone->zone_pgdat); @@ -2452,9 +2451,6 @@ static void __split_huge_page(struct page *page, struct list_head *list, /* complete memcg works before add pages to LRU */ mem_cgroup_split_huge_fixup(head); - if (!PageAnon(page)) - end = DIV_ROUND_UP(i_size_read(head->mapping->host), PAGE_SIZE); - for (i = HPAGE_PMD_NR - 1; i >= 1; i--) { __split_huge_page_tail(head, i, lruvec, list); /* Some pages can be beyond i_size: drop them from page cache */ @@ -2626,6 +2622,7 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) int count, mapcount, extra_pins, ret; bool mlocked; unsigned long flags; + pgoff_t end; VM_BUG_ON_PAGE(is_huge_zero_page(page), page); VM_BUG_ON_PAGE(!PageLocked(page), page); @@ -2648,6 +2645,7 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) ret = -EBUSY; goto out; } + end = -1; mapping = NULL; anon_vma_lock_write(anon_vma); } else { @@ -2661,6 +2659,15 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) anon_vma = NULL; i_mmap_lock_read(mapping); + + /* + *__split_huge_page() may need to trim off pages beyond EOF: + * but on 32-bit, i_size_read() takes an irq-unsafe seqlock, + * which cannot be nested inside the page tree lock. So note + * end now: i_size itself may be changed at any moment, but + * head page lock is good enough to serialize the trimming. + */ + end = DIV_ROUND_UP(i_size_read(mapping->host), PAGE_SIZE); } /* @@ -2707,7 +2714,7 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) if (mapping) __dec_node_page_state(page, NR_SHMEM_THPS); spin_unlock(&pgdata->split_queue_lock); - __split_huge_page(page, list, flags); + __split_huge_page(page, list, end, flags); if (PageSwapCache(head)) { swp_entry_t entry = { .val = page_private(head) }; From patchwork Mon Nov 26 23:23:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 10699335 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EC60713AD for ; Mon, 26 Nov 2018 23:23:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D914528574 for ; Mon, 26 Nov 2018 23:23:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C748A28536; Mon, 26 Nov 2018 23:23:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9728228536 for ; Mon, 26 Nov 2018 23:23:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C71EA6B445D; Mon, 26 Nov 2018 18:23:40 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C20B26B445E; Mon, 26 Nov 2018 18:23:40 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B37146B445F; Mon, 26 Nov 2018 18:23:40 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) by kanga.kvack.org (Postfix) with ESMTP id 730786B445D for ; Mon, 26 Nov 2018 18:23:40 -0500 (EST) Received: by mail-pg1-f200.google.com with SMTP id s27so8827838pgm.4 for ; Mon, 26 Nov 2018 15:23:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :in-reply-to:message-id:references:user-agent:mime-version; bh=9cldGFQbC7ugVQhpvHUWhG/FaC/+yZV4xhPFMtVf1Ag=; b=KyxoKbp4ElVr5muLusauXeiRviGeYTnBUl79CNPrURfHhFfcvlN1dZmXRd0JeqFTHW AypHby+luKaK/5ds47fsbyaYSiSbg6UnNqGeZJRq4FxqYeKZrvlL2K2azONmElhnrkic 6i0EBfL+U/mGtmn3/Ihjx9deMuda+/9XmsyHRotO7aDNLXvzeGLUN4wbuozSHrUWgcm9 jZr/3+a50x6sNhyNe0S7YBWU2c84pRMT3/M+0ujg9n/IchtHiokh0otphEqIffm2aBoY T8yJpK/mH7k8XUwhMMhccDJ+6E9OI7G18JvQjgoVDHGEWnr+y/yh0Xv/O234JKE96axC xTjA== X-Gm-Message-State: AA+aEWZpxdZJ77lRZjMIr9dYuEeDFCPkAIa96tAEEbJyGhl0c7fjv+xv wnKKDsz1bNjp1sHx2RrBQJdu4BgNRdbXw8GvtDtqDWk0suuwTEmdCnL9za75Csg8tRlw6KwubM6 0m7hHuP6CRYepgNEL8qOjR+o4+SFelCQQr/up9Ai3pfIlvTKTJ4rwEIEw1Sk7fYSCzcYYurOX/0 0YMQEJcFQ1CLIGnzQZIZX6u+aF2NekgW03e9bcuRml56k+RZFj4w65e1mWMm9LXo+hLfTcMHsn8 8gPgjMjmHuGdHSYZq/jfhG/eMXU8Rnsr/wagArJx938wiKzDmxVtLb8x3MNM1VYIsC9hBDoYwN4 OyXi5CBEZGLbavwxhQezX1J2d8erJJSrdMtaVEz8cFf4/OojkPC4q7P0aL3mOEbnOUorfDerhmB y X-Received: by 2002:a17:902:b595:: with SMTP id a21mr17330042pls.120.1543274620144; Mon, 26 Nov 2018 15:23:40 -0800 (PST) X-Received: by 2002:a17:902:b595:: with SMTP id a21mr17330014pls.120.1543274619483; Mon, 26 Nov 2018 15:23:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543274619; cv=none; d=google.com; s=arc-20160816; b=0nvqvAfMMHTAEE5tkdp3uzr39JTrSZN4OzPJWtoiBmgQHcCXI4lVHJPf7DDOB77m1A rdIjsBjcBRi4BuPh6/2Ez+2QZmBqXtTjiY90JC4Vgft/vFs9G0ysczLZMpA150EO8zCd MlPgM5BY6RpkmMIWWDUenHF3x/MSuhQWLuUD/AKxjZiPQ4WVWNHVIoSMyLLKjBGg+Cmw GQAemNg7Vb+LSsiCkynLVUZYbcX9/xDIgYdoUJNByYE0fPVQ9dRwvIUwd68vHefUJ2G6 iLxL79yUFsDsK760FWE7lK2gBORGeHbBjkUNKEgSXkr6Z1dLHjJf3tGEgwUGsg9+U7oN 6vgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:message-id:in-reply-to:subject :cc:to:from:date:dkim-signature; bh=9cldGFQbC7ugVQhpvHUWhG/FaC/+yZV4xhPFMtVf1Ag=; b=VhkRLFMmu0ygTgAZrBpM9DG3+eYjMIr0Mw+C3oYvK10Qj4zej83Rchs1huJWT2oyNW HnL5lVzsXxdv3IQnvQDOAhQChHDZ8bLWsS9D0sx6emWeP5uaUbJ0YIj6se6uqY6QXSBF v5SL4B2qCDG5ndRXE/hOM3VOqGAI3Kbdxhyw1/v94v7aS2X67IO+aE6f1zwSr2Z0GO0i 8IynoKquB5T8BfJ1SjVxJqK8Y0uXz6Osr/yfWWWjz+3qKjhTTfgqtKLe+/EJDaF2WyAO UJY7JCmKmg27EUZDIdRSA0Ul7Au+1uAQxxV46piGrFs1w2roAV/VxxH5xeeiichygx7z zXLA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=PoORlVPT; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r12sor2512417plo.58.2018.11.26.15.23.39 for (Google Transport Security); Mon, 26 Nov 2018 15:23:39 -0800 (PST) Received-SPF: pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=PoORlVPT; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=9cldGFQbC7ugVQhpvHUWhG/FaC/+yZV4xhPFMtVf1Ag=; b=PoORlVPTllS4vjsb7s92Iy1DBeDXNsgWTOPmFyqlf3qFTjneNHG3pRALYXmn89V7gW +lXRpg2mSpvFRe0U3BYsqpeNnHSxvMjxU4Fx0XX3m3cFPVQt5f/wkFVhqLiVrrT2OV5a QeYFum0dTEUT0UJbMCSjuYtzks5Qx2xXNuihRRfooHyuYoB2MvHzX4eTi88Cx8MLJvRP 3d6WxZFvdzc1zoHHXyCg7HqhPJNylo/H4N25Mhjy9YY/poA4HFqcbby38/EG+s9XO2Gi vJQvo79hnBL9gGr7g6VYq7Igd/jfdPofUVIbbVkLzTRGLHow5LwK6TLV6I9kZQoy8C5d sK+A== X-Google-Smtp-Source: AFSGD/Xoopmx/Dzvh24KYesC4RcW1ax4aSv4v6Hw/eD8IpqltZMmC3GYySoE75EvRmM8LbY46/uJaw== X-Received: by 2002:a17:902:6bc3:: with SMTP id m3-v6mr10621997plt.106.1543274618778; Mon, 26 Nov 2018 15:23:38 -0800 (PST) Received: from [100.112.89.103] ([104.133.8.103]) by smtp.gmail.com with ESMTPSA id t12sm1949179pfi.45.2018.11.26.15.23.37 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Nov 2018 15:23:38 -0800 (PST) Date: Mon, 26 Nov 2018 15:23:37 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Andrew Morton cc: "Kirill A. Shutemov" , "Kirill A. Shutemov" , Matthew Wilcox , linux-mm@kvack.org Subject: [PATCH 04/10] mm/khugepaged: collapse_shmem() stop if punched or truncated In-Reply-To: Message-ID: References: User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Huge tmpfs testing showed that although collapse_shmem() recognizes a concurrently truncated or hole-punched page correctly, its handling of holes was liable to refill an emptied extent. Add check to stop that. Fixes: f3f0e1d2150b2 ("khugepaged: add support of collapse for tmpfs/shmem pages") Signed-off-by: Hugh Dickins Cc: Kirill A. Shutemov Cc: stable@vger.kernel.org # 4.8+ Reviewed-by: Matthew Wilcox --- mm/khugepaged.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index c13625c1ad5e..2070c316f06e 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1359,6 +1359,17 @@ static void collapse_shmem(struct mm_struct *mm, VM_BUG_ON(index != xas.xa_index); if (!page) { + /* + * Stop if extent has been truncated or hole-punched, + * and is now completely empty. + */ + if (index == start) { + if (!xas_next_entry(&xas, end - 1)) { + result = SCAN_TRUNCATED; + break; + } + xas_set(&xas, index); + } if (!shmem_charge(mapping->host, 1)) { result = SCAN_FAIL; break; From patchwork Mon Nov 26 23:25:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 10699337 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8B4F613BB for ; Mon, 26 Nov 2018 23:25:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 787432A5B2 for ; Mon, 26 Nov 2018 23:25:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 676BF2A63A; Mon, 26 Nov 2018 23:25:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D9A392A5B2 for ; Mon, 26 Nov 2018 23:25:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE22E6B4461; Mon, 26 Nov 2018 18:25:05 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E91696B4462; Mon, 26 Nov 2018 18:25:05 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA80A6B4463; Mon, 26 Nov 2018 18:25:05 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id 9BA216B4461 for ; Mon, 26 Nov 2018 18:25:05 -0500 (EST) Received: by mail-pf1-f198.google.com with SMTP id 74so12513874pfk.12 for ; Mon, 26 Nov 2018 15:25:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :in-reply-to:message-id:references:user-agent:mime-version; bh=nCRmhahI7kY4j643ILpESoqKGQRCrmHSVJ/3uA5AXlw=; b=rFLdPWsS5QgwFJ5YQNmVqu1xgkwzpp+0yeAIIL7+iCqW9sLnwDZhVyG1H9vwps+B+4 ahi4SwmCQIRdjqUu/Qrd11m9qio/nnOzY6REGlzZKLsfmip0jzYf1U5s5Q1gKyhN6jqd 0D91pVAFDMbKj1+39dEMO3x9CjvnLWSpSh1zP5j0jULk03SvQgIKultKFv6wmvgQaloC a61Kzbj99UFqt/OAyBD26xUJ3KQQCAAGvDAxDknQdnswWy7roDaO+9euFWdOD7YtGh+u LsEd5jOo4d/ssH37XxemonacWITH304LtIFQ9UJGj55ufxUiEMNphdyH25vgeL7pVEnb Ssmg== X-Gm-Message-State: AA+aEWa7uwGmBFwmvr1AX3QNLrcOG6gPfVZK2b+Zg/pt3AzBshXh4Oy5 3whp/aoXs0N1LQJYyGfzMk71vDuaiGq9KZqtiQ1TmfVpuc+tMpVKpZrKoC6koFGqwtrNDBdLK8w hCXVo3a3EFaOaIWfdJDTTXByMPEWyWKaY1Clw8oEbVaSBwj0mbc+qf8waAAhfOGor1eC9Om8VQk o5sL4gemHcEh+Rxo59RKiffkSLtnLPEAyzA5KsGOOO26Ft7+G5LulZgfkCDKUBf+RfEoFIqlZp5 B5JJIY/YrV9tJRHRUlDcsTcpb3bQJtNfn3LB8pcVkhDizUwsQ/7paJsEmGvg9qAJHsSF0t47ZPF EvnXmkXE8pGL50Be+7uzk6nPvf0fOuKqrzXyAyk2jWp+GXzDych6ZJkfIMPCE19WV020VzsjT9L y X-Received: by 2002:a65:64c8:: with SMTP id t8mr26583563pgv.31.1543274705293; Mon, 26 Nov 2018 15:25:05 -0800 (PST) X-Received: by 2002:a65:64c8:: with SMTP id t8mr26583540pgv.31.1543274704595; Mon, 26 Nov 2018 15:25:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543274704; cv=none; d=google.com; s=arc-20160816; b=E3os14fr5rdgotCmMBJzKc4nSD4LJTLaBQqlt4LJVaj4Sg7REsccpbm4cU6SPBokmm a7ITr850dgSZDEKnf+rdOg7ZxJ71tDY7Ncr4k7P36RQj3GJMhA14B7E4MYf5AUcQ6hiO NMEjjDl+FiTg3ABljas/e81ySszqqZcOhuCELwm2HAXKW3XQapJbH2JZrXnsczr67OL+ tEvbxAveOcpzyYGe+T1CXVZE/tcr1Z58m5OoYec800+T4GaRkfnXTO4Z62LJE0x63HJi uxlGg3av+CYcV+WzwrNVQ9N8HM/A3Zlkvudthl9mwPH+qx/zGCpfWF94uxEJ+GIPz6Dv dFqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:message-id:in-reply-to:subject :cc:to:from:date:dkim-signature; bh=nCRmhahI7kY4j643ILpESoqKGQRCrmHSVJ/3uA5AXlw=; b=IH22i+3mcwXL2uA14ZNuTxN9ICDwQWjBR/zGf6wv680Mrs2jfxDZb19VBa8fhjZ9Rc Hk1H9fpAr9/baSYiza92lO8xUWIQfDp8wJYRRjC6eh8aVJJlHakrrNO0HW10veEvmemd GO6Kyta6wg4erkEwjlKkDXQf2XzpJM/aSdDQ/UXlL8n+Me3htVb7PwpP0Hk2O0gyoGCt xZXWCKj38LAPUxaEzyZWciuc2mAK3JLGt2BhasD82OtzyamsvVHt1JjFDSy1XQX/eKMF ZElQmjbaFZXKuomDTRuEIy2dNodC1AyKxLx+kdZqDHIey0URHs+dagvs95X3Y2ks1n+c RV5Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Pdtzoav8; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id h191sor2685046pge.48.2018.11.26.15.25.04 for (Google Transport Security); Mon, 26 Nov 2018 15:25:04 -0800 (PST) Received-SPF: pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=Pdtzoav8; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=nCRmhahI7kY4j643ILpESoqKGQRCrmHSVJ/3uA5AXlw=; b=Pdtzoav835mYbtf6SXhoxvtwfqBZtwTZwsWDylluLmILa46SW55/nsYTLCqh6d+Kw4 v0k4QvP0irC2WHz7YxGoB0NJ0yJygaY3lrHrB8LwlpVebCtb1y1xFcdOS5XXWY0UWz8R XUzozTz+byBOBKsZJ4p0URaUfa+WgMwtZjU6Lgzgtpaz/XrUnB7sTUcAGoOagPo7Y0BK RTku/zk9mUmhflZUUHyVdYqKu6iaIPWa2wcLXYk/pOVIm0gtvLOAtyhAntCQHRGf+nV/ vCI+ZD2NM2zbD+kV/9Ro+PedH7JSYRK7LTli7VWNghi6K4Z9AkBjxMXrmmTSpRyv+cO/ 3YWA== X-Google-Smtp-Source: AFSGD/XxPCrVlSdAO0vLUHG3etB0kcQwV02MaaUz6ANN2A5QkUrF5psVaI5wPt0d4iR7t/S45HpzuA== X-Received: by 2002:a63:b94c:: with SMTP id v12mr26777485pgo.221.1543274703650; Mon, 26 Nov 2018 15:25:03 -0800 (PST) Received: from [100.112.89.103] ([104.133.8.103]) by smtp.gmail.com with ESMTPSA id h15sm1473213pgl.43.2018.11.26.15.25.02 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Nov 2018 15:25:02 -0800 (PST) Date: Mon, 26 Nov 2018 15:25:01 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Andrew Morton cc: "Kirill A. Shutemov" , "Kirill A. Shutemov" , Matthew Wilcox , linux-mm@kvack.org Subject: [PATCH 05/10] mm/khugepaged: fix crashes due to misaccounted holes In-Reply-To: Message-ID: References: User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Huge tmpfs testing on a shortish file mapped into a pmd-rounded extent hit shmem_evict_inode()'s WARN_ON(inode->i_blocks) followed by clear_inode()'s BUG_ON(inode->i_data.nrpages) when the file was later closed and unlinked. khugepaged's collapse_shmem() was forgetting to update mapping->nrpages on the rollback path, after it had added but then needs to undo some holes. There is indeed an irritating asymmetry between shmem_charge(), whose callers want it to increment nrpages after successfully accounting blocks, and shmem_uncharge(), when __delete_from_page_cache() already decremented nrpages itself: oh well, just add a comment on that to them both. And shmem_recalc_inode() is supposed to be called when the accounting is expected to be in balance (so it can deduce from imbalance that reclaim discarded some pages): so change shmem_charge() to update nrpages earlier (though it's rare for the difference to matter at all). Fixes: 800d8c63b2e98 ("shmem: add huge pages support") Fixes: f3f0e1d2150b2 ("khugepaged: add support of collapse for tmpfs/shmem pages") Signed-off-by: Hugh Dickins Cc: Kirill A. Shutemov Cc: stable@vger.kernel.org # 4.8+ Acked-by: Kirill A. Shutemov --- mm/khugepaged.c | 5 ++++- mm/shmem.c | 6 +++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 2070c316f06e..65e82f665c7c 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1506,9 +1506,12 @@ static void collapse_shmem(struct mm_struct *mm, khugepaged_pages_collapsed++; } else { struct page *page; + /* Something went wrong: roll back page cache changes */ - shmem_uncharge(mapping->host, nr_none); xas_lock_irq(&xas); + mapping->nrpages -= nr_none; + shmem_uncharge(mapping->host, nr_none); + xas_set(&xas, start); xas_for_each(&xas, page, end - 1) { page = list_first_entry_or_null(&pagelist, diff --git a/mm/shmem.c b/mm/shmem.c index ea26d7a0342d..e6558e49b42a 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -297,12 +297,14 @@ bool shmem_charge(struct inode *inode, long pages) if (!shmem_inode_acct_block(inode, pages)) return false; + /* nrpages adjustment first, then shmem_recalc_inode() when balanced */ + inode->i_mapping->nrpages += pages; + spin_lock_irqsave(&info->lock, flags); info->alloced += pages; inode->i_blocks += pages * BLOCKS_PER_PAGE; shmem_recalc_inode(inode); spin_unlock_irqrestore(&info->lock, flags); - inode->i_mapping->nrpages += pages; return true; } @@ -312,6 +314,8 @@ void shmem_uncharge(struct inode *inode, long pages) struct shmem_inode_info *info = SHMEM_I(inode); unsigned long flags; + /* nrpages adjustment done by __delete_from_page_cache() or caller */ + spin_lock_irqsave(&info->lock, flags); info->alloced -= pages; inode->i_blocks -= pages * BLOCKS_PER_PAGE; From patchwork Mon Nov 26 23:26:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 10699339 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 35F2D17D5 for ; Mon, 26 Nov 2018 23:26:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 231F22A5B2 for ; Mon, 26 Nov 2018 23:26:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 114AB2A63A; Mon, 26 Nov 2018 23:26:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 99D0D2A5B2 for ; Mon, 26 Nov 2018 23:26:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CBDD06B4464; Mon, 26 Nov 2018 18:26:38 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C6C746B4465; Mon, 26 Nov 2018 18:26:38 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B5EDA6B4466; Mon, 26 Nov 2018 18:26:38 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) by kanga.kvack.org (Postfix) with ESMTP id 76D426B4464 for ; Mon, 26 Nov 2018 18:26:38 -0500 (EST) Received: by mail-pl1-f197.google.com with SMTP id h10so22534067plk.12 for ; Mon, 26 Nov 2018 15:26:38 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :in-reply-to:message-id:references:user-agent:mime-version; bh=ES/MPQaKtrXNGEigMK4kdNgyIwCP/oFzqKMlT72/m1g=; b=ttZ3MnW3iJjL60K8B7d4QNl7+RLrq8NkVcUxHbCFIrBNZwqAky9Nfp/IS1mtzPf00e QnVNuVk1nXzGErNeCHCDIO/xRT3lQ9CoG76SX8ZkkKZH0aV9+3K3Ik0gBeYXLk81rGrB BMb7NnB/UDF27o+o7Hs+L52dajOYmMV/GMcQBOS4G+u1u6HB2DBVQQHSA5tpMXncn3Ut TaqO10kE47TkgmKZMMjqmXQeWLnZuuFychIVupcjvVbclFsLmw2KhBeYEfNjqu1Qhflo Ab9NAj91LixNSuvnZFZ0OedYqPFH3O4L9IOjxAHnXDhTKEHYaHDbaluyEL7Na1c+EL0U vbpw== X-Gm-Message-State: AA+aEWYvdsQAI2lJ1cSGWgiNdXv/b86eDjh4VnpkZ4WPQMQKkhaLXjAk KGbco3h9d4y4QH9nC+0FYPJJjAgvmQKK1V/7xMZvhlsCifjDziB6q4omTC5zGmeCukK0vxBP8PE tSVpUGvwyM+4XiK7X3Szo/8tgyL5M7KEXHBKwecK/Q2EF9Wz9IB0tPFesYjCEVaE/VLKmZOqXrN GT4S6nFjgYOqUzQSEiX4FC3oxNoZyuXrLOhJ3bdiOXnAA1TqCE6pIdMRSbKpIB00ICvYkHEOjL3 bOD8K1eLdP3GhhsdQLa+zYUXFNCxc/z5XUvevUY60yLGggQmVFzpo239taYl8JtM+n6T+qUp4sZ SYn3YvsgAm6pLbCRHx9zl2OqzeuBYv2GgWEdgZ48Ft0tFWgoxLrqZzOQ1PejTmrcEV9fAQFp/tm U X-Received: by 2002:a63:e915:: with SMTP id i21mr26422565pgh.409.1543274798123; Mon, 26 Nov 2018 15:26:38 -0800 (PST) X-Received: by 2002:a63:e915:: with SMTP id i21mr26422521pgh.409.1543274797402; Mon, 26 Nov 2018 15:26:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543274797; cv=none; d=google.com; s=arc-20160816; b=aJ4QeXraFzoNjac30tBou3zQq8nfP13WgVXgO21qRO5JiPL0IZi6GyAZuDXUoo5N0k i0/JkQhWnkI2Cj6ouS+9+0AivSeKEmyHr/fR4VMWHht5+d49BJF9kKalujQqJBKS8WRP egh1Lv4IPN2pqln+Odc1BMwGU9REvm0XRwSyXHO9JhnNLtqKmB73MVi5dpLGpDhMmqLX j34zJp3oCeaSJfW2w/ZfyIM1gFouwH2M13iOVNZ533zBdicCnw3I/Tt6NhkVD1qhWkUX JZrAbMsWqEqzD9W/HbJYEe2LCGAaCbZgG8CwJHJ8pcJDposQwpOMnJjZZ1YHqq0cqexG uQGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:message-id:in-reply-to:subject :cc:to:from:date:dkim-signature; bh=ES/MPQaKtrXNGEigMK4kdNgyIwCP/oFzqKMlT72/m1g=; b=loxiR/MtXD+Dm3xh3N7Z+Ul/h/PydrgQ1GEnW7Kp9ov9Ic6YtflPF8X05zpWBNXxgJ 6lpJtaXoRHW1ck/+zlXQmzy7er3uQHMZBKmCazFeBtnioHBJdGwCINFw93rodZA/mORR jA3jRigZvfUTFZwwgWnmk+rXsBEQeqXGhqPPjZiUtDC9RVaBTvTVe+ND/YgQ2ql8NKmo Dblw4CzzNUDUq/jKHRIIJOP4w5bdg3PF0BNEeyHH9ULBFFU9neAzh910BOmLbq8aK2Bi o+/SvMOEhTYZXIA7JTn9M0a0YVCAiqzGtY1snLvkKgIH27jHjqQQjRtjXGAw03i1/YXh acPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=JeEAmBlZ; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id gn15sor2447092plb.64.2018.11.26.15.26.37 for (Google Transport Security); Mon, 26 Nov 2018 15:26:37 -0800 (PST) Received-SPF: pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=JeEAmBlZ; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=ES/MPQaKtrXNGEigMK4kdNgyIwCP/oFzqKMlT72/m1g=; b=JeEAmBlZsaBdId52Bq97/6asHJtPkltKiD3thGhn0bTdkiyOW9INJqZyTQIAdBakqT VZxwJCTi9xyko7l+XikzW6vo8EHNrskKo+P2RlQocdLAElHahwvkWk54BS0SOVA7DQzy BpYFWIvTuqTaC4YHgNymvZEm7T+MQ8N+Zhk5sEO0oFC5p+eL7rlc3XAufX8969Rua2Ni 0flKTX6+3C1k7KBQRsyhlMpFeogvO//l3A136G9od8GXvIYez8JIIEdIXazCwc4WKu8j tw0HZeeOw5DQaWDE1TCgMzOCFZ4hPi+3tuNv706rAcqyyZi31wZUDC19dhIdoXUs7cA9 QRQQ== X-Google-Smtp-Source: AFSGD/WY+t/6WquRIc621kcDoOJCimgJQyjg9NZnd0TWWFlJRVdupL8oL/cPgaiJXMArvFZ68coTYg== X-Received: by 2002:a17:902:7587:: with SMTP id j7mr30114848pll.191.1543274796640; Mon, 26 Nov 2018 15:26:36 -0800 (PST) Received: from [100.112.89.103] ([104.133.8.103]) by smtp.gmail.com with ESMTPSA id f13sm1981535pfa.132.2018.11.26.15.26.35 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Nov 2018 15:26:36 -0800 (PST) Date: Mon, 26 Nov 2018 15:26:34 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Andrew Morton cc: "Kirill A. Shutemov" , "Kirill A. Shutemov" , Matthew Wilcox , linux-mm@kvack.org Subject: [PATCH 06/10] mm/khugepaged: collapse_shmem() remember to clear holes In-Reply-To: Message-ID: References: User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Huge tmpfs testing reminds us that there is no __GFP_ZERO in the gfp flags khugepaged uses to allocate a huge page - in all common cases it would just be a waste of effort - so collapse_shmem() must remember to clear out any holes that it instantiates. The obvious place to do so, where they are put into the page cache tree, is not a good choice: because interrupts are disabled there. Leave it until further down, once success is assured, where the other pages are copied (before setting PageUptodate). Fixes: f3f0e1d2150b2 ("khugepaged: add support of collapse for tmpfs/shmem pages") Signed-off-by: Hugh Dickins Cc: Kirill A. Shutemov Cc: stable@vger.kernel.org # 4.8+ Acked-by: Kirill A. Shutemov --- mm/khugepaged.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 65e82f665c7c..1c402d33547e 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1467,7 +1467,12 @@ static void collapse_shmem(struct mm_struct *mm, * Replacing old pages with new one has succeeded, now we * need to copy the content and free the old pages. */ + index = start; list_for_each_entry_safe(page, tmp, &pagelist, lru) { + while (index < page->index) { + clear_highpage(new_page + (index % HPAGE_PMD_NR)); + index++; + } copy_highpage(new_page + (page->index % HPAGE_PMD_NR), page); list_del(&page->lru); @@ -1477,6 +1482,11 @@ static void collapse_shmem(struct mm_struct *mm, ClearPageActive(page); ClearPageUnevictable(page); put_page(page); + index++; + } + while (index < end) { + clear_highpage(new_page + (index % HPAGE_PMD_NR)); + index++; } local_irq_disable(); From patchwork Mon Nov 26 23:27:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 10699347 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BB7FF13BB for ; Mon, 26 Nov 2018 23:27:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AC6082A5B2 for ; Mon, 26 Nov 2018 23:27:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A09402A63A; Mon, 26 Nov 2018 23:27:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 077022A5D1 for ; Mon, 26 Nov 2018 23:27:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 321F76B4467; Mon, 26 Nov 2018 18:27:56 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2A9336B4468; Mon, 26 Nov 2018 18:27:56 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 14A916B4469; Mon, 26 Nov 2018 18:27:56 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by kanga.kvack.org (Postfix) with ESMTP id C46696B4467 for ; Mon, 26 Nov 2018 18:27:55 -0500 (EST) Received: by mail-pl1-f199.google.com with SMTP id v2so10936786plg.6 for ; Mon, 26 Nov 2018 15:27:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :in-reply-to:message-id:references:user-agent:mime-version; bh=gNGmozdeLydO2lSANHRatkPVmfiAppDs7BSNKAYbn3s=; b=dhM2mXRQFFbFv0Iyvpnp3UBnUotG6fVNg08hpDGwfVj3DMOzg7hJJMDKnuHEEGRFtv HtLQbByJYsyN8p04ZZI6pYAMUIMYAFiFTEKpscT/wQTy9tEgsTe9mCHFDwn/04XGhrnp VqzKod+Xl28mAnthovzpbZ0CeVq8MgapWcOgnKO+09f0+JXGKklbePGY9kNUa40tw3hO zFZaKA0yVfEyvMZZ1hnjxG14e4clwX4p8m2pZK3mS+gLisf6GXU0dQLMMd9+o/4xrWrd WQN15Gl/fVqMyrjHOZjuH//Zg3mnDFCEbHQrnE4dk5hW0k0XgKt2UN7385jimkMyn7iX GvnQ== X-Gm-Message-State: AA+aEWYJLo5PwIN2+GOdAvOapICNoJSvGF+5gsJJ8pvjc8xf5Bd90vd7 h/455ozZH+StUqbI2VgtiAizZXXu486Mq3zprXWakkMEsg/+utuatLOLQXYb/724QgeKNxOl5Ck K6599om+Yk6rQjWqlSwwvrKeHkDUFVRiXLF6UFZ/kwXMWA82ciaPZLWv01NjWtH4hfNFrfwh89S eNTWykSyNQetrNDYTNXuV26Y6cLTVAFVj6NpimrYt8nGd10KzlJTZ4DclnouS0eyuHpmNzz1VWT gpc0IV1nJRF7II9SxQt01IvorD6qb8KimMmN/NQfmaTEcy09Lmydl8oQvJv9eDYHCMfdpIrBiUP dB90MULPfXMSK2wD8XhuGTV3UTCsuFQSfurguo5rWwP6mBeu8w0FIQESty9T9xiLT4PdEK5tkvW x X-Received: by 2002:a17:902:820d:: with SMTP id x13mr30562853pln.229.1543274875428; Mon, 26 Nov 2018 15:27:55 -0800 (PST) X-Received: by 2002:a17:902:820d:: with SMTP id x13mr30562810pln.229.1543274874596; Mon, 26 Nov 2018 15:27:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543274874; cv=none; d=google.com; s=arc-20160816; b=Po+4N1Axd6A3EZA6jZvtyW7jCRPMrWT+iRwU5LNsg/2yaJmuPwNioDucQlTABqdRQC c6olFdR+sNtWWxzcccR0zthGVdlf9rxjKPOYFetDVmT4isHJffeiy8iV6oZfZEAwc0PG 8nBgFwHN3p+zhD77Vj2YZkom4VWxnLuBGOs0uwJoIgqJbtX/5cdH3ES+h4mdVPrZyB92 bZggLQlHARekIFGldQElXsYhOKGx+NmOLvd5OPmKKzlagLETx3ckmW9cK6lgiLmoLG5k SiI7AuHxaXoEIN2v2d/BdZsLhi3wvYFP/fOS/GuQio/qKKMxvWe+FmCzX/Lsxf2zxLKG /b7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:message-id:in-reply-to:subject :cc:to:from:date:dkim-signature; bh=gNGmozdeLydO2lSANHRatkPVmfiAppDs7BSNKAYbn3s=; b=nK+ACfqqZJfcTG33F1Zw5DYUrm1lmLTpainoCePqy2w0ellY6Y4biu8h5AZDZ59lmM JDjv8luzd6A6b+evs+1y4nlZkba9yvRqAziCG+n9y6tq6j3pqwiYxHHPOEdnQkdXzSTN cER2sSQS6Li70clrlmwWMtCMtuqvBPPbS+ky7kjTJ/sEPRjuGqQqKaH+oB013Okss37d O6yzMzEevJ0asvRSJXXn003kbzNuZKnGVQF/d1twVFlCtmwFHgygyXwGuYoffrtLQjkG NbPWDHl4yqHKktXfrr5J7svVElbTarZo0qUPj5Dbg/ad0jQfU15xHd1FezUC/2+HRJYP WlhQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=BJ82sdYv; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id 38sor2512811pld.67.2018.11.26.15.27.54 for (Google Transport Security); Mon, 26 Nov 2018 15:27:54 -0800 (PST) Received-SPF: pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=BJ82sdYv; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=gNGmozdeLydO2lSANHRatkPVmfiAppDs7BSNKAYbn3s=; b=BJ82sdYvHkZVvWoaqZ8ZLjHKNOzu3E/Yf+ZjIH9kQ2E6lziw6F+gXGe/Rh5YBeKTmR zqqPhYumWKpw9UTmzxPsrdc5FDeQSFMCZklqjsynW7+vGbm39FOG3mSmAoJubHbUD8zX 0c1HjKOUbTc2+4uUQJmu1dHs64Eh/WQ7Qp1QxuVow2dSDVncrJHMPqbWPC0KkXc4JJOf YHcvonYDI85TzmaXBO9y5AAGVBuHzBeZ4FK++wuD7NqxlfJbL9nIIVuTg6IQgqDfdKqE oNEeydEMldkJnHATjqkIl7V2Q0o3/7RJfgwmtuJyMgwIIp0lgKZe+JNkjY62U8g042N5 u1gQ== X-Google-Smtp-Source: AFSGD/XOwOs9IeycZUMd03Rn345TXyBLOMkysnyNi/KTJxp8MLM6J53HxXqgChMOf8Y1J8dCWZoJAQ== X-Received: by 2002:a17:902:24d:: with SMTP id 71mr29118169plc.225.1543274873842; Mon, 26 Nov 2018 15:27:53 -0800 (PST) Received: from [100.112.89.103] ([104.133.8.103]) by smtp.gmail.com with ESMTPSA id w10-v6sm1930848pfg.1.2018.11.26.15.27.52 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Nov 2018 15:27:53 -0800 (PST) Date: Mon, 26 Nov 2018 15:27:52 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Andrew Morton cc: "Kirill A. Shutemov" , "Kirill A. Shutemov" , Matthew Wilcox , linux-mm@kvack.org Subject: [PATCH 07/10] mm/khugepaged: minor reorderings in collapse_shmem() In-Reply-To: Message-ID: References: User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Several cleanups in collapse_shmem(): most of which probably do not really matter, beyond doing things in a more familiar and reassuring order. Simplify the failure gotos in the main loop, and on success update stats while interrupts still disabled from the last iteration. Fixes: f3f0e1d2150b2 ("khugepaged: add support of collapse for tmpfs/shmem pages") Signed-off-by: Hugh Dickins Cc: Kirill A. Shutemov Cc: stable@vger.kernel.org # 4.8+ --- mm/khugepaged.c | 72 ++++++++++++++++++++++--------------------------- 1 file changed, 32 insertions(+), 40 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 1c402d33547e..9d4e9ff1af95 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1329,10 +1329,10 @@ static void collapse_shmem(struct mm_struct *mm, goto out; } + __SetPageLocked(new_page); + __SetPageSwapBacked(new_page); new_page->index = start; new_page->mapping = mapping; - __SetPageSwapBacked(new_page); - __SetPageLocked(new_page); BUG_ON(!page_ref_freeze(new_page, 1)); /* @@ -1366,13 +1366,13 @@ static void collapse_shmem(struct mm_struct *mm, if (index == start) { if (!xas_next_entry(&xas, end - 1)) { result = SCAN_TRUNCATED; - break; + goto xa_locked; } xas_set(&xas, index); } if (!shmem_charge(mapping->host, 1)) { result = SCAN_FAIL; - break; + goto xa_locked; } xas_store(&xas, new_page + (index % HPAGE_PMD_NR)); nr_none++; @@ -1387,13 +1387,12 @@ static void collapse_shmem(struct mm_struct *mm, result = SCAN_FAIL; goto xa_unlocked; } - xas_lock_irq(&xas); - xas_set(&xas, index); } else if (trylock_page(page)) { get_page(page); + xas_unlock_irq(&xas); } else { result = SCAN_PAGE_LOCK; - break; + goto xa_locked; } /* @@ -1408,11 +1407,10 @@ static void collapse_shmem(struct mm_struct *mm, result = SCAN_TRUNCATED; goto out_unlock; } - xas_unlock_irq(&xas); if (isolate_lru_page(page)) { result = SCAN_DEL_PAGE_LRU; - goto out_isolate_failed; + goto out_unlock; } if (page_mapped(page)) @@ -1432,7 +1430,9 @@ static void collapse_shmem(struct mm_struct *mm, */ if (!page_ref_freeze(page, 3)) { result = SCAN_PAGE_COUNT; - goto out_lru; + xas_unlock_irq(&xas); + putback_lru_page(page); + goto out_unlock; } /* @@ -1444,24 +1444,26 @@ static void collapse_shmem(struct mm_struct *mm, /* Finally, replace with the new page. */ xas_store(&xas, new_page + (index % HPAGE_PMD_NR)); continue; -out_lru: - xas_unlock_irq(&xas); - putback_lru_page(page); -out_isolate_failed: - unlock_page(page); - put_page(page); - goto xa_unlocked; out_unlock: unlock_page(page); put_page(page); - break; + goto xa_unlocked; } - xas_unlock_irq(&xas); + __inc_node_page_state(new_page, NR_SHMEM_THPS); + if (nr_none) { + struct zone *zone = page_zone(new_page); + + __mod_node_page_state(zone->zone_pgdat, NR_FILE_PAGES, nr_none); + __mod_node_page_state(zone->zone_pgdat, NR_SHMEM, nr_none); + } + +xa_locked: + xas_unlock_irq(&xas); xa_unlocked: + if (result == SCAN_SUCCEED) { struct page *page, *tmp; - struct zone *zone = page_zone(new_page); /* * Replacing old pages with new one has succeeded, now we @@ -1476,11 +1478,11 @@ static void collapse_shmem(struct mm_struct *mm, copy_highpage(new_page + (page->index % HPAGE_PMD_NR), page); list_del(&page->lru); - unlock_page(page); - page_ref_unfreeze(page, 1); page->mapping = NULL; + page_ref_unfreeze(page, 1); ClearPageActive(page); ClearPageUnevictable(page); + unlock_page(page); put_page(page); index++; } @@ -1489,28 +1491,17 @@ static void collapse_shmem(struct mm_struct *mm, index++; } - local_irq_disable(); - __inc_node_page_state(new_page, NR_SHMEM_THPS); - if (nr_none) { - __mod_node_page_state(zone->zone_pgdat, NR_FILE_PAGES, nr_none); - __mod_node_page_state(zone->zone_pgdat, NR_SHMEM, nr_none); - } - local_irq_enable(); - - /* - * Remove pte page tables, so we can re-fault - * the page as huge. - */ - retract_page_tables(mapping, start); - /* Everything is ready, let's unfreeze the new_page */ - set_page_dirty(new_page); SetPageUptodate(new_page); page_ref_unfreeze(new_page, HPAGE_PMD_NR); + set_page_dirty(new_page); mem_cgroup_commit_charge(new_page, memcg, false, true); lru_cache_add_anon(new_page); - unlock_page(new_page); + /* + * Remove pte page tables, so we can re-fault the page as huge. + */ + retract_page_tables(mapping, start); *hpage = NULL; khugepaged_pages_collapsed++; @@ -1543,8 +1534,8 @@ static void collapse_shmem(struct mm_struct *mm, xas_store(&xas, page); xas_pause(&xas); xas_unlock_irq(&xas); - putback_lru_page(page); unlock_page(page); + putback_lru_page(page); xas_lock_irq(&xas); } VM_BUG_ON(nr_none); @@ -1553,9 +1544,10 @@ static void collapse_shmem(struct mm_struct *mm, /* Unfreeze new_page, caller would take care about freeing it */ page_ref_unfreeze(new_page, 1); mem_cgroup_cancel_charge(new_page, memcg, true); - unlock_page(new_page); new_page->mapping = NULL; } + + unlock_page(new_page); out: VM_BUG_ON(!list_empty(&pagelist)); /* TODO: tracepoints */ From patchwork Mon Nov 26 23:29:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 10699355 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 31CFE13BB for ; Mon, 26 Nov 2018 23:29:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 213622A63D for ; Mon, 26 Nov 2018 23:29:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 158542A63F; Mon, 26 Nov 2018 23:29:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 714092A63D for ; Mon, 26 Nov 2018 23:29:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8761F6B446B; Mon, 26 Nov 2018 18:29:21 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 826446B446C; Mon, 26 Nov 2018 18:29:21 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6C6BA6B446D; Mon, 26 Nov 2018 18:29:21 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) by kanga.kvack.org (Postfix) with ESMTP id 268A36B446B for ; Mon, 26 Nov 2018 18:29:21 -0500 (EST) Received: by mail-pf1-f199.google.com with SMTP id t72so5709741pfi.21 for ; Mon, 26 Nov 2018 15:29:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :in-reply-to:message-id:references:user-agent:mime-version; bh=eqKOhg2s8YPm+nNGmwuBjgOu5boUks9xB84QcRhub3U=; b=YHqjRR6HjGKuI6/6erYfxPldUw10/3XRucE+rzU6Vvma3cRNcb0L+5QQbByVsNzXGL 8KT09XRQDsnfq1kcfvCqKaFSrCclJztjH2Iv6aYHCXLmRPk3FnZDB0AbmunE2jM+XLCL 9BrNQ/nVhNfAWqNZJxPg1LFxkvqpG0BcJ3BxLUlk++VW4oC34SvJnd2T/lH/LJAGUdXJ 7y8fxPAZIR7im0gx9MMhT19RtiygMu+OfnnJj1z/TMM7nGz4DP+gy4KlUKhDN0yBfd+W Nc1PsfXi+3nKIPsxeYWfhwy58OzSZGmSmFeIiRi6hN5CZEEPQAYbKWk0Ajw6OP+zRpu4 lr2A== X-Gm-Message-State: AA+aEWa3LaS8+327Y+Hm7dfIGJtzGYVxU816TdDoZ++H3J+KGWZoV2AY g82WNj/C9o3mYBRcay4mxBTjb7sjPaW69vRuq3ry1ONzCbcB5zBtL/rvkAyVKt58J2C8u0mP9ic ya4oYP/NLy8DQ6/1Dn3DaIRgPChA82gqS2evCYHye8/e40l1Lw2JZS5CqkQ5fbr5icN8medBS6h zzU91Eg+5+mfu4+Sr2Rwo1tgcyAZnYir79BxcMhxSv/JTPdQH6mpDaQwCY9fo3AlxC1c2m4ztz4 Fa2BSK42ZylwrP2pqxMZloLWdDWx2H1fU69lgrGLWEHWcF2FHqeCywdZbU4Wgvw+hf57VXoTcR0 YjRzKgru5JIH83mnplE6P7Hi6FfVBkosrx5K9aKFooLqXp93I/8fgSAP6eGavIYjxY3Qmr3RDgW r X-Received: by 2002:a63:4c6:: with SMTP id 189mr27221553pge.391.1543274960687; Mon, 26 Nov 2018 15:29:20 -0800 (PST) X-Received: by 2002:a63:4c6:: with SMTP id 189mr27221519pge.391.1543274959798; Mon, 26 Nov 2018 15:29:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543274959; cv=none; d=google.com; s=arc-20160816; b=g+yWQPcsC97jF+aWPiTj1ynArttw3d+1ilXh3A9jYKp2wx4do00cDABHVQBQ7HrqbU koS9MzCeeejiVaoezaw9+NTF1VPh52cbrHU9s+DemjoZxS+sleZxly3eJOGjAqpPfIQT UWoo77wpc4MeJvk1Exs3VgVM4bjjdOKwiiEf2/t2f0WJ9QZilM+Nhiak3QX7WVmuHwtA ak1YQaJiqXgPK/2MRAyABCwhHFcEouc4qBZiKykrMRKo9LxltrNd+KFVl45u60QORBD5 Vg5ucyBi8TZobHFFlPODi6FO9P/Hm3qyZjCigOEF7owj9/DfYsYX0VAvxwyrqDHcQWkz Xmkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:message-id:in-reply-to:subject :cc:to:from:date:dkim-signature; bh=eqKOhg2s8YPm+nNGmwuBjgOu5boUks9xB84QcRhub3U=; b=Edk/1HmdYozulDKCrnP5X4nVvgGi4OMKmZeeT8ea9RVDXnCp5QUDrfa9iFc/Ne+Jbt R1GW/mn/u3/DWeQ16xs3B/bTVFIkmo5n3r3QHBIXauquArXshiU4PDkv4BgTQJqfBj3Z 59iiAfP4lk3RGn7Y01V6QyVqQCrNKRuG4I8oLkjEkNC+e5Cw59RMB3XIs9f7T7Xd2ztf qjWNCNMFLIlqPhF9cbnES9PrpfAc/PcrID8vVmTjq9NGArRMFYErZeroFGx4/mF/uI9F IgIitAP6kpGsNJDnqsU7Z5ZRgGJB0R2rI7T6R5wHT/cDYN4CBfM/TfvkXxMK0Xq/oUbc jcTg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=AmOizD+Y; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id c14sor2696584pgl.37.2018.11.26.15.29.19 for (Google Transport Security); Mon, 26 Nov 2018 15:29:19 -0800 (PST) Received-SPF: pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=AmOizD+Y; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=eqKOhg2s8YPm+nNGmwuBjgOu5boUks9xB84QcRhub3U=; b=AmOizD+Ya5/bexjYbLnc/pRFcowiEsrCZ3F0tz2yd/kSfm0yy3SByh3CM8zhxMoan0 G/pAmn0Byodre3y0sLE+q+X5PTBHrV6HqOGqh8KGabUCMH8p86ZYCSiZupe1+LdnrprF XjITqtUh66ZsyihjCOm+EOi9LYbUrpgbqrMgt0G+kFCdEhW+N3brBcWIAALLERmgdxjp 1mKGWK7DNk9RhnIKmgTkaItC6O4RLe222ohXuSfbTJ5N2tJo9w9KJLS4QpNtj67/h6qj LOhRsfKKTml5NUWV9w+Yi6BIA4E3P8zOrrVnwT+eDhnHJQSy5Jzb+gbr7kVcU25wy6i1 Bp9A== X-Google-Smtp-Source: AFSGD/WNHgtkukQy+RcrFw/eDOwV9GkQN4ywQO2hPViWZEJZDCeswClU7aX1pBBEsNrN8yuTa9gvww== X-Received: by 2002:a63:cc43:: with SMTP id q3mr26607820pgi.291.1543274959069; Mon, 26 Nov 2018 15:29:19 -0800 (PST) Received: from [100.112.89.103] ([104.133.8.103]) by smtp.gmail.com with ESMTPSA id c23sm1782774pfi.83.2018.11.26.15.29.18 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Nov 2018 15:29:18 -0800 (PST) Date: Mon, 26 Nov 2018 15:29:17 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Andrew Morton cc: "Kirill A. Shutemov" , "Kirill A. Shutemov" , Matthew Wilcox , linux-mm@kvack.org Subject: [PATCH 08/10] mm/khugepaged: collapse_shmem() without freezing new_page In-Reply-To: Message-ID: References: User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP khugepaged's collapse_shmem() does almost all of its work, to assemble the huge new_page from 512 scattered old pages, with the new_page's refcount frozen to 0 (and refcounts of all old pages so far also frozen to 0). Including shmem_getpage() to read in any which were out on swap, memory reclaim if necessary to allocate their intermediate pages, and copying over all the data from old to new. Imagine the frozen refcount as a spinlock held, but without any lock debugging to highlight the abuse: it's not good, and under serious load heads into lockups - speculative getters of the page are not expecting to spin while khugepaged is rescheduled. One can get a little further under load by hacking around elsewhere; but fortunately, freezing the new_page turns out to have been entirely unnecessary, with no hacks needed elsewhere. The huge new_page lock is already held throughout, and guards all its subpages as they are brought one by one into the page cache tree; and anything reading the data in that page, without the lock, before it has been marked PageUptodate, would already be in the wrong. So simply eliminate the freezing of the new_page. Each of the old pages remains frozen with refcount 0 after it has been replaced by a new_page subpage in the page cache tree, until they are all unfrozen on success or failure: just as before. They could be unfrozen sooner, but cause no problem once no longer visible to find_get_entry(), filemap_map_pages() and other speculative lookups. Fixes: f3f0e1d2150b2 ("khugepaged: add support of collapse for tmpfs/shmem pages") Signed-off-by: Hugh Dickins Cc: Kirill A. Shutemov Cc: stable@vger.kernel.org # 4.8+ Acked-by: Kirill A. Shutemov --- mm/khugepaged.c | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 9d4e9ff1af95..55930cbed3fd 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1287,7 +1287,7 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) * collapse_shmem - collapse small tmpfs/shmem pages into huge one. * * Basic scheme is simple, details are more complex: - * - allocate and freeze a new huge page; + * - allocate and lock a new huge page; * - scan page cache replacing old pages with the new one * + swap in pages if necessary; * + fill in gaps; @@ -1295,11 +1295,11 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff) * - if replacing succeeds: * + copy data over; * + free old pages; - * + unfreeze huge page; + * + unlock huge page; * - if replacing failed; * + put all pages back and unfreeze them; * + restore gaps in the page cache; - * + free huge page; + * + unlock and free huge page; */ static void collapse_shmem(struct mm_struct *mm, struct address_space *mapping, pgoff_t start, @@ -1333,13 +1333,11 @@ static void collapse_shmem(struct mm_struct *mm, __SetPageSwapBacked(new_page); new_page->index = start; new_page->mapping = mapping; - BUG_ON(!page_ref_freeze(new_page, 1)); /* - * At this point the new_page is 'frozen' (page_count() is zero), - * locked and not up-to-date. It's safe to insert it into the page - * cache, because nobody would be able to map it or use it in other - * way until we unfreeze it. + * At this point the new_page is locked and not up-to-date. + * It's safe to insert it into the page cache, because nobody would + * be able to map it or use it in another way until we unlock it. */ /* This will be less messy when we use multi-index entries */ @@ -1491,9 +1489,8 @@ static void collapse_shmem(struct mm_struct *mm, index++; } - /* Everything is ready, let's unfreeze the new_page */ SetPageUptodate(new_page); - page_ref_unfreeze(new_page, HPAGE_PMD_NR); + page_ref_add(new_page, HPAGE_PMD_NR - 1); set_page_dirty(new_page); mem_cgroup_commit_charge(new_page, memcg, false, true); lru_cache_add_anon(new_page); @@ -1541,8 +1538,6 @@ static void collapse_shmem(struct mm_struct *mm, VM_BUG_ON(nr_none); xas_unlock_irq(&xas); - /* Unfreeze new_page, caller would take care about freeing it */ - page_ref_unfreeze(new_page, 1); mem_cgroup_cancel_charge(new_page, memcg, true); new_page->mapping = NULL; } From patchwork Mon Nov 26 23:31:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 10699369 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B806813BB for ; Mon, 26 Nov 2018 23:31:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A927A2A63F for ; Mon, 26 Nov 2018 23:31:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9D9442A642; Mon, 26 Nov 2018 23:31:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 32AC02A63F for ; Mon, 26 Nov 2018 23:31:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5DE2E6B4471; Mon, 26 Nov 2018 18:31:19 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 58C956B4472; Mon, 26 Nov 2018 18:31:19 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4552F6B4473; Mon, 26 Nov 2018 18:31:19 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) by kanga.kvack.org (Postfix) with ESMTP id 01C436B4471 for ; Mon, 26 Nov 2018 18:31:19 -0500 (EST) Received: by mail-pl1-f197.google.com with SMTP id w19-v6so22292428plq.1 for ; Mon, 26 Nov 2018 15:31:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :in-reply-to:message-id:references:user-agent:mime-version; bh=zVzA8Mp4h/6ZwRFyfdsiGQfp0RMyxpY9T9eRR6DfAgs=; b=cVXUmdusBy8BBQ/ysA4ZUUZPCE8uKCgs9jhUGXWYCjeAYG3DWGph9o0hgxl+mWzJYg aIBOe37KOdYP68vAE23c6AM6ZFwdCXtYvCCJctkd2yTTFaVTq5Pgmc+5iCwkT7RdUU6l q5if516dimHBUC4uB561axvZ+d3jrF4oC3uCldvk5xO4MPP6e/4BgCfz5B3pxL6HB2GY G5kso9Losy0xwyC4W6GIpb//uL6dCJvJZpuH0E/MzNo3sAioNwkn/Poe1kDFKGGD7oce RI9oyAmxVrMYgAaqgn2izQpO1/zG2BDPUaELzm7pPa3GPtnd6tkp1Ph8QG5t85BuQZqN GrIg== X-Gm-Message-State: AA+aEWYDyCAyQovNHmBgm7od/wFyxuWYM8Ced6M027b/pH+DzqjLworA 0HhlobximYwUXHpLPsnsn5OaXsCXxA9ukSvkGEgF+LRGDSd0muEpLo0KbUa1VYNYiLm1PhKcJ4K Jp4jj8NL3y/WxahOBI9NCijQZ6c7L0VBZF+qGVcnwncaEtYWXTwFAN5UssjVkUaeLeGx3WH75PI TRwtVHmoEpbnjAMs15/H60y9HOIAuFs2/biyV+/lmghb+7PCruHb5A8fBrPMLrBRnKacWorEo0L +fALDqiyW/YI2MgzxrAQw+Phdq3kg1ZOaVTrDsuHJcz0q6ll/oL2nmwZouN7QxcG1FUoxgVM0Fl lXx/LpfqA0LFrRR3Q2TDip1Fqr1tMI8KKiPg0KpCr2WFqmmt46oYfWfb/SWRkVx0WkKgiJbOmRL v X-Received: by 2002:a63:4f20:: with SMTP id d32mr26898746pgb.47.1543275078645; Mon, 26 Nov 2018 15:31:18 -0800 (PST) X-Received: by 2002:a63:4f20:: with SMTP id d32mr26898696pgb.47.1543275077838; Mon, 26 Nov 2018 15:31:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543275077; cv=none; d=google.com; s=arc-20160816; b=IVimCFFvaxuYExO80G5SA+rChmFTnRU6ZnV/OJljgvGzrYzuiljuyKgyh1HY76/ZoA Q3qbISbnbVQPvWWjaaV75C77HGPDSuAo67UCx+Wyu6iT7TXIT77+O8xoJkljCr8tbuVQ AUEwSHUNHGtQye1sqmHJASs8hrYYmNlehgYRXHQCsSTR/RI1RyCNLK9pJWjxFcmTMMb4 Z1XVfz0oNRhXzl4U9O7Vzh6LrmawAqKJba+5yZvwbLK6oM7uIUAL5w3ZQUjp25VX/zdF vkurKEMSmdY4g2l+kt5vnkzvbjtkSw4CHZNjtdo7rzsmRtieJEew54I+DICmSmadNhSL C0rA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:message-id:in-reply-to:subject :cc:to:from:date:dkim-signature; bh=zVzA8Mp4h/6ZwRFyfdsiGQfp0RMyxpY9T9eRR6DfAgs=; b=GsR8d9MNXTZijgzKtJRxNV8cegH3oQqdNOYG5awFP+O6GHzF3htAWAw0Cp51lxI3P2 eliwcJRSbl1lsPsfqfJY3EaKwgLyQcEKZJAnmA16509FHZHyGE3PsNaPM0ZyQoeZKr61 NRFZ1/OKbz4Snjhqwotjwm6qjK/7btCVhXL5vVtPpg5ctacpmCaU5hQ8p6fq/Vs60/AS nvBciQ0PEnuhFj4gaayNPeLe2zY3MqbI4TEj3g2tWRRnzazZL5aT/VER0yIKSOanY4fO 7GDAUmK7vxwfYzQ5T0RMbWpixLQaxSvprO+uveAqbu3eXRLhY/8EHKn2WfHVORKWl1+u nkXA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=HsJSpizO; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id q15sor2537704plr.34.2018.11.26.15.31.17 for (Google Transport Security); Mon, 26 Nov 2018 15:31:17 -0800 (PST) Received-SPF: pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=HsJSpizO; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=zVzA8Mp4h/6ZwRFyfdsiGQfp0RMyxpY9T9eRR6DfAgs=; b=HsJSpizOSdjB3Vxed4Eyk8bCyskpFvAnzktWRkGCPqDa5PwDqEj1MeLHgA4NkFVKUV GfjWDIze+EhWGLami33PyCHqq1MEdL8ZUqIH+m8cT6rca50RI/wAEI1PCvArqCH5H6vZ dPf07LlmcHy6Uln73m16E2nmZP8zjU+c+1gIxy+5QZtvtg3Vxf5JwuOtwUwTGv/N55bt 6UojtCxNnCG3xRyi9y8ybzwhLa7gAjMSdDtyePxzOMt0OMkEawEBwC32NEXDPKeuV0Hd K3FxSzcf+J7EkuEaoHDfYl9pEMbBAT1yrX9u2gzLHV9CR3v+m82Kb0gLbYB+ifbksxcu yJuw== X-Google-Smtp-Source: AFSGD/X3JGNl8UgIBSdaKqOITIAiEm3NGBiWWvaqT1RtOflNNijsYX/JDpHQwkGVxwD/J459+OUE+A== X-Received: by 2002:a17:902:7c0a:: with SMTP id x10mr30351675pll.65.1543275077124; Mon, 26 Nov 2018 15:31:17 -0800 (PST) Received: from [100.112.89.103] ([104.133.8.103]) by smtp.gmail.com with ESMTPSA id f62sm1594471pgc.67.2018.11.26.15.31.16 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Nov 2018 15:31:16 -0800 (PST) Date: Mon, 26 Nov 2018 15:31:15 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Andrew Morton cc: "Kirill A. Shutemov" , "Kirill A. Shutemov" , Matthew Wilcox , linux-mm@kvack.org Subject: [PATCH 09/10] mm/khugepaged: collapse_shmem() do not crash on Compound In-Reply-To: Message-ID: References: User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP collapse_shmem()'s VM_BUG_ON_PAGE(PageTransCompound) was unsafe: before it holds page lock of the first page, racing truncation then extension might conceivably have inserted a hugepage there already. Fail with the SCAN_PAGE_COMPOUND result, instead of crashing (CONFIG_DEBUG_VM=y) or otherwise mishandling the unexpected hugepage - though later we might code up a more constructive way of handling it, with SCAN_SUCCESS. Fixes: f3f0e1d2150b2 ("khugepaged: add support of collapse for tmpfs/shmem pages") Signed-off-by: Hugh Dickins Cc: Kirill A. Shutemov Cc: stable@vger.kernel.org # 4.8+ --- mm/khugepaged.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 55930cbed3fd..2c5fe4f7a0c6 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1399,7 +1399,15 @@ static void collapse_shmem(struct mm_struct *mm, */ VM_BUG_ON_PAGE(!PageLocked(page), page); VM_BUG_ON_PAGE(!PageUptodate(page), page); - VM_BUG_ON_PAGE(PageTransCompound(page), page); + + /* + * If file was truncated then extended, or hole-punched, before + * we locked the first page, then a THP might be there already. + */ + if (PageTransCompound(page)) { + result = SCAN_PAGE_COMPOUND; + goto out_unlock; + } if (page_mapping(page) != mapping) { result = SCAN_TRUNCATED; From patchwork Mon Nov 26 23:33:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 10699375 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7243A13BB for ; Mon, 26 Nov 2018 23:33:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6232D2A63F for ; Mon, 26 Nov 2018 23:33:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 54DB82A646; Mon, 26 Nov 2018 23:33:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CA6912A63F for ; Mon, 26 Nov 2018 23:33:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 00E766B4475; Mon, 26 Nov 2018 18:33:18 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F00656B4476; Mon, 26 Nov 2018 18:33:17 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA5FD6B4477; Mon, 26 Nov 2018 18:33:17 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f198.google.com (mail-pl1-f198.google.com [209.85.214.198]) by kanga.kvack.org (Postfix) with ESMTP id 961956B4475 for ; Mon, 26 Nov 2018 18:33:17 -0500 (EST) Received: by mail-pl1-f198.google.com with SMTP id w19-v6so22295595plq.1 for ; Mon, 26 Nov 2018 15:33:17 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :in-reply-to:message-id:references:user-agent:mime-version; bh=Gn5WzFahKDFmpa30/2NDbGP8DwU+8tgzl9BF6DVEeRU=; b=WaK4KcakbbVTAMPJU7kIYUXwmeOSVNS9d/o52wwmYcBYGOKIEFZ1wM7D4mFV1LoLlG Wh4aoshAkZ9Uv7vLTNPbL8OlF95UaLfAy1e9JbyIgFGWhMX12c4XhLVqZ1I6Dy2g5c7n 5Jbz9Zc/xaM8N2fxoSKdW2Si792l+4j3+PEwWiKBr/zyoSf2VCjd+qYeQA2iCNhEn/sC a9ibWaYTxSOO78WSfiarqAn8fBtgJv3c5DN4aoBzAsLPIg6VxnrYYsxN1Z/zh8GHFj3x BgYQ2MG2pxji0erDclqjR/73SDXIzXC4KoJPliRoBEHGrQu6AYlMllhBSANunTaQQOxy 3ubQ== X-Gm-Message-State: AA+aEWZBG/nHlzSXqoV5f50Tl4GdKMrheTn9SHl49DIC/pYdL6iIJMRw KiaAmf+JKWOd8JXlLWcEcjOcN2B1/g+P5lvftLzY/aaw0XhGC4ShRI2MlKQ64kBTEehAr2JD676 fBHfumjZakitOjJaxRFu8ZMKp+Xuu54zlZAUxwoKu+r3ICN46j0QZ4FkahDhWuZBSS8tOOAxQFt jTnL3E37LWMNkHHEIUSs6O7+yRCxiWebw+o70P1uBuUzNG4VuO8NW2rrzaFbavkl+zAjKijgpwc fAJD39eYJfwJv5i267zGcXjFThyVYziJGVQSEyS699HE4N/bDTaN8nz6pi1yoytAVFItmMq0BGC 5ypXr2RFlfyDZ8lxbzJVT7fn5RIHxomM/5Q8M2hCD9m7GwdYVcrtSKtdHi6/GEp+TmLlJ5FJ+/e q X-Received: by 2002:a63:d604:: with SMTP id q4mr26688641pgg.175.1543275197271; Mon, 26 Nov 2018 15:33:17 -0800 (PST) X-Received: by 2002:a63:d604:: with SMTP id q4mr26688607pgg.175.1543275196581; Mon, 26 Nov 2018 15:33:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543275196; cv=none; d=google.com; s=arc-20160816; b=jlKQoESfLKJ9y+cPM2bYXEchvF8ZMo6cywUTxvwaEmPPGgvXfB8CoGx6NYjx2IsC+F 6Y7CRjL9kSVxqKSOt6iIYZGvDSfecciiUbSKZq0Pf9vjxLVIjy/vblaHMxPhJRfxEsKd HpK2dCXsTiCjGtjSajh6ZrsOD3vb+sROh6jH0xilt5Ay3VDdV1xADGGZbX/n1H8MlZtP 3yH2PEu8hqVPGh6ATrQJhd9t9hNe/g+O6vCok8dwClS0V4GVOmsT5loK0wBo1/YLp4xc TjOq3FZHBPIhNrNcxzqeT961qGboZCr886l0xZgFOzNANuYZkbPo++txmvckGro2fzjH ySpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:message-id:in-reply-to:subject :cc:to:from:date:dkim-signature; bh=Gn5WzFahKDFmpa30/2NDbGP8DwU+8tgzl9BF6DVEeRU=; b=rCfn6jBYNIbupRaGz8yx46q7QrK+VDX5qajES5LtcPJYEyj5+GOEMEjlc32yBk/lP9 OFZuaoMHqj3a0zhZ2B0L0koav7uZIEZI3p/938a0RgrQWKq86K4+05Aykbp3ZHjNF4F2 KlVs74TsFs15tHdtFNHofkHs/tANgeQB6TMOaiwMwnyIMfb7XwpiZw7jRa245STreJds v4QlMRhyt0+mw3YtfVmQ5GgfL7j3cUWtVJvO0+jKRKTYj6TsIEkLaFi1C9qHWHKsnN5H r9mpoF/Bu2xpU11HNzjHAK8Wa8UklztyUTPNbJUlGYuWs4KdrtrRqqIdzqd6WJXJf8dS xnOg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=d7JkLxnu; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id c84sor2876236pfe.49.2018.11.26.15.33.16 for (Google Transport Security); Mon, 26 Nov 2018 15:33:16 -0800 (PST) Received-SPF: pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=d7JkLxnu; spf=pass (google.com: domain of hughd@google.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=hughd@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=Gn5WzFahKDFmpa30/2NDbGP8DwU+8tgzl9BF6DVEeRU=; b=d7JkLxnu9k7CIQwnZzNVNTqJYebgfKTts683dxF7n4UvUqyJDHcsKIxUoZrLbF4j0i n6BcXscIHeA8NncV15JObW0fJlFOyNekTbDZZ5bgyI+0B7kqFy1vVEoa/yatfyfg9RwZ SnHYLmyjLWG5NkTotJWEPRCJDF9vhJ1V1+WSofeoBdv0V8uLUgJwtykkkNEsSy4jfq5W iCRpFK83CMSdjwxVyWcXFteVarvtk1nrQikQddte6BsWLs+8m52p4lL1xj4eJsqYvvJ5 6nd/p+Oo6d9joT1+Rf9BPYMIzxEW6Wv9RgsEsCE8WwRqMgCFUEQqk5E0aMiESCvun4EJ RhtA== X-Google-Smtp-Source: AJdET5f50/WgEk/RfZl8C0LYOZc+o7YOeYAz9JSfv2TN/XIGb5Extn+Zl6CFyrTa6P+OsN4nPcFGug== X-Received: by 2002:a62:9111:: with SMTP id l17mr30171177pfe.200.1543275195826; Mon, 26 Nov 2018 15:33:15 -0800 (PST) Received: from [100.112.89.103] ([104.133.8.103]) by smtp.gmail.com with ESMTPSA id c13sm2028740pfo.121.2018.11.26.15.33.14 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Nov 2018 15:33:15 -0800 (PST) Date: Mon, 26 Nov 2018 15:33:13 -0800 (PST) From: Hugh Dickins X-X-Sender: hugh@eggly.anvils To: Andrew Morton cc: "Kirill A. Shutemov" , "Kirill A. Shutemov" , Matthew Wilcox , linux-mm@kvack.org Subject: [PATCH 10/10] mm/khugepaged: fix the xas_create_range() error path In-Reply-To: Message-ID: References: User-Agent: Alpine 2.11 (LSU 23 2013-08-11) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP collapse_shmem()'s xas_nomem() is very unlikely to fail, but it is rightly given a failure path, so move the whole xas_create_range() block up before __SetPageLocked(new_page): so that it does not need to remember to unlock_page(new_page). Add the missing mem_cgroup_cancel_charge(), and set (currently unused) result to SCAN_FAIL rather than SCAN_SUCCEED. Fixes: 77da9389b9d5 ("mm: Convert collapse_shmem to XArray") Signed-off-by: Hugh Dickins Cc: Matthew Wilcox Cc: Kirill A. Shutemov Acked-by: Kirill A. Shutemov --- mm/khugepaged.c | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 2c5fe4f7a0c6..8e2ff195ecb3 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1329,6 +1329,20 @@ static void collapse_shmem(struct mm_struct *mm, goto out; } + /* This will be less messy when we use multi-index entries */ + do { + xas_lock_irq(&xas); + xas_create_range(&xas); + if (!xas_error(&xas)) + break; + xas_unlock_irq(&xas); + if (!xas_nomem(&xas, GFP_KERNEL)) { + mem_cgroup_cancel_charge(new_page, memcg, true); + result = SCAN_FAIL; + goto out; + } + } while (1); + __SetPageLocked(new_page); __SetPageSwapBacked(new_page); new_page->index = start; @@ -1340,17 +1354,6 @@ static void collapse_shmem(struct mm_struct *mm, * be able to map it or use it in another way until we unlock it. */ - /* This will be less messy when we use multi-index entries */ - do { - xas_lock_irq(&xas); - xas_create_range(&xas); - if (!xas_error(&xas)) - break; - xas_unlock_irq(&xas); - if (!xas_nomem(&xas, GFP_KERNEL)) - goto out; - } while (1); - xas_set(&xas, start); for (index = start; index < end; index++) { struct page *page = xas_next(&xas);