From patchwork Fri Apr 1 18:21:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12798589 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2F88C433EF for ; Fri, 1 Apr 2022 18:23:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C8F348D0008; Fri, 1 Apr 2022 14:21:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C3D2E8D0003; Fri, 1 Apr 2022 14:21:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B04B48D0008; Fri, 1 Apr 2022 14:21:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id A37DA8D0003 for ; Fri, 1 Apr 2022 14:21:18 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 723EE24D5B for ; Fri, 1 Apr 2022 18:21:08 +0000 (UTC) X-FDA: 79309127016.11.34022A9 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf15.hostedemail.com (Postfix) with ESMTP id B41BFA002B for ; Fri, 1 Apr 2022 18:21:07 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7BFA160DFC; Fri, 1 Apr 2022 18:21:07 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CC7B0C2BBE4; Fri, 1 Apr 2022 18:21:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1648837266; bh=2hQH4xIfEI7cKb8n8lwC1lNjdFUqMLTsrFO0c/gJyxI=; h=Date:To:From:In-Reply-To:Subject:From; b=GnseGmscywRYaqMi8sVobJCE51GyukTlo/RKGWXRtoUuggCnLkN2CVYBQq25kjBtj 6iqB/EZMV/69BcRpfsIVJ+KqR1f+IgS/Q3RnJQFkTCDMn7qk41R37nM8dp9HbttKAK +6Q67auFFqb7LkkRvnz2lA5ncudI28cSg1dFzR7Y= Date: Fri, 01 Apr 2022 11:21:06 -0700 To: vbabka@suse.cz,efault@gmx.de,hughd@google.com,akpm@linux-foundation.org,patches@lists.linux.dev,linux-mm@kvack.org,mm-commits@vger.kernel.org,torvalds@linux-foundation.org,akpm@linux-foundation.org From: Andrew Morton In-Reply-To: Subject: [patch 06/16] mm/munlock: add lru_add_drain() to fix memcg_stat_test Message-Id: <20220401182106.CC7B0C2BBE4@smtp.kernel.org> X-Stat-Signature: mrzjy7grotc9n8dx3n1ee9idw3njxu5x Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=GnseGmsc; dmarc=none; spf=pass (imf15.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: B41BFA002B X-HE-Tag: 1648837267-605263 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Hugh Dickins Subject: mm/munlock: add lru_add_drain() to fix memcg_stat_test Mike reports that LTP memcg_stat_test usually leads to... memcg_stat_test 3 TINFO: Test unevictable with MAP_LOCKED memcg_stat_test 3 TINFO: Running memcg_process --mmap-lock1 -s 135168 memcg_stat_test 3 TINFO: Warming up pid: 3460 memcg_stat_test 3 TINFO: Process is still here after warm up: 3460 memcg_stat_test 3 TFAIL: unevictable is 122880, 135168 expected ...but may lead to... memcg_stat_test 4 TINFO: Test unevictable with mlock memcg_stat_test 4 TINFO: Running memcg_process --mmap-lock2 -s 135168 memcg_stat_test 4 TINFO: Warming up pid: 4271 memcg_stat_test 4 TINFO: Process is still here after warm up: 4271 memcg_stat_test 4 TFAIL: unevictable is 122880, 135168 expected ...or both. A wee bit flaky. follow_page_pte() used to have an lru_add_drain() per each page mlocked, and the test came to rely on accurate stats. The pagevec to be drained is different now, but still covered by lru_add_drain(); and, never mind the test, I believe it's in everyone's interest that a bulk faulting interface like populate_vma_page_range() or faultin_vma_page_range() should drain its local pagevecs at the end, to save others sometimes needing the much more expensive lru_add_drain_all(). This does not absolutely guarantee exact stats - the mlocking task can be migrated between CPUs as it proceeds - but it's good enough and the tests pass. Link: https://lkml.kernel.org/r/47f6d39c-a075-50cb-1cfb-26dd957a48af@google.com Fixes: b67bf49ce7aa ("mm/munlock: delete FOLL_MLOCK and FOLL_POPULATE") Signed-off-by: Hugh Dickins Reported-by: Mike Galbraith Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- mm/gup.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) --- a/mm/gup.c~mm-munlock-add-lru_add_drain-to-fix-memcg_stat_test +++ a/mm/gup.c @@ -1404,6 +1404,7 @@ long populate_vma_page_range(struct vm_a struct mm_struct *mm = vma->vm_mm; unsigned long nr_pages = (end - start) / PAGE_SIZE; int gup_flags; + long ret; VM_BUG_ON(!PAGE_ALIGNED(start)); VM_BUG_ON(!PAGE_ALIGNED(end)); @@ -1438,8 +1439,10 @@ long populate_vma_page_range(struct vm_a * We made sure addr is within a VMA, so the following will * not result in a stack expansion that recurses back here. */ - return __get_user_pages(mm, start, nr_pages, gup_flags, + ret = __get_user_pages(mm, start, nr_pages, gup_flags, NULL, NULL, locked); + lru_add_drain(); + return ret; } /* @@ -1471,6 +1474,7 @@ long faultin_vma_page_range(struct vm_ar struct mm_struct *mm = vma->vm_mm; unsigned long nr_pages = (end - start) / PAGE_SIZE; int gup_flags; + long ret; VM_BUG_ON(!PAGE_ALIGNED(start)); VM_BUG_ON(!PAGE_ALIGNED(end)); @@ -1498,8 +1502,10 @@ long faultin_vma_page_range(struct vm_ar if (check_vma_flags(vma, gup_flags)) return -EINVAL; - return __get_user_pages(mm, start, nr_pages, gup_flags, + ret = __get_user_pages(mm, start, nr_pages, gup_flags, NULL, NULL, locked); + lru_add_drain(); + return ret; } /*