From patchwork Sat Apr 15 00:08:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13212214 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7CE4C77B72 for ; Sat, 15 Apr 2023 00:08:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1A46C900003; Fri, 14 Apr 2023 20:08:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 154D7900002; Fri, 14 Apr 2023 20:08:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 04361900003; Fri, 14 Apr 2023 20:08:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id EAD38900002 for ; Fri, 14 Apr 2023 20:08:24 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id B4CB21A0449 for ; Sat, 15 Apr 2023 00:08:24 +0000 (UTC) X-FDA: 80681688528.18.B66B016 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf23.hostedemail.com (Postfix) with ESMTP id 1438B14000F for ; Sat, 15 Apr 2023 00:08:22 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=HXjWck7P; spf=pass (imf23.hostedemail.com: domain of 39uo5ZAYKCNsPROBK8DLLDIB.9LJIFKRU-JJHS79H.LOD@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=39uo5ZAYKCNsPROBK8DLLDIB.9LJIFKRU-JJHS79H.LOD@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681517303; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=HTZMn+gdePQ6w7ytfojAbFuEgA1Exnj51R99edjgsk4=; b=GBuXTJ/1JNYpEB2Z38Ps+5VGME8rb3Xtd2w3MFPxsaahUu+vcQKFWJMfGSnURf+0Yd363l C1bDOkFkFSe6REPjFCBwAJT82WsjfiS8Oi80MjBeDO9HUON0ok3eVDR5UBKaLlHONX6/XL rsc5wHIZCcReuIXxz0R1OntbYwgifkQ= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=HXjWck7P; spf=pass (imf23.hostedemail.com: domain of 39uo5ZAYKCNsPROBK8DLLDIB.9LJIFKRU-JJHS79H.LOD@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=39uo5ZAYKCNsPROBK8DLLDIB.9LJIFKRU-JJHS79H.LOD@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681517303; a=rsa-sha256; cv=none; b=YBHYStwMFiKIEp0j0wJYf1BljntUuq5w3qdym/8n5c2/rNNUQevZwS9bHqUcSVs/JtRuu2 n4JgZt2q2WSCQaIlNo/+nH6Uj1qvZUXMQ3qaiYUrQyEO3hRl43fTeD+LhzHGdUo2x8LgXy FQalNfBxo2Pu3TVe+tkb65JwioqZUEs= Received: by mail-yb1-f201.google.com with SMTP id h206-20020a2521d7000000b00b8f3681db1eso10688010ybh.11 for ; Fri, 14 Apr 2023 17:08:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681517302; x=1684109302; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=HTZMn+gdePQ6w7ytfojAbFuEgA1Exnj51R99edjgsk4=; b=HXjWck7PIwn2MOmDJ61gXB6mzp/zcBEjD3VXA3dtP3JJsKBX66dn/fLDKzYLSzjpIV uB6IAS5hoNhvCHiUomWn2VQqw2IPnD5EyGKpS2wAoHPc6fmCfv5WdV4AdkiAndxqG0/f A3iGSOwBT+Wc9uyp37v4l1yGWipIaT8+Fddu6G6fOKU+GvtsudqwOPNOy+I2UADevQPI P6HALgBGheXxZj4ZF4896PfGAnRZtV9TnQ8x06UepUb5yCvAn51CJ7jiAOvqNUJUJpOy P7bkZdc8Fc1YaoWGW3I/Npq2iAY8MJJCIYd4bz5+Vt3agxk4OVsmono5B9lYJtW3B4Ql Kn0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681517302; x=1684109302; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=HTZMn+gdePQ6w7ytfojAbFuEgA1Exnj51R99edjgsk4=; b=fqUf6+29Sv2+j905oAwv2ahPV93J7RMXS67kFjp8VJfh6fRqCseAmjBZs1HL+W4sv2 sT5wX5UyWsG8acyllBylUokHEZAl1Ar/dV4teRRAw4Ze+DV2nI7KiZGMHvkOXK03offf BBnhzb5AnSjHG/+JIdGTXofIzUPA/0S0E2gSE6A3YuJIygf31EQukUO4eFtyamfLCbJW ARjqY79XYabajDaS0FruvJhITS6aiUstjIyHe0eGkkf0R2Ljf07/MtGyJi5/+67AgX3Z eRjb8RK1fXCOnbYLIFKl4LtKSOUMaYax5uD1fIyzp5i8q8tcyH3OyWKHw7qnQ9cVL2tg TgYg== X-Gm-Message-State: AAQBX9dwAcEbMPHFPc273auxtf6Ci6/qyb8Hf+ZkKMkao9AkdKgKZqft GIJJ5lly2WGEvs64ZKZJoYnYZPDWDv8= X-Google-Smtp-Source: AKy350Zdqn2SuZ84MXlUvPdnfLvMmZOEwQDahaWw8QbW+PES4oLAtpiiNIVM9mruaQNMWSGM6qirf/zEdBQ= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:46c0:7584:f020:e09f]) (user=surenb job=sendgmr) by 2002:a81:b71c:0:b0:54f:b931:adf7 with SMTP id v28-20020a81b71c000000b0054fb931adf7mr5042050ywh.4.1681517302208; Fri, 14 Apr 2023 17:08:22 -0700 (PDT) Date: Fri, 14 Apr 2023 17:08:18 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog Message-ID: <20230415000818.1955007-1-surenb@google.com> Subject: [PATCH v2 1/1] mm: do not increment pgfault stats when page fault handler retries From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: willy@infradead.org, hannes@cmpxchg.org, mhocko@suse.com, josef@toxicpanda.com, jack@suse.cz, ldufour@linux.ibm.com, laurent.dufour@fr.ibm.com, michel@lespinasse.org, liam.howlett@oracle.com, jglisse@google.com, vbabka@suse.cz, minchan@google.com, dave@stgolabs.net, punit.agrawal@bytedance.com, lstoakes@gmail.com, surenb@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com X-Stat-Signature: 96qm4hzbf97ksuua5rerem4koixx8yc9 X-Rspam-User: X-Rspamd-Queue-Id: 1438B14000F X-Rspamd-Server: rspam06 X-HE-Tag: 1681517302-393175 X-HE-Meta: U2FsdGVkX19b+4KhrwQOo0HoYxjQTAMMPUyXgs9Ybj1WRAystdzG2eYsN0kA7KOS+JGGdIg7gkRUUmuZuJOxnCeniElKqAFU4J3Ht1cm+vPV+udRDCV9iG2rL/HAsvLni/dIJWoZ4DPg9M7Q2eSRs2rqNSqdAPMvJvGyI6ppq3pVBefW/oCWSFr8a6IG8vAjCZehsyK/eWnS4sllEmRyQMt1LWtmjR65L+oP3zaNhtkiaNBAKL0tRQKATc2sJVxEpjPCUhaws2nEDQCHe2VRPWGtY8/ta7a/84AKBAx4BDw2ckQni6OJ35YcMlQJeWCEuy8M/o2lECWnWjKg0it8ZlwCg+9bAOO2fIdg5InCxRrN5Ha3rCE7ute/G/360FPejzVl69eeimdpPgN9ZLnkt0TDz4RWeyP25ZuRdj3DRdEUIvs5oAYfCRgIcCD2fNFzU+V6oYA17jqFdmAS3wv7+9WvCXDpJHbEh1IpjnrjDAgUGbNH0dQIgTmLY2OO/KiAnsNGgz+rA1auJIMXd9YMqM5YVeRE5mYq4dJvqHPcx4drfXunFCN6fCcZGvJW8YupJexAfO2/9b9mmY+DudW7fgirocTdNAL35fPGl32e/WgqX9OLaSSCD3Y1a2NEO7VhpGobWJKA1wssQM/sBRV4kpTHSTTgu/KsQzX2EzIGhpHE47CI7QC5vD8fu7rukraJPw5QGIWB8jSnq/LkaWY58X/1ObkGewdO3IyBwllZ4wJo9px3YKwcjer/66n37UeCPaKXUqu2Q9T9mjSeXJOCQZBRuvPK+uqSUFOAX8DUbW1rtjfcMyQrYgv+/Dam3/Rl7MVF7Z6FjMczMKK1LD3cZMvC1F2zF34OjCXR44UapSSak4ETeZ3gNq5wUPZ4sEH8GZr5SpFRsqp+eb/MKClEuitEQHbtlX5O62SwA7efXUM7GnNfUiNk+tpPvA9qS/gwzaIvHe2G4ElfIb5MeFa tOHAxAi7 /7hVPiB0ck1DxaX8mNUHrkUAppK4tn9zadDX7chaJjfFEUuFOi8kdtYOGN8zfzoCLcvrAkhBAAJOTIJrflR5wvP69+VKn/YqYGbk2+nJgrkktq1KVqlSF0jS7Ad9Zn2jp/9Y5nucXth+8hiuXOKcR30lC2+OZ+wdr+I4KWMEaFRMQP65fhtRM0JomSq379gHtj1d/wFupiZk3ndYibg8UE0ClFeYYBNUuZf1VFoZfM6HpnU+WnwBDdpLqh7N/tZMxpXt2mmM4rxAPsTjokpbqS0k/X8WGvBd+ih2TeN3FP2ivNYn/JXRW9MsliT8rfawo27pPyXNXwnz5aifR27lO7HV1mnd7Ekkj1Z/rXl4MTe30/yuRO4Zlh0wr+x4l1qZgMPvwoHY+PUUKUxFbUPf+FyQ0XjHRaF/1sl5tiP4MlD9HeAfcjANSZcr/Ap5Dq92PuspowYHh3wG0yQLNlHSuM4tOr7Atot3ofjguEgbTrjm+HmqTM3F+8hA1BfeMfATQDBaBlh6fOqGVuJxWJKBFxNOgrJ2cc+T+0Kfpj5Ybl5mkZxqdDwgh2H5rpAYr2XWwaNmR5CRwRwEqbmlQGP9dL5CFNiERYyswDbyTPcrOjmnWq+2y1MCiVoytNw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If the page fault handler requests a retry, we will count the fault multiple times. This is a relatively harmless problem as the retry paths are not often requested, and the only user-visible problem is that the fault counter will be slightly higher than it should be. Nevertheless, userspace only took one fault, and should not see the fact that the kernel had to retry the fault multiple times. Move page fault accounting into mm_account_fault() and skip incomplete faults which will be accounted upon completion. Fixes: d065bd810b6d ("mm: retry page fault when blocking on disk transfer") Signed-off-by: Suren Baghdasaryan --- mm/memory.c | 45 ++++++++++++++++++++++++++------------------- 1 file changed, 26 insertions(+), 19 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 01a23ad48a04..c3b709ceeed7 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5080,24 +5080,30 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, * updates. However, note that the handling of PERF_COUNT_SW_PAGE_FAULTS should * still be in per-arch page fault handlers at the entry of page fault. */ -static inline void mm_account_fault(struct pt_regs *regs, +static inline void mm_account_fault(struct mm_struct *mm, struct pt_regs *regs, unsigned long address, unsigned int flags, vm_fault_t ret) { bool major; /* - * We don't do accounting for some specific faults: - * - * - Unsuccessful faults (e.g. when the address wasn't valid). That - * includes arch_vma_access_permitted() failing before reaching here. - * So this is not a "this many hardware page faults" counter. We - * should use the hw profiling for that. - * - * - Incomplete faults (VM_FAULT_RETRY). They will only be counted - * once they're completed. + * Do not account for incomplete faults (VM_FAULT_RETRY). They will be + * counted upon completion. */ - if (ret & (VM_FAULT_ERROR | VM_FAULT_RETRY)) + if (ret & VM_FAULT_RETRY) + return; + + /* Register both successful and failed faults in PGFAULT counters. */ + count_vm_event(PGFAULT); + count_memcg_event_mm(mm, PGFAULT); + + /* + * Do not account for unsuccessful faults (e.g. when the address wasn't + * valid). That includes arch_vma_access_permitted() failing before + * reaching here. So this is not a "this many hardware page faults" + * counter. We should use the hw profiling for that. + */ + if (ret & VM_FAULT_ERROR) return; /* @@ -5180,21 +5186,22 @@ static vm_fault_t sanitize_fault_flags(struct vm_area_struct *vma, vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, unsigned int flags, struct pt_regs *regs) { + /* Copy vma->vm_mm in case mmap_lock is dropped and vma becomes unstable. */ + struct mm_struct *mm = vma->vm_mm; vm_fault_t ret; __set_current_state(TASK_RUNNING); - count_vm_event(PGFAULT); - count_memcg_event_mm(vma->vm_mm, PGFAULT); - ret = sanitize_fault_flags(vma, &flags); if (ret) - return ret; + goto out; if (!arch_vma_access_permitted(vma, flags & FAULT_FLAG_WRITE, flags & FAULT_FLAG_INSTRUCTION, - flags & FAULT_FLAG_REMOTE)) - return VM_FAULT_SIGSEGV; + flags & FAULT_FLAG_REMOTE)) { + ret = VM_FAULT_SIGSEGV; + goto out; + } /* * Enable the memcg OOM handling for faults triggered in user @@ -5223,8 +5230,8 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, if (task_in_memcg_oom(current) && !(ret & VM_FAULT_OOM)) mem_cgroup_oom_synchronize(false); } - - mm_account_fault(regs, address, flags, ret); +out: + mm_account_fault(mm, regs, address, flags, ret); return ret; }