From patchwork Fri Nov 12 08:38:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: CGEL X-Patchwork-Id: 12616381 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50F25C433F5 for ; Fri, 12 Nov 2021 08:38:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DE92D60FE3 for ; Fri, 12 Nov 2021 08:38:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org DE92D60FE3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 781866B0074; Fri, 12 Nov 2021 03:38:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 731DD6B0078; Fri, 12 Nov 2021 03:38:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5FA036B007B; Fri, 12 Nov 2021 03:38:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0187.hostedemail.com [216.40.44.187]) by kanga.kvack.org (Postfix) with ESMTP id 51A246B0074 for ; Fri, 12 Nov 2021 03:38:49 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 14DD51815EB5F for ; Fri, 12 Nov 2021 08:38:49 +0000 (UTC) X-FDA: 78799627578.23.89D7118 Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by imf26.hostedemail.com (Postfix) with ESMTP id 4A70620019E2 for ; Fri, 12 Nov 2021 08:38:49 +0000 (UTC) Received: by mail-pj1-f45.google.com with SMTP id o6-20020a17090a0a0600b001a64b9a11aeso7012065pjo.3 for ; Fri, 12 Nov 2021 00:38:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=THvKTAD+2ygLpLxemj7xlxNSJCoUu3plQl2okiS9kUM=; b=UYpKXRbBtLQVwoz2vZk9P8VPvOrO0CzaHClnda2haPmLFzUkDB6JiDoybpH4f45rpD Ufu5ILp5NBbwh2w7QFw/LQyKp1INvEHjsBJjvydRGkhqYB/PKWogyLOwQdecN/adyI6z HFeE+dGFphy2S4HpVFHsw2byfarevPvQxfzuW9N2hl5gjdMBc4Wk2CthOm7iIEtolIrU ydrgmPMijAa+k9ze1NhVTD/7rYx8vOH0GZk2s0VzXiymE9MUSuuGZ54N/MNo2F94+DyH ju5D5G4sz7aeQsbE6z7EmoOM4GfBBsfM1Mc/UAnJiwRKdbIZV5ZzzkWS8qAm5XhVYuVJ R9BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=THvKTAD+2ygLpLxemj7xlxNSJCoUu3plQl2okiS9kUM=; b=V4A/SMqFyeRfiZWPIokKTS3G/K9HLLvsjWAwdHNK8bcMiKjgvtvEqphSxdpuJbKnHx dpICxs/bh+PVZu2IFRh+HXZFVfRIxFwiBLGeT2ef7n3xrFdDTDEDwOx0IUZtqtXpEzCY w+Uq67KTge437EIK2kycOkdS4AWm3UoDnymXhoWtYVzijyVyo0neq1/UjHAns9kM/sHs LENhRSmCysWsCYU4sxZaGgR2wskz+amB6C47f5dzBuXDTrLrD0023eIyt+7iKK5tPwOs ZXQpp5ZzyP+nZADPJhrDUJD5fW9Ko1WhIKZg9jJtZbeI1cG5vhv/nnLlx3MQXooyxrNF oaBg== X-Gm-Message-State: AOAM530iKeP5Ucz1DhFa27zYnML3lYlc5NQxxfhWyLBtueymfm0QU8Qp Ld2k2EypZmVgDZ0l+CS+E80= X-Google-Smtp-Source: ABdhPJwttMqMHe+SwBdmxvgraPSXuel1XIyVnfgtklPzvhkNkV/yNDejxFOr3K05M3OJd2ld+VLb4g== X-Received: by 2002:a17:90b:1a86:: with SMTP id ng6mr34145817pjb.142.1636706327619; Fri, 12 Nov 2021 00:38:47 -0800 (PST) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id f5sm4425010pju.15.2021.11.12.00.38.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Nov 2021 00:38:47 -0800 (PST) From: cgel.zte@gmail.com X-Google-Original-From: yang.yang29@zte.com.cn To: bsingharora@gmail.com, akpm@linux-foundation.org, peterz@infradead.org, mingo@kernel.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Yang Yang , Zeal Robot Subject: [PATCH] delayacct: Support swapin delay accounting for swapping without blkio Date: Fri, 12 Nov 2021 08:38:14 +0000 Message-Id: <20211112083813.8559-1-yang.yang29@zte.com.cn> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 4A70620019E2 X-Stat-Signature: jw7rnnkkec3d5mj8h5gcz7x6o1yi6wkz Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=UYpKXRbB; spf=pass (imf26.hostedemail.com: domain of cgel.zte@gmail.com designates 209.85.216.45 as permitted sender) smtp.mailfrom=cgel.zte@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1636706329-919408 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Yang Currently delayacct accounts swapin delay only for swapping that cause blkio. If we use zram for swapping, tools/accounting/getdelays can't get any SWAP delay. It's useful to get zram swapin delay information, for example to adjust compress algorithm or /proc/sys/vm/swappiness. Reference to PSI, it accounts any kind of swapping by doing it's work in swap_readpage(), no matter whether swapping cause blkio. Let delayacct do the similar work. Reported-by: Zeal Robot Signed-off-by: Yang Yang --- include/linux/delayacct.h | 44 +++++++++++++++++++-------------------- kernel/delayacct.c | 34 +++++++++++++++++------------- mm/memory.c | 4 ---- mm/page_io.c | 3 +++ 4 files changed, 44 insertions(+), 41 deletions(-) diff --git a/include/linux/delayacct.h b/include/linux/delayacct.h index af7e6eb50283..b96d68f310a2 100644 --- a/include/linux/delayacct.h +++ b/include/linux/delayacct.h @@ -9,14 +9,6 @@ #include -/* - * Per-task flags relevant to delay accounting - * maintained privately to avoid exhausting similar flags in sched.h:PF_* - * Used to set current->delays->flags - */ -#define DELAYACCT_PF_SWAPIN 0x00000001 /* I am doing a swapin */ -#define DELAYACCT_PF_BLKIO 0x00000002 /* I am waiting on IO */ - #ifdef CONFIG_TASK_DELAY_ACCT struct task_delay_info { raw_spinlock_t lock; @@ -37,13 +29,13 @@ struct task_delay_info { * associated with the operation is added to XXX_delay. * XXX_delay contains the accumulated delay time in nanoseconds. */ - u64 blkio_start; /* Shared by blkio, swapin */ + u64 blkio_start; u64 blkio_delay; /* wait for sync block io completion */ - u64 swapin_delay; /* wait for swapin block io completion */ + u64 swapin_start; + u64 swapin_delay; /* wait for swapin */ u32 blkio_count; /* total count of the number of sync block */ /* io operations performed */ - u32 swapin_count; /* total count of the number of swapin block */ - /* io operations performed */ + u32 swapin_count; /* total count of swapin */ u64 freepages_start; u64 freepages_delay; /* wait for memory reclaim */ @@ -79,14 +71,8 @@ extern void __delayacct_freepages_start(void); extern void __delayacct_freepages_end(void); extern void __delayacct_thrashing_start(void); extern void __delayacct_thrashing_end(void); - -static inline int delayacct_is_task_waiting_on_io(struct task_struct *p) -{ - if (p->delays) - return (p->delays->flags & DELAYACCT_PF_BLKIO); - else - return 0; -} +extern void __delayacct_swapin_start(void); +extern void __delayacct_swapin_end(void); static inline void delayacct_set_flag(struct task_struct *p, int flag) { @@ -123,7 +109,6 @@ static inline void delayacct_blkio_start(void) if (!static_branch_unlikely(&delayacct_key)) return; - delayacct_set_flag(current, DELAYACCT_PF_BLKIO); if (current->delays) __delayacct_blkio_start(); } @@ -135,7 +120,6 @@ static inline void delayacct_blkio_end(struct task_struct *p) if (p->delays) __delayacct_blkio_end(p); - delayacct_clear_flag(p, DELAYACCT_PF_BLKIO); } static inline __u64 delayacct_blkio_ticks(struct task_struct *tsk) @@ -169,6 +153,18 @@ static inline void delayacct_thrashing_end(void) __delayacct_thrashing_end(); } +static inline void delayacct_swapin_start(void) +{ + if (current->delays) + __delayacct_swapin_start(); +} + +static inline void delayacct_swapin_end(void) +{ + if (current->delays) + __delayacct_swapin_end(); +} + #else static inline void delayacct_set_flag(struct task_struct *p, int flag) {} @@ -199,6 +195,10 @@ static inline void delayacct_thrashing_start(void) {} static inline void delayacct_thrashing_end(void) {} +static inline void delayacct_swapin_start(void) +{} +static inline void delayacct_swapin_end(void) +{} #endif /* CONFIG_TASK_DELAY_ACCT */ diff --git a/kernel/delayacct.c b/kernel/delayacct.c index 51530d5b15a8..11f3cd82d7e4 100644 --- a/kernel/delayacct.c +++ b/kernel/delayacct.c @@ -100,19 +100,10 @@ void __delayacct_blkio_start(void) */ void __delayacct_blkio_end(struct task_struct *p) { - struct task_delay_info *delays = p->delays; - u64 *total; - u32 *count; - - if (p->delays->flags & DELAYACCT_PF_SWAPIN) { - total = &delays->swapin_delay; - count = &delays->swapin_count; - } else { - total = &delays->blkio_delay; - count = &delays->blkio_count; - } - - delayacct_end(&delays->lock, &delays->blkio_start, total, count); + delayacct_end(&p->delays->lock, + &p->delays->blkio_start, + &p->delays->blkio_delay, + &p->delays->blkio_count); } int delayacct_add_tsk(struct taskstats *d, struct task_struct *tsk) @@ -179,8 +170,7 @@ __u64 __delayacct_blkio_ticks(struct task_struct *tsk) unsigned long flags; raw_spin_lock_irqsave(&tsk->delays->lock, flags); - ret = nsec_to_clock_t(tsk->delays->blkio_delay + - tsk->delays->swapin_delay); + ret = nsec_to_clock_t(tsk->delays->blkio_delay); raw_spin_unlock_irqrestore(&tsk->delays->lock, flags); return ret; } @@ -210,3 +200,17 @@ void __delayacct_thrashing_end(void) ¤t->delays->thrashing_delay, ¤t->delays->thrashing_count); } + +void __delayacct_swapin_start(void) +{ + current->delays->swapin_start = local_clock(); +} + +void __delayacct_swapin_end(void) +{ + delayacct_end(¤t->delays->lock, + ¤t->delays->swapin_start, + ¤t->delays->swapin_delay, + ¤t->delays->swapin_count); +} + diff --git a/mm/memory.c b/mm/memory.c index 39e7a1495c3c..8379fcda2b9f 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3515,7 +3515,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (unlikely(!si)) goto out; - delayacct_set_flag(current, DELAYACCT_PF_SWAPIN); page = lookup_swap_cache(entry, vma, vmf->address); swapcache = page; @@ -3563,7 +3562,6 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) vmf->address, &vmf->ptl); if (likely(pte_same(*vmf->pte, vmf->orig_pte))) ret = VM_FAULT_OOM; - delayacct_clear_flag(current, DELAYACCT_PF_SWAPIN); goto unlock; } @@ -3577,13 +3575,11 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) * owner processes (which may be unknown at hwpoison time) */ ret = VM_FAULT_HWPOISON; - delayacct_clear_flag(current, DELAYACCT_PF_SWAPIN); goto out_release; } locked = lock_page_or_retry(page, vma->vm_mm, vmf->flags); - delayacct_clear_flag(current, DELAYACCT_PF_SWAPIN); if (!locked) { ret |= VM_FAULT_RETRY; goto out_release; diff --git a/mm/page_io.c b/mm/page_io.c index d597bc6e6e45..cf8549433afd 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -25,6 +25,7 @@ #include #include #include +#include void end_swap_bio_write(struct bio *bio) { @@ -372,6 +373,7 @@ int swap_readpage(struct page *page, bool synchronous) * significant part of overall IO time. */ psi_memstall_enter(&pflags); + delayacct_swapin_start(); if (frontswap_load(page) == 0) { SetPageUptodate(page); @@ -436,6 +438,7 @@ int swap_readpage(struct page *page, bool synchronous) out: psi_memstall_leave(&pflags); + delayacct_swapin_end(); return ret; }