From patchwork Mon Dec 12 00:37:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 13070695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AAE4C4332F for ; Mon, 12 Dec 2022 00:37:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B5B078E0006; Sun, 11 Dec 2022 19:37:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B318B8E0002; Sun, 11 Dec 2022 19:37:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9AB9C8E0006; Sun, 11 Dec 2022 19:37:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 8BE4E8E0002 for ; Sun, 11 Dec 2022 19:37:48 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 5B8841C5C32 for ; Mon, 12 Dec 2022 00:37:48 +0000 (UTC) X-FDA: 80231791416.04.194263E Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf09.hostedemail.com (Postfix) with ESMTP id BC5FF14000F for ; Mon, 12 Dec 2022 00:37:46 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=aylXhwHR; spf=pass (imf09.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670805466; a=rsa-sha256; cv=none; b=BrIaoZeKLcmphBFHG+usdJkAb15qrGNmuaoK5Tfh167fqady3MSuWCMeOMt+T3G3S6mjMd yfNPAEXIundVtX2cqAQMSi7l1pumUKyzHM7VbkDw8xVeB+oEQNGppFzFDcepslO+WzDQQs 8Aw+slniXdt9+B6CMXJXgwIA8A/miTg= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=aylXhwHR; spf=pass (imf09.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670805466; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rGtC3SN6Ifn7NlhGu9Dw24wgt0iqbWERqtVVRikZ7FU=; b=5rsEHrEw8Vdf0hy7h8zZT1TCET4MVIWCG3mA5NzI4luLNV0gd5hnDwoD+r5hRBmi6mEKGu m9cK9WaHyHVifVnH29uU2m7CSGL8VbdKYzlyr/Pg14hQZmfkfITfgBe33n77ftRWldjbh8 dIr3c3V9KXlBheTUWZPag03m5HZztk4= Received: by mail-pl1-f180.google.com with SMTP id d3so10410187plr.10 for ; Sun, 11 Dec 2022 16:37:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rGtC3SN6Ifn7NlhGu9Dw24wgt0iqbWERqtVVRikZ7FU=; b=aylXhwHRB/AnaqV0nbRReAnU29I+73XtbQYit2HtT6Nf4V02lXAmWcUcXfAMaKKMyW ongHgiHPo/1Yo/hBjs1Yonph+tyD/hp39THh63O0jAOOrpoVfVW/gFXkc1QVjqQ1sYiL Q5Pobduy9ttgieGJeoJ3ggbEV9QYNbSZ2mX4u1ANqmQlGpY9GKRtcV7baGzZ6FgIVx+r +kOxzFrlGcZCgssVQNCEtuJh9I0VFURrNHQW8x1ib2kSYXZO8Wpiu8uGtc2/lvGfclsb 8VSviT9bHd5lIOEooS7oy7agNzaiKd6m4bpfeDCtz+Mex01Uu0FR1DKPDPgjyDTOXIqa XzEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rGtC3SN6Ifn7NlhGu9Dw24wgt0iqbWERqtVVRikZ7FU=; b=uKTUarUhYTOPRtWqf1VU7R45uzIhtXXl4XCOV0nhK5oCRf8SYf0/tKAhAfCHn20Oiv +cuDrjK5GRXvclqoIft3VWsq4SQG7/x4E74QwMit4KckA/gD2MbcFjYZw2z+MagzB43s fhR9e2yF8mfmNwLeq9BGdy2YYDzJbzECL2ql4MWPEpQmo7VnW5IBURxe40EhstsydE2H ujndknipX0wMcpqDabEsOlg7j3yyjvYikdbj9o99bnXrnTqCMSpvdavjd91SUQp2bMim 1A1j060RooNrfQ/TLwjlJwrePfyYEjpetBj9JQ+xxSYyvA05JyQz4yIhr4y+lwVi7hdn EIPg== X-Gm-Message-State: ANoB5pkcfU/Afw+J42sw1vopDYPe9oy9OmY9B7uvAYqU67Lzm7spm/tk HLluGKxiVGe9YNJa7G1PvKA= X-Google-Smtp-Source: AA0mqf5OlxYfuHjLp+MFFLeSNr64rxHfH89FJoaBzG1fbT222yfAv8sK9MnrXVllA3XEBIYOCtCxTg== X-Received: by 2002:a05:6a20:a681:b0:ac:1266:bda with SMTP id ba1-20020a056a20a68100b000ac12660bdamr19043512pzb.7.1670805466209; Sun, 11 Dec 2022 16:37:46 -0800 (PST) Received: from vultr.guest ([2001:19f0:7002:8c7:5400:4ff:fe3d:656a]) by smtp.gmail.com with ESMTPSA id w9-20020a170902e88900b00177fb862a87sm4895960plg.20.2022.12.11.16.37.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 11 Dec 2022 16:37:45 -0800 (PST) From: Yafang Shao To: ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kafai@fb.com, songliubraving@fb.com, yhs@fb.com, john.fastabend@gmail.com, kpsingh@kernel.org, sdf@google.com, haoluo@google.com, jolsa@kernel.org, tj@kernel.org, dennis@kernel.org, cl@linux.com, akpm@linux-foundation.org, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com Cc: linux-mm@kvack.org, bpf@vger.kernel.org, Yafang Shao Subject: [RFC PATCH bpf-next 2/9] mm: Allow using active vm in all contexts Date: Mon, 12 Dec 2022 00:37:04 +0000 Message-Id: <20221212003711.24977-3-laoar.shao@gmail.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20221212003711.24977-1-laoar.shao@gmail.com> References: <20221212003711.24977-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: BC5FF14000F X-Stat-Signature: g9ye74e4x9gpu485i758zwrw5siuohmc X-HE-Tag: 1670805466-374049 X-HE-Meta: U2FsdGVkX1+r5Eu9eFfPaXM0eaY8jN3m2Kypr61Sn8ggqeM2izFKzSNX2SXpuO5IoHxYcQuK8pPf8rF1wBAPGzhlv2+FXT5yFdmQkrRPJ2LKdf+65SVC3XSHznyem/9pbZp37IoXEQvNttJCjH3+uyMml5Uyv7Mcz9Yq2sBrDgP0mL1sEL4GPudLLpaAexXc19bhHc1gD0s5G4baHGQpcHvUWiew7uDlbjZTDmWNQCzRX1KZTF5ieYqKKs9Lu6t7lDb8P+pKtPoLNfjKbbpXytpP8ltJeQOH+RgZ/ObR0HHdvIyVg/K9xLqHx79pTVssQuqFMva2ksz3ZfcB2NVcpLfrxQpMrJvCh9LdbW5Au9CFgl63wn7qlYtKGX43AQBRKFd3aV9KgfrZFmK/JQD2XGWdQQpfEZy4pX5ijQWWOz/EYtjRrt5Hum5jH4c5+MowTtaG4y/WbImuwVYcYb7CuYEXSA+rlR+VRBhyr/4vEEEvvqjw73aOLo5w7sNNe3vYDV9h/8rhD6ZVlxcP43Qj3Dwss+Z+oAmjyZVDeoth5M5IMHCXTTU2jSbppeeSbjZ/XDiB5H2z4f/HeyZv8AUdldbSnFTWGAq21I6JQ44FuQYkm5l5K5A6+P844dRTJ8zc6wQ+OXXaKoTZ7wkGNA1wrA2B/F7HTQ5nrC1DRfgCRDlOLfAF2nF1JZx02qaN1+Yzrmol8+Tokfy742ITSpSO0QDLLitUYYHMdmfctdOHbOj15nHOVflYN9+7igaZPnIsUD03Filn/i85p/d4AZ+7sUpxMYXS2Do8vEeLd+cgIq7aSZGM8oG3inGWkQc5SitQPLs3GexonbPh2GOBERifvCCH+n8jsn/bmx0y+S3xaqQdCjyPtFApV/soTsjRxNummsSF9zxzqLBCsrte+tgws1DMOm7Tb/EwqKFlorlefoWyGbUYKDQZ0i+/6X55O9mVIGLv1TQvIkCyv3A/nb2 cknmCzjX Z5DpmyxfMxTJkyHuxWPQo6EwWyfi52NZkwi7QWmgSMgsuto5mYY8Mv7p/zNJ7ZHC3XWgm4d+mzoU8KYwlZEL4D/ex60c4p3BCW11wO/OpWSTaibfNjnXLO5u5SV43qnbis55Zp9BjjXI48+sQ2gx2ungqAB5V+kUc2RPxBOJ8WQXWgzWbVxevoMi+MYXAFRgQtmdLF4bzsMn4MyehpDPnRB74N4Krbm17X8Vzmt5PKTADd3g= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We can use active vm in task, softirq and irq to account specific memory usage. Two new helpers active_vm_{add,sub} are introduced. A dummy item is introduced here, which will be removed when we introduce real item. Signed-off-by: Yafang Shao --- include/linux/active_vm.h | 50 +++++++++++++++++++++++++++++++++++++++ include/linux/sched.h | 5 ++++ kernel/fork.c | 4 ++++ mm/active_vm.c | 23 ++++++++++++++++++ mm/active_vm.h | 38 +++++++++++++++++++++++++++++ 5 files changed, 120 insertions(+) diff --git a/include/linux/active_vm.h b/include/linux/active_vm.h index 899e578e94fa..21f9aaca12c4 100644 --- a/include/linux/active_vm.h +++ b/include/linux/active_vm.h @@ -4,6 +4,9 @@ #ifdef CONFIG_ACTIVE_VM #include +#include +#include +#include extern struct static_key_true active_vm_disabled; @@ -14,10 +17,57 @@ static inline bool active_vm_enabled(void) return true; } + +enum active_vm_item { + DUMMY_ITEM = 1, + NR_ACTIVE_VM_ITEM = DUMMY_ITEM, +}; + +struct active_vm_stat { + long stat[NR_ACTIVE_VM_ITEM]; +}; + +DECLARE_PER_CPU(struct active_vm_stat, active_vm_stats); +DECLARE_PER_CPU(int, irq_active_vm_item); +DECLARE_PER_CPU(int, soft_active_vm_item); + +static inline int +active_vm_item_set(int item) +{ + int old_item; + + if (in_irq()) { + old_item = this_cpu_read(irq_active_vm_item); + this_cpu_write(irq_active_vm_item, item); + } else if (in_softirq()) { + old_item = this_cpu_read(soft_active_vm_item); + this_cpu_write(soft_active_vm_item, item); + } else { + old_item = current->active_vm_item; + current->active_vm_item = item; + } + + return old_item; +} + +long active_vm_item_sum(int item); + #else static inline bool active_vm_enabled(void) { return false; } + +static inline int +active_vm_item_set(int item) +{ + return 0; +} + +static inline long active_vm_item_sum(int item) +{ + return 0; +} + #endif /* CONFIG_ACTIVE_VM */ #endif /* __INCLUDE_ACTIVE_VM_H */ diff --git a/include/linux/sched.h b/include/linux/sched.h index ffb6eb55cd13..05acefd383d4 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1441,6 +1441,11 @@ struct task_struct { struct mem_cgroup *active_memcg; #endif +#ifdef CONFIG_ACTIVE_VM + /* Used for scope-based memory accounting */ + int active_vm_item; +#endif + #ifdef CONFIG_BLK_CGROUP struct request_queue *throttle_queue; #endif diff --git a/kernel/fork.c b/kernel/fork.c index 08969f5aa38d..590d949ff131 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -1043,6 +1043,10 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node) tsk->active_memcg = NULL; #endif +#ifdef CONFIG_ACTIVE_VM + tsk->active_vm_item = 0; +#endif + #ifdef CONFIG_CPU_SUP_INTEL tsk->reported_split_lock = 0; #endif diff --git a/mm/active_vm.c b/mm/active_vm.c index 60849930a7d3..541b2ba22da9 100644 --- a/mm/active_vm.c +++ b/mm/active_vm.c @@ -1,5 +1,6 @@ // SPDX-License-Identifier: GPL-2.0 #include +#include static bool __active_vm_enabled __initdata = IS_ENABLED(CONFIG_ACTIVE_VM); @@ -31,3 +32,25 @@ struct page_ext_operations active_vm_ops = { .need = need_active_vm, .init = init_active_vm, }; + +DEFINE_PER_CPU(int, irq_active_vm_item); +DEFINE_PER_CPU(int, soft_active_vm_item); +EXPORT_PER_CPU_SYMBOL(irq_active_vm_item); +EXPORT_PER_CPU_SYMBOL(soft_active_vm_item); +DEFINE_PER_CPU(struct active_vm_stat, active_vm_stats); +EXPORT_PER_CPU_SYMBOL(active_vm_stats); + +long active_vm_item_sum(int item) +{ + struct active_vm_stat *this; + long sum = 0; + int cpu; + + WARN_ON_ONCE(item <= 0); + for_each_online_cpu(cpu) { + this = &per_cpu(active_vm_stats, cpu); + sum += this->stat[item - 1]; + } + + return sum; +} diff --git a/mm/active_vm.h b/mm/active_vm.h index 72978955833e..1df088d768ef 100644 --- a/mm/active_vm.h +++ b/mm/active_vm.h @@ -3,6 +3,44 @@ #define __MM_ACTIVE_VM_H #ifdef CONFIG_ACTIVE_VM +#include + extern struct page_ext_operations active_vm_ops; + +static inline int active_vm_item(void) +{ + if (in_irq()) + return this_cpu_read(irq_active_vm_item); + + if (in_softirq()) + return this_cpu_read(soft_active_vm_item); + + return current->active_vm_item; +} + +static inline void active_vm_item_add(int item, long delta) +{ + WARN_ON_ONCE(item <= 0); + this_cpu_add(active_vm_stats.stat[item - 1], delta); +} + +static inline void active_vm_item_sub(int item, long delta) +{ + WARN_ON_ONCE(item <= 0); + this_cpu_sub(active_vm_stats.stat[item - 1], delta); +} +#else /* CONFIG_ACTIVE_VM */ +static inline int active_vm_item(void) +{ + return 0; +} + +static inline void active_vm_item_add(int item, long delta) +{ +} + +static inline void active_vm_item_sub(int item, long delta) +{ +} #endif /* CONFIG_ACTIVE_VM */ #endif /* __MM_ACTIVE_VM_H */