From patchwork Sat Feb 22 14:46:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 11398117 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9DE7A92A for ; Sat, 22 Feb 2020 14:47:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5C587208C3 for ; Sat, 22 Feb 2020 14:47:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TbpoZYf/" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5C587208C3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6A0E26B0005; Sat, 22 Feb 2020 09:47:00 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 651F36B0006; Sat, 22 Feb 2020 09:47:00 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 519766B0007; Sat, 22 Feb 2020 09:47:00 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0249.hostedemail.com [216.40.44.249]) by kanga.kvack.org (Postfix) with ESMTP id 364876B0005 for ; Sat, 22 Feb 2020 09:47:00 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id D4DC84FF4 for ; Sat, 22 Feb 2020 14:46:59 +0000 (UTC) X-FDA: 76518040158.09.price53_77399b796c82e X-Spam-Summary: 2,0,0,36de5bf51c2093d4,d41d8cd98f00b204,laoar.shao@gmail.com,,RULES_HIT:2:41:355:379:541:800:960:973:988:989:1260:1345:1437:1535:1605:1730:1747:1777:1792:2393:2559:2562:2910:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4049:4119:4250:4321:4605:5007:6119:6261:6653:7514:7903:8957:9040:9413:10004:10226:11026:11232:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12663:12986:13161:13229:13972:14096:14394:14687:14877:21080:21444:21450:21451:21627:21666:21740:21966:21990:30003:30045:30054:30056,0,RBL:209.85.216.65:@gmail.com:.lbl8.mailshell.net-66.100.201.100 62.50.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: price53_77399b796c82e X-Filterd-Recvd-Size: 8950 Received: from mail-pj1-f65.google.com (mail-pj1-f65.google.com [209.85.216.65]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Sat, 22 Feb 2020 14:46:59 +0000 (UTC) Received: by mail-pj1-f65.google.com with SMTP id j17so2080124pjz.3 for ; Sat, 22 Feb 2020 06:46:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=ZUN0QoqtAwCcbcCvyhddxIhhjY3uc6WZiHY0zOqSM+Y=; b=TbpoZYf/2g3eTjRTPvtLsW2nyH712Iejhztj4Jz2/6ahmfmRoDAoj+yc4EFMx122qv IZ3usqbgQbsmq6BQPOkTJuYe5KjlVZJ1/S36Gq+VEMrxwnGWWCeVZxI4h43E0HsM/iER iDQV/ZmGKe7rADWAZPydKkU1J1z9d3eqyEMMyub8GWYyTv2S7JS2mm6nGaJZcJ9x2SJY /tnlKe41FS6RDBW93MpyeyDqSHttRsPrlHeCFaYhBZUQcx4jVZpGRWDlVA1Vh2Lk/acd r3mpsVfF9FMFU6YmJl6UH0HadYA6aSSo8I7Tll7CpZ+7E+JjKTU6IyQr7LgHEJmmEWDM VLAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=ZUN0QoqtAwCcbcCvyhddxIhhjY3uc6WZiHY0zOqSM+Y=; b=A7LiGm69RXV2nZ56ureUpn/RAk1ZsiqmHXj3UW7gRTr86YSbbJVa02QuIRyirCoYRQ zQKnlgXvRg7nrnYMo5EIsa5DYBzalj3oO4Vxr2In8eEc7RgfNPHxXeJcc6eAWyvcX0/y aGXNjGdC9XptkAd3ytccLWu9/f6bZIS7riFDhS+WXF4Yqt5yVzQ4d+AzBrUUWn394iZY ++QVm6x9rk7TQbBngEv0ZCXMPivf37Y7HbgIuH2U3QbColbt/3DkjTqOCnwUyOKeGBoy 6NWU0egAu9Eq/DuBwwSFHhULfSedcjLpHKXDRwPtRUvjc0/y1unW9sfsQ6cOC+if2vUk arkA== X-Gm-Message-State: APjAAAVHOLT+fIHFuNQ+6HfbZ6ephIsmXmV7QpHWHeUxoDxEdEGUReyY yVBQozo3TKmKGG9kHmUG0ns= X-Google-Smtp-Source: APXvYqx8tLUlcYFBP6kg8jKYvnw0BkcEB7L1Tz8BmC/Jxy/DM520ZLaRX8cRjHeQ/A6cskS/LYWXeQ== X-Received: by 2002:a17:90b:3011:: with SMTP id hg17mr9397539pjb.90.1582382818060; Sat, 22 Feb 2020 06:46:58 -0800 (PST) Received: from snappy-silo-1.localdomain ([107.182.183.158]) by smtp.gmail.com with ESMTPSA id hg11sm6179226pjb.14.2020.02.22.06.46.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 22 Feb 2020 06:46:57 -0800 (PST) From: Yafang Shao To: hannes@cmpxchg.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yafang Shao Subject: [PATCH] psi: move PF_MEMSTALL into psi specific psi_flags Date: Sat, 22 Feb 2020 09:46:47 -0500 Message-Id: <20200222144647.10120-1-laoar.shao@gmail.com> X-Mailer: git-send-email 2.18.2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The task->flags is a 32-bits flag, in which 31 bits have already been consumed. So it is hardly to introduce other new per process flag. As there's a psi specific flag psi_flags, we'd better move the psi specific per process flag PF_MEMSTALL into it. Signed-off-by: Yafang Shao --- include/linux/psi_types.h | 12 +++++++++++- include/linux/sched.h | 7 +++++-- kernel/sched/psi.c | 15 ++++++++------- kernel/sched/stats.h | 10 +++++----- 4 files changed, 29 insertions(+), 15 deletions(-) diff --git a/include/linux/psi_types.h b/include/linux/psi_types.h index 07aaf9b82241..411dbbf57d51 100644 --- a/include/linux/psi_types.h +++ b/include/linux/psi_types.h @@ -17,11 +17,21 @@ enum psi_task_count { NR_PSI_TASK_COUNTS = 3, }; -/* Task state bitmasks */ +/* + * Task state bitmasks: + * These flags are stored in the lower PSI_TSK_BITS bits of + * task->psi_flags, and the higher bits are set with per process flag which + * persists across sleeps. + */ +#define PSI_TSK_STATE_BITS 16 +#define PSI_TSK_STATE_MASK ((1 << PSI_TSK_STATE_BITS) - 1) #define TSK_IOWAIT (1 << NR_IOWAIT) #define TSK_MEMSTALL (1 << NR_MEMSTALL) #define TSK_RUNNING (1 << NR_RUNNING) +/* Stalled due to lack of memory, that's per process flag. */ +#define PSI_PF_MEMSTALL (1 << PSI_TSK_STATE_BITS) + /* Resources that workloads could be stalled on */ enum psi_res { PSI_IO, diff --git a/include/linux/sched.h b/include/linux/sched.h index f314790cb527..2d4c04d35d9b 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1025,7 +1025,11 @@ struct task_struct { struct task_io_accounting ioac; #ifdef CONFIG_PSI - /* Pressure stall state */ + /* + * Pressure stall state: + * Bits 0 ~ PSI_TSK_STATE_BITS-1: PSI task states + * Bits PSI_TSK_STATE_BITS ~ 31: Per process flags + */ unsigned int psi_flags; #endif #ifdef CONFIG_TASK_XACCT @@ -1490,7 +1494,6 @@ extern struct pid *cad_pid; #define PF_KTHREAD 0x00200000 /* I am a kernel thread */ #define PF_RANDOMIZE 0x00400000 /* Randomize virtual address space */ #define PF_SWAPWRITE 0x00800000 /* Allowed to write to swap */ -#define PF_MEMSTALL 0x01000000 /* Stalled due to lack of memory */ #define PF_UMH 0x02000000 /* I'm an Usermodehelper process */ #define PF_NO_SETAFFINITY 0x04000000 /* Userland is not allowed to meddle with cpus_mask */ #define PF_MCE_EARLY 0x08000000 /* Early kill for mce process policy */ diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c index 028520702717..34363fc77ecc 100644 --- a/kernel/sched/psi.c +++ b/kernel/sched/psi.c @@ -759,7 +759,8 @@ void psi_task_change(struct task_struct *task, int clear, int set) !psi_bug) { printk_deferred(KERN_ERR "psi: inconsistent task state! task=%d:%s cpu=%d psi_flags=%x clear=%x set=%x\n", task->pid, task->comm, cpu, - task->psi_flags, clear, set); + task->psi_flags & PSI_TSK_STATE_MASK, + clear, set); psi_bug = 1; } @@ -818,17 +819,17 @@ void psi_memstall_enter(unsigned long *flags) if (static_branch_likely(&psi_disabled)) return; - *flags = current->flags & PF_MEMSTALL; + *flags = current->psi_flags & PSI_PF_MEMSTALL; if (*flags) return; /* - * PF_MEMSTALL setting & accounting needs to be atomic wrt + * PSI_PF_MEMSTALL setting & accounting needs to be atomic wrt * changes to the task's scheduling state, otherwise we can * race with CPU migration. */ rq = this_rq_lock_irq(&rf); - current->flags |= PF_MEMSTALL; + current->psi_flags |= PSI_PF_MEMSTALL; psi_task_change(current, 0, TSK_MEMSTALL); rq_unlock_irq(rq, &rf); @@ -851,13 +852,13 @@ void psi_memstall_leave(unsigned long *flags) if (*flags) return; /* - * PF_MEMSTALL clearing & accounting needs to be atomic wrt + * PSI_PF_MEMSTALL clearing & accounting needs to be atomic wrt * changes to the task's scheduling state, otherwise we could * race with CPU migration. */ rq = this_rq_lock_irq(&rf); - current->flags &= ~PF_MEMSTALL; + current->psi_flags &= ~PSI_PF_MEMSTALL; psi_task_change(current, TSK_MEMSTALL, 0); rq_unlock_irq(rq, &rf); @@ -921,7 +922,7 @@ void cgroup_move_task(struct task_struct *task, struct css_set *to) else if (task->in_iowait) task_flags = TSK_IOWAIT; - if (task->flags & PF_MEMSTALL) + if (task->psi_flags & PSI_PF_MEMSTALL) task_flags |= TSK_MEMSTALL; if (task_flags) diff --git a/kernel/sched/stats.h b/kernel/sched/stats.h index ba683fe81a6e..164f97b1ce7f 100644 --- a/kernel/sched/stats.h +++ b/kernel/sched/stats.h @@ -70,7 +70,7 @@ static inline void psi_enqueue(struct task_struct *p, bool wakeup) return; if (!wakeup || p->sched_psi_wake_requeue) { - if (p->flags & PF_MEMSTALL) + if (p->psi_flags & PSI_PF_MEMSTALL) set |= TSK_MEMSTALL; if (p->sched_psi_wake_requeue) p->sched_psi_wake_requeue = 0; @@ -90,7 +90,7 @@ static inline void psi_dequeue(struct task_struct *p, bool sleep) return; if (!sleep) { - if (p->flags & PF_MEMSTALL) + if (p->psi_flags & PSI_PF_MEMSTALL) clear |= TSK_MEMSTALL; } else { if (p->in_iowait) @@ -109,14 +109,14 @@ static inline void psi_ttwu_dequeue(struct task_struct *p) * deregister its sleep-persistent psi states from the old * queue, and let psi_enqueue() know it has to requeue. */ - if (unlikely(p->in_iowait || (p->flags & PF_MEMSTALL))) { + if (unlikely(p->in_iowait || (p->psi_flags & PSI_PF_MEMSTALL))) { struct rq_flags rf; struct rq *rq; int clear = 0; if (p->in_iowait) clear |= TSK_IOWAIT; - if (p->flags & PF_MEMSTALL) + if (p->psi_flags & PSI_PF_MEMSTALL) clear |= TSK_MEMSTALL; rq = __task_rq_lock(p, &rf); @@ -131,7 +131,7 @@ static inline void psi_task_tick(struct rq *rq) if (static_branch_likely(&psi_disabled)) return; - if (unlikely(rq->curr->flags & PF_MEMSTALL)) + if (unlikely(rq->curr->psi_flags & PSI_PF_MEMSTALL)) psi_memstall_tick(rq->curr, cpu_of(rq)); } #else /* CONFIG_PSI */