From patchwork Sat Aug 4 13:29:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tetsuo Handa X-Patchwork-Id: 10555771 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7CCE0157D for ; Sat, 4 Aug 2018 13:30:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6B6DC29C2E for ; Sat, 4 Aug 2018 13:30:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5FC0829C34; Sat, 4 Aug 2018 13:30:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4CFE629C2E for ; Sat, 4 Aug 2018 13:30:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A9B716B000D; Sat, 4 Aug 2018 09:30:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A21816B000E; Sat, 4 Aug 2018 09:30:15 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C2C86B0010; Sat, 4 Aug 2018 09:30:15 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) by kanga.kvack.org (Postfix) with ESMTP id 44D4A6B000D for ; Sat, 4 Aug 2018 09:30:15 -0400 (EDT) Received: by mail-pg1-f197.google.com with SMTP id 132-v6so3743171pga.18 for ; Sat, 04 Aug 2018 06:30:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=6Q6MJCkn0qaKi0kQwwfDppqn1K3XrIvUs7xwrrEeua4=; b=QXWgEH+FqCB9EVozY52rfWCEOdopSHzUfNDKXMRMrzgaP/qko9ZNWYmabqvlc+BHHp h0XKyKJ/sZ1iIuJ5UltPODWHLs6vIhvYmecbUIHD4rECYLDUCh9Zt9nvfSXuvJZsM9vT bt3A9QEgoVp5MpjzsqEePtqttlGK5AU7f75I3FccJg3X8gmsS5+lUHaa3PHH4TuxZLEf +44F7fbQEmc8aIYT6uyH9mp+w9QXcWOKVG1ribA/itb2FkLpojHUQvTV4hFYsAwS3m5d dFfwF5Br9kkIG5PNZc10HBE3HR2qCmV+j5KZxjstjeegxtYYK1/VSdk8WHLlnSEaZV44 oODg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of penguin-kernel@i-love.sakura.ne.jp designates 202.181.97.72 as permitted sender) smtp.mailfrom=penguin-kernel@i-love.sakura.ne.jp X-Gm-Message-State: AOUpUlHURapJ3QVv/0AKufngbGw6jgTKbKQQY+hnx+qcQgA8AxTNlDcm GCHr8lWjSqqW1RJnyezjJqKKTtOPiHkyAQyjHx1PF7q2bfA97V5VS4TUJBa8MJdg4DckLWPxXLG SyOg8u8MHdcUxaDH32AWgKpGhfNeFEcPsh8oCANHI372nIeKHA4fXSRVVKeCjakj6qg== X-Received: by 2002:a17:902:6907:: with SMTP id j7-v6mr7319077plk.323.1533389414927; Sat, 04 Aug 2018 06:30:14 -0700 (PDT) X-Google-Smtp-Source: AAOMgpfmRUvY3xixJhly52R2Me1q2gC+lzkqxC1+MBsYtyWOkVhQUEQR8XNKglbsKDgyqGvpk2K8 X-Received: by 2002:a17:902:6907:: with SMTP id j7-v6mr7318952plk.323.1533389412231; Sat, 04 Aug 2018 06:30:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1533389412; cv=none; d=google.com; s=arc-20160816; b=iEA5ZF4fJYhO1VgaJ4dR2sN2o7vyNB9hH6wFNOfWXLi8fsqqnG40oFLGf2gwbfSvUi r0RvatpzNV1JHgboRCUhbRlImxO/seaOwjS0MmW3PXl9gN1GA4oXKHDBP7VtyKioCXPb Sk7h3BKBm8NhaLTR9boicKf8iXpFLwaN4q1a1551vU2wBakjU7EtIT3jlLxI6NJL0qhB SJQJh0JycgjcB0UtZZAf9+CNu1Hq/uY2yGjk/+MgF0n8bbSZYkfuHAvgS5ihYzOLSdFj 7cUi78XOl6dUNEw89d0Njpx/ofUZZRNmLLGPv0w8cMEcf9WiqlJXRHjk9mrwQ+xXE+U7 VDNA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=6Q6MJCkn0qaKi0kQwwfDppqn1K3XrIvUs7xwrrEeua4=; b=n7d0xhL+GgQ0BhmMmniAjdYp6PpoWa8yPjIhAdLB2F3cxJ1V2cibFZeh4MXe0qu+6D NViCq0GaaLsZqS06L1xg+pWh4dBQHpnlr95Ny41BKuKrYEst7DbLp9JTL7kNgq2eSQlV /A2bGqptgjOyEW7laylpPjIB2UBVRz5tlwXjeWYIi5jj3lOWpIqgtyW/AG0Jwma9so3c ZlNuFQ3+ARy365zFILVPvKpVIAM3AvN0zgbnkNpUMQj1fknqMRwL4UsQKrxuzPbdEFCX KCfta5HnWzojdPlAytAfkAC/G36W1lr9H9TaS5PFYh8ivIDZgsJUI84K2mFagoNWsqQW yCnQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of penguin-kernel@i-love.sakura.ne.jp designates 202.181.97.72 as permitted sender) smtp.mailfrom=penguin-kernel@i-love.sakura.ne.jp Received: from www262.sakura.ne.jp (www262.sakura.ne.jp. [202.181.97.72]) by mx.google.com with ESMTPS id 2-v6si7850000pgq.479.2018.08.04.06.30.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 04 Aug 2018 06:30:12 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of penguin-kernel@i-love.sakura.ne.jp designates 202.181.97.72 as permitted sender) client-ip=202.181.97.72; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of penguin-kernel@i-love.sakura.ne.jp designates 202.181.97.72 as permitted sender) smtp.mailfrom=penguin-kernel@i-love.sakura.ne.jp Received: from fsav105.sakura.ne.jp (fsav105.sakura.ne.jp [27.133.134.232]) by www262.sakura.ne.jp (8.15.2/8.15.2) with ESMTP id w74DTn7p005268; Sat, 4 Aug 2018 22:29:49 +0900 (JST) (envelope-from penguin-kernel@I-love.SAKURA.ne.jp) Received: from www262.sakura.ne.jp (202.181.97.72) by fsav105.sakura.ne.jp (F-Secure/fsigk_smtp/530/fsav105.sakura.ne.jp); Sat, 04 Aug 2018 22:29:49 +0900 (JST) X-Virus-Status: clean(F-Secure/fsigk_smtp/530/fsav105.sakura.ne.jp) Received: from ccsecurity.localdomain (softbank126074194044.bbtec.net [126.74.194.44]) (authenticated bits=0) by www262.sakura.ne.jp (8.15.2/8.15.2) with ESMTPSA id w74DTiDJ005246 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 4 Aug 2018 22:29:49 +0900 (JST) (envelope-from penguin-kernel@I-love.SAKURA.ne.jp) From: Tetsuo Handa To: linux-mm@kvack.org Cc: Tetsuo Handa , David Rientjes , Michal Hocko , Roman Gushchin Subject: [PATCH 2/4] mm, oom: Check pending victims earlier in out_of_memory(). Date: Sat, 4 Aug 2018 22:29:44 +0900 Message-Id: <1533389386-3501-2-git-send-email-penguin-kernel@I-love.SAKURA.ne.jp> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1533389386-3501-1-git-send-email-penguin-kernel@I-love.SAKURA.ne.jp> References: <1533389386-3501-1-git-send-email-penguin-kernel@I-love.SAKURA.ne.jp> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Regarding CONFIG_MMU=y case, we have a list of inflight OOM victim threads which are chained to oom_reaper_list. Therefore, by doing the same thing for CONFIG_MMU=n case, we can check whether there are inflight OOM victims before starting process/memcg list traversal. Since it is likely that only few threads are chained to oom_reaper_list, checking all victims' OOM domain will not matter. Thus, check whether there are inflight OOM victims before starting process/memcg list traversal. To do so, we need to chain OOM victims until MMF_OOM_SKIP is set. Thus, this patch changes the OOM reaper to wait for an request from the OOM killer using oom_reap_target variable. This change allows the OOM reaper to preferentially reclaim from mm which the OOM killer is waiting for the OOM reaper to reclaim. Signed-off-by: Tetsuo Handa Cc: Michal Hocko Cc: David Rientjes Cc: Roman Gushchin --- include/linux/oom.h | 1 + include/linux/sched.h | 4 +-- kernel/fork.c | 2 ++ mm/oom_kill.c | 97 +++++++++++++++++++++++++++++---------------------- 4 files changed, 60 insertions(+), 44 deletions(-) diff --git a/include/linux/oom.h b/include/linux/oom.h index 69864a5..4a147871 100644 --- a/include/linux/oom.h +++ b/include/linux/oom.h @@ -104,6 +104,7 @@ extern unsigned long oom_badness(struct task_struct *p, extern bool out_of_memory(struct oom_control *oc); extern void exit_oom_victim(void); +extern void exit_oom_mm(struct mm_struct *mm); extern int register_oom_notifier(struct notifier_block *nb); extern int unregister_oom_notifier(struct notifier_block *nb); diff --git a/include/linux/sched.h b/include/linux/sched.h index 9e686dc..589fe78 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1173,9 +1173,7 @@ struct task_struct { unsigned long task_state_change; #endif int pagefault_disabled; -#ifdef CONFIG_MMU - struct task_struct *oom_reaper_list; -#endif + struct list_head oom_victim_list; #ifdef CONFIG_VMAP_STACK struct vm_struct *stack_vm_area; #endif diff --git a/kernel/fork.c b/kernel/fork.c index 276fdc6..ba1260d 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -1010,6 +1010,8 @@ static inline void __mmput(struct mm_struct *mm) } if (mm->binfmt) module_put(mm->binfmt->module); + if (unlikely(mm_is_oom_victim(mm))) + exit_oom_mm(mm); mmdrop(mm); } diff --git a/mm/oom_kill.c b/mm/oom_kill.c index dad0409..a743a8e 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -321,18 +321,6 @@ static int oom_evaluate_task(struct task_struct *task, void *arg) goto next; /* - * This task already has access to memory reserves and is being killed. - * Don't allow any other task to have access to the reserves unless - * the task has MMF_OOM_SKIP because chances that it would release - * any memory is quite low. - */ - if (!is_sysrq_oom(oc) && tsk_is_oom_victim(task)) { - if (test_bit(MMF_OOM_SKIP, &task->signal->oom_mm->flags)) - goto next; - goto abort; - } - - /* * If task is allocating a lot of memory and has been marked to be * killed first if it triggers an oom, then select it. */ @@ -356,11 +344,6 @@ static int oom_evaluate_task(struct task_struct *task, void *arg) oc->chosen_points = points; next: return 0; -abort: - if (oc->chosen) - put_task_struct(oc->chosen); - oc->chosen = (void *)-1UL; - return 1; } /* @@ -478,6 +461,8 @@ bool process_shares_mm(struct task_struct *p, struct mm_struct *mm) return false; } +static LIST_HEAD(oom_victim_list); + #ifdef CONFIG_MMU /* * OOM Reaper kernel thread which tries to reap the memory used by the OOM @@ -485,7 +470,7 @@ bool process_shares_mm(struct task_struct *p, struct mm_struct *mm) */ static struct task_struct *oom_reaper_th; static DECLARE_WAIT_QUEUE_HEAD(oom_reaper_wait); -static struct task_struct *oom_reaper_list; +static struct task_struct *oom_reap_target; bool __oom_reap_task_mm(struct mm_struct *mm) { @@ -598,33 +583,21 @@ static void oom_reap_task(struct task_struct *tsk) debug_show_all_locks(); done: - tsk->oom_reaper_list = NULL; - /* * Hide this mm from OOM killer because it has been either reaped or * somebody can't call up_write(mmap_sem). */ set_bit(MMF_OOM_SKIP, &mm->flags); - - /* Drop a reference taken by mark_oom_victim(). */ - put_task_struct(tsk); } static int oom_reaper(void *unused) { while (true) { - struct task_struct *tsk = NULL; - - wait_event_freezable(oom_reaper_wait, oom_reaper_list != NULL); - mutex_lock(&oom_lock); - if (oom_reaper_list != NULL) { - tsk = oom_reaper_list; - oom_reaper_list = tsk->oom_reaper_list; - } - mutex_unlock(&oom_lock); - - if (tsk) - oom_reap_task(tsk); + wait_event_freezable(oom_reaper_wait, oom_reap_target != NULL); + oom_reap_task(oom_reap_target); + /* Drop a reference taken by oom_has_pending_victims(). */ + put_task_struct(oom_reap_target); + oom_reap_target = NULL; } return 0; @@ -661,13 +634,8 @@ static void mark_oom_victim(struct task_struct *tsk) if (!cmpxchg(&tsk->signal->oom_mm, NULL, mm)) { mmgrab(tsk->signal->oom_mm); set_bit(MMF_OOM_VICTIM, &mm->flags); -#ifdef CONFIG_MMU get_task_struct(tsk); - tsk->oom_reaper_list = oom_reaper_list; - oom_reaper_list = tsk; - trace_wake_reaper(tsk->pid); - wake_up(&oom_reaper_wait); -#endif + list_add(&tsk->oom_victim_list, &oom_victim_list); } /* @@ -681,6 +649,21 @@ static void mark_oom_victim(struct task_struct *tsk) trace_mark_victim(tsk->pid); } +void exit_oom_mm(struct mm_struct *mm) +{ + struct task_struct *p, *tmp; + + mutex_lock(&oom_lock); + list_for_each_entry_safe(p, tmp, &oom_victim_list, oom_victim_list) { + if (mm != p->signal->oom_mm) + continue; + list_del(&p->oom_victim_list); + /* Drop a reference taken by mark_oom_victim(). */ + put_task_struct(p); + } + mutex_unlock(&oom_lock); +} + /** * exit_oom_victim - note the exit of an OOM victim */ @@ -1020,6 +1003,35 @@ int unregister_oom_notifier(struct notifier_block *nb) } EXPORT_SYMBOL_GPL(unregister_oom_notifier); +static bool oom_has_pending_victims(struct oom_control *oc) +{ + struct task_struct *p; + + if (is_sysrq_oom(oc)) + return false; + /* + * Since oom_reap_task()/exit_mmap() will set MMF_OOM_SKIP, let's + * wait for pending victims until MMF_OOM_SKIP is set or __mmput() + * completes. + */ + list_for_each_entry(p, &oom_victim_list, oom_victim_list) { + if (oom_unkillable_task(p, oc->memcg, oc->nodemask)) + continue; + if (!test_bit(MMF_OOM_SKIP, &p->signal->oom_mm->flags)) { +#ifdef CONFIG_MMU + if (!oom_reap_target) { + get_task_struct(p); + oom_reap_target = p; + trace_wake_reaper(p->pid); + wake_up(&oom_reaper_wait); + } +#endif + return true; + } + } + return false; +} + /** * out_of_memory - kill the "best" process when we run out of memory * @oc: pointer to struct oom_control @@ -1072,6 +1084,9 @@ bool out_of_memory(struct oom_control *oc) oc->nodemask = NULL; check_panic_on_oom(oc, constraint); + if (oom_has_pending_victims(oc)) + return true; + if (!is_memcg_oom(oc) && sysctl_oom_kill_allocating_task && current->mm && !oom_unkillable_task(current, NULL, oc->nodemask) && current->signal->oom_score_adj != OOM_SCORE_ADJ_MIN) {