From patchwork Tue Jul 3 16:01:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Paul E. McKenney" X-Patchwork-Id: 10504609 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id BD340602BC for ; Tue, 3 Jul 2018 15:58:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A9F6B28D48 for ; Tue, 3 Jul 2018 15:58:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9D82728D51; Tue, 3 Jul 2018 15:58:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F15E228D48 for ; Tue, 3 Jul 2018 15:58:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F011B6B0003; Tue, 3 Jul 2018 11:58:56 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EB0F86B0005; Tue, 3 Jul 2018 11:58:56 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA1436B0006; Tue, 3 Jul 2018 11:58:56 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-wr0-f197.google.com (mail-wr0-f197.google.com [209.85.128.197]) by kanga.kvack.org (Postfix) with ESMTP id 7E7CB6B0003 for ; Tue, 3 Jul 2018 11:58:56 -0400 (EDT) Received: by mail-wr0-f197.google.com with SMTP id s15-v6so1192725wrn.16 for ; Tue, 03 Jul 2018 08:58:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:date:from:to :cc:subject:reply-to:references:mime-version:content-disposition :in-reply-to:user-agent:message-id; bh=P0rvnx8tQpdGXJrHq+E5FJgc2sxhjb5hwTd7J7eGB/4=; b=cnFHKiGnUtAsELYU7ubR7Qi2S+T7rf4nhMl/opnDPtOpRo1NNoylntNXA63zsMNLWA nDJTK9WbV6uyPxXhHHACKo6mmfwT0L2qa35YNNJEd6Qzbhw7WEo2aImGyqFTeXGPYkYQ v1WIkZPG2bUyKfBrXuv9VODU1bMiqEUfga8ezsyZo7pt5Rj5RwDzIigbqgpgtGHkTwym W9Eqs/OXKD2JP2eOtXnU72Pv5doCuAC8ZoYnhsY9MMtBtuUh/e9uVNIEpoNvbwnn74cS zmSLj3C5/rssQIGRSQSCSxyvNKi79RW6StE4De6PgEqoZ1YYWrd0P76OiIq9e3BYNwK1 AbUg== X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 148.163.158.5 is neither permitted nor denied by best guess record for domain of paulmck@linux.vnet.ibm.com) smtp.mailfrom=paulmck@linux.vnet.ibm.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com X-Gm-Message-State: APt69E2P24/9NdLIKJ/riRA7tEVz12/wNPVay4upDKlY5+J+mk16H2i3 0EwaArorbfD8Xc6TvTvpX3A09TGynxHZy9OewMLxvLyXa18uS+aGQo78WaPk2WaqYAgAMJvfNTg brlhaBXNPX2jTGmN6VmmN6YH2IXYqYv4RViKLsBF4SABOnUaBgL7kbKtPzU/jOCU= X-Received: by 2002:adf:b112:: with SMTP id l18-v6mr5484428wra.101.1530633536051; Tue, 03 Jul 2018 08:58:56 -0700 (PDT) X-Google-Smtp-Source: AAOMgpfhvFISMyZuftlUYUk10VJ6cqZUi7ficBz6wqJf0TcGwrneMoFxo6YmzCbZHQ0zbdNyeavu X-Received: by 2002:adf:b112:: with SMTP id l18-v6mr5484366wra.101.1530633534738; Tue, 03 Jul 2018 08:58:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1530633534; cv=none; d=google.com; s=arc-20160816; b=KdTZ33Qtxjoh7zX/KohyEZ5Okzy5XlraL4NJmR8VLHoSefsCE3pxui+QSPWW0MiLlD heKLHhvBfrlc9jxoKIJ/2wvlV8yD2RP2QjjaN/6j3FDVUBK0wFi5Ndbs6m5WBRFhmX9W JVDfXXzTQwazW6BmQT5a2JZl6eTVb9PFgSVbLqjnkA3mqiqJr8rdVCXVTmvxA1C8MYFS m26M1WZhtxrgKCTJ44bJo3/qpoE9+B7zLSQqhoMHMKaUH3cIT+OMDAqxoPx1lU6EQClh 8G3S5cYO74UA98kcdI0iLhLGYZj7oIlcEWHH0cLjLFHd713PidcHQ7P7rcv5IQhXLOdF ecXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:user-agent:in-reply-to:content-disposition:mime-version :references:reply-to:subject:cc:to:from:date :arc-authentication-results; bh=P0rvnx8tQpdGXJrHq+E5FJgc2sxhjb5hwTd7J7eGB/4=; b=rqVnlX9s62SJgZinzwEMd/HLiKS/rXSM/kx60YepknsGDCMIsNQL7ofZGS9xENThul yJxA+42IJq3BpXgAzFtECKpWxbB8Gs8XBzHdU70nvk3G80YttKPTW8ciDJbqL2DqahoZ UumF+DQbR7XZcFpZP9AlPv0pvIbMNKwbGUGQTCgvCd1a5AsnnWIdiU2cwTBwCp3KIF54 whtlLHbkOf3yHw/fogKYAG442gHHpucl2adNixzh/dV/CYhA6Rpf02VpaMrI9yvHTt6t W/tfsZ+u2NH1PhovCh5OWwEsJYQ/TbxUM5GJ+RAfjLva57kWVbIfmzUH/4JPYgVLhakS xyyg== ARC-Authentication-Results: i=1; mx.google.com; spf=neutral (google.com: 148.163.158.5 is neither permitted nor denied by best guess record for domain of paulmck@linux.vnet.ibm.com) smtp.mailfrom=paulmck@linux.vnet.ibm.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com. [148.163.158.5]) by mx.google.com with ESMTPS id b73-v6si1229952wme.127.2018.07.03.08.58.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 03 Jul 2018 08:58:54 -0700 (PDT) Received-SPF: neutral (google.com: 148.163.158.5 is neither permitted nor denied by best guess record for domain of paulmck@linux.vnet.ibm.com) client-ip=148.163.158.5; Authentication-Results: mx.google.com; spf=neutral (google.com: 148.163.158.5 is neither permitted nor denied by best guess record for domain of paulmck@linux.vnet.ibm.com) smtp.mailfrom=paulmck@linux.vnet.ibm.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w63FtRFL102848 for ; Tue, 3 Jul 2018 11:58:53 -0400 Received: from e12.ny.us.ibm.com (e12.ny.us.ibm.com [129.33.205.202]) by mx0b-001b2d01.pphosted.com with ESMTP id 2k09dh8nft-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 03 Jul 2018 11:58:53 -0400 Received: from localhost by e12.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 3 Jul 2018 11:58:52 -0400 Received: from b01cxnp23034.gho.pok.ibm.com (9.57.198.29) by e12.ny.us.ibm.com (146.89.104.199) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Tue, 3 Jul 2018 11:58:49 -0400 Received: from b01ledav003.gho.pok.ibm.com (b01ledav003.gho.pok.ibm.com [9.57.199.108]) by b01cxnp23034.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w63FwmeB8978782 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Tue, 3 Jul 2018 15:58:48 GMT Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0C11AB206A; Tue, 3 Jul 2018 11:58:32 -0400 (EDT) Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D85C5B2072; Tue, 3 Jul 2018 11:58:31 -0400 (EDT) Received: from paulmck-ThinkPad-W541 (unknown [9.70.82.159]) by b01ledav003.gho.pok.ibm.com (Postfix) with ESMTP; Tue, 3 Jul 2018 11:58:31 -0400 (EDT) Received: by paulmck-ThinkPad-W541 (Postfix, from userid 1000) id CC37B16C649E; Tue, 3 Jul 2018 09:01:01 -0700 (PDT) Date: Tue, 3 Jul 2018 09:01:01 -0700 From: "Paul E. McKenney" To: Michal Hocko Cc: Tetsuo Handa , David Rientjes , linux-mm@kvack.org, Andrew Morton , linux-kernel@vger.kernel.org Subject: Re: [PATCH] mm, oom: Bring OOM notifier callbacks to outside of OOM killer. Reply-To: paulmck@linux.vnet.ibm.com References: <20180627072207.GB32348@dhcp22.suse.cz> <20180627143125.GW3593@linux.vnet.ibm.com> <20180628113942.GD32348@dhcp22.suse.cz> <20180628213105.GP3593@linux.vnet.ibm.com> <20180629090419.GD13860@dhcp22.suse.cz> <20180629125218.GX3593@linux.vnet.ibm.com> <20180629132638.GD5963@dhcp22.suse.cz> <20180630170522.GZ3593@linux.vnet.ibm.com> <20180702213714.GA7604@linux.vnet.ibm.com> <20180703072413.GD16767@dhcp22.suse.cz> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20180703072413.GD16767@dhcp22.suse.cz> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 x-cbid: 18070315-0060-0000-0000-00000286C1CE X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00009301; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000266; SDB=6.01056005; UDB=6.00541670; IPR=6.00833928; MB=3.00021978; MTD=3.00000008; XFM=3.00000015; UTC=2018-07-03 15:58:50 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18070315-0061-0000-0000-000045AA8987 Message-Id: <20180703160101.GC3593@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-07-03_06:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1806210000 definitions=main-1807030181 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP On Tue, Jul 03, 2018 at 09:24:13AM +0200, Michal Hocko wrote: > On Mon 02-07-18 14:37:14, Paul E. McKenney wrote: > [...] > > commit d2b8d16b97ac2859919713b2d98b8a3ad22943a2 > > Author: Paul E. McKenney > > Date: Mon Jul 2 14:30:37 2018 -0700 > > > > rcu: Remove OOM code > > > > There is reason to believe that RCU's OOM code isn't really helping > > that much, given that the best it can hope to do is accelerate invoking > > callbacks by a few seconds, and even then only if some CPUs have no > > non-lazy callbacks, a condition that has been observed to be rare. > > This commit therefore removes RCU's OOM code. If this causes problems, > > it can easily be reinserted. > > > > Reported-by: Michal Hocko > > Reported-by: Tetsuo Handa > > Signed-off-by: Paul E. McKenney > > I would also note that waiting in the notifier might be a problem on its > own because we are holding the oom_lock and the system cannot trigger > the OOM killer while we are holding it and waiting for oom_callback_wq > event. I am not familiar with the code to tell whether this can deadlock > but from a quick glance I _suspect_ that we might depend on __rcu_reclaim > and basically an arbitrary callback so no good. > > Acked-by: Michal Hocko > > Thanks! Like this? Thanx, Paul ------------------------------------------------------------------------ commit d3cf56333af5e65054f533f6f271a7bd36b868f4 Author: Paul E. McKenney Date: Mon Jul 2 14:30:37 2018 -0700 rcu: Remove OOM code There is reason to believe that RCU's OOM code isn't really helping that much, given that the best it can hope to do is accelerate invoking callbacks by a few seconds, and even then only if some CPUs have no non-lazy callbacks, a condition that has been observed to be rare. In addition, it can block on the oom_callback_wq wait queue, which could delay more urgent and productive OOM actions. This commit therefore removes RCU's OOM code. If this causes problems, this commit can easily be reverted. Reported-by: Michal Hocko Reported-by: Tetsuo Handa Signed-off-by: Paul E. McKenney Acked-by: Michal Hocko diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index 3f3796b10c71..3d7ce73e7309 100644 --- a/kernel/rcu/tree_plugin.h +++ b/kernel/rcu/tree_plugin.h @@ -1722,87 +1722,6 @@ static void rcu_idle_count_callbacks_posted(void) __this_cpu_add(rcu_dynticks.nonlazy_posted, 1); } -/* - * Data for flushing lazy RCU callbacks at OOM time. - */ -static atomic_t oom_callback_count; -static DECLARE_WAIT_QUEUE_HEAD(oom_callback_wq); - -/* - * RCU OOM callback -- decrement the outstanding count and deliver the - * wake-up if we are the last one. - */ -static void rcu_oom_callback(struct rcu_head *rhp) -{ - if (atomic_dec_and_test(&oom_callback_count)) - wake_up(&oom_callback_wq); -} - -/* - * Post an rcu_oom_notify callback on the current CPU if it has at - * least one lazy callback. This will unnecessarily post callbacks - * to CPUs that already have a non-lazy callback at the end of their - * callback list, but this is an infrequent operation, so accept some - * extra overhead to keep things simple. - */ -static void rcu_oom_notify_cpu(void *unused) -{ - struct rcu_state *rsp; - struct rcu_data *rdp; - - for_each_rcu_flavor(rsp) { - rdp = raw_cpu_ptr(rsp->rda); - if (rcu_segcblist_n_lazy_cbs(&rdp->cblist)) { - atomic_inc(&oom_callback_count); - rsp->call(&rdp->oom_head, rcu_oom_callback); - } - } -} - -/* - * If low on memory, ensure that each CPU has a non-lazy callback. - * This will wake up CPUs that have only lazy callbacks, in turn - * ensuring that they free up the corresponding memory in a timely manner. - * Because an uncertain amount of memory will be freed in some uncertain - * timeframe, we do not claim to have freed anything. - */ -static int rcu_oom_notify(struct notifier_block *self, - unsigned long notused, void *nfreed) -{ - int cpu; - - /* Wait for callbacks from earlier instance to complete. */ - wait_event(oom_callback_wq, atomic_read(&oom_callback_count) == 0); - smp_mb(); /* Ensure callback reuse happens after callback invocation. */ - - /* - * Prevent premature wakeup: ensure that all increments happen - * before there is a chance of the counter reaching zero. - */ - atomic_set(&oom_callback_count, 1); - - for_each_online_cpu(cpu) { - smp_call_function_single(cpu, rcu_oom_notify_cpu, NULL, 1); - cond_resched_tasks_rcu_qs(); - } - - /* Unconditionally decrement: no need to wake ourselves up. */ - atomic_dec(&oom_callback_count); - - return NOTIFY_OK; -} - -static struct notifier_block rcu_oom_nb = { - .notifier_call = rcu_oom_notify -}; - -static int __init rcu_register_oom_notifier(void) -{ - register_oom_notifier(&rcu_oom_nb); - return 0; -} -early_initcall(rcu_register_oom_notifier); - #endif /* #else #if !defined(CONFIG_RCU_FAST_NO_HZ) */ #ifdef CONFIG_RCU_FAST_NO_HZ