From patchwork Thu Oct 25 08:24:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Hocko X-Patchwork-Id: 10655589 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 88F9213B5 for ; Thu, 25 Oct 2018 08:24:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 776482B236 for ; Thu, 25 Oct 2018 08:24:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6B6E12B252; Thu, 25 Oct 2018 08:24:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0BF4B2B236 for ; Thu, 25 Oct 2018 08:24:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 030E66B026B; Thu, 25 Oct 2018 04:24:14 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F214C6B026C; Thu, 25 Oct 2018 04:24:13 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0FF96B026D; Thu, 25 Oct 2018 04:24:13 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id 8963E6B026B for ; Thu, 25 Oct 2018 04:24:13 -0400 (EDT) Received: by mail-ed1-f69.google.com with SMTP id k17-v6so4185095edr.18 for ; Thu, 25 Oct 2018 01:24:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:mime-version:content-transfer-encoding; bh=2r80qbszvU6jBT+csI7xjz9erJcqYEzjVONob5JYS1M=; b=enJmOQiAIhA2POOkWDRDNk4YC8Tw3hpnn6myzn6CbiPpqeH6Dpw/uK1/y0cZZn4Hl2 8jPXzLaSVzzGiOPmQE2JeHDIJdspwpApodsRonyp68VINGz+vu6ylj1aKLF0nikJU7JQ DXu0OTFQjo0hbF3VGAQbcZUYnT4OEINvy1TXdvWt24oEg986+42UoExFwT6W0YfUdcr7 33e07Y5cVp5USC/vdImj3Yj4komebQ4Kff7n91Obx2CNtqC/zlOUL2apKIjbdhZ+wQGD Zdp+anwE0bi9sxINjg9bt92NhVD0dgiDEh+zUumDUirt+9BnZytGZE13bJ7MKLCqMpNN rKZQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Gm-Message-State: AGRZ1gIuVWJP7dFMh958oU8+YMr4ywTdNoDTEQ1Q1+7LcXCHhSTXoXi5 4QwwejFXPIfITIfEG+wpWfGpghwVjt+ZqiWpUAzDKdZP2HIyxyqSDZngc4FrmOAaCMhaxEtKI9d 7xkhetSjSEURKbKseJi9vIwb72dNiYvEc6J9exLoAWhwGZLAUWwk0FHJevpYgopovCVeuTdDCUZ M7mEeM5rQsUltgzUNlLORNFkpTl2gcLcv0EQcQmod/0ckpzgyQmwUjhD+Sl/luPm4O0g34JqbDn ggblhO34Dwd6aTGxKZEbBuc5XgYaPVzAXEinwMKoX162+LLgwXckt0JEpejuRvSu4m1WgTgEeUl GWKMu0t6M8SWOUyoeCNmXfXixzDkExDLVVbKAxS696TeWF0ORZ4Mh7czYYbhBU9ueVJZYQixbw= = X-Received: by 2002:a17:906:c446:: with SMTP id ck6-v6mr603660ejb.215.1540455852932; Thu, 25 Oct 2018 01:24:12 -0700 (PDT) X-Received: by 2002:a17:906:c446:: with SMTP id ck6-v6mr603625ejb.215.1540455851987; Thu, 25 Oct 2018 01:24:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1540455851; cv=none; d=google.com; s=arc-20160816; b=nW0t/ohrhFa4NGVOUmgTFwzTSeHxwP9fxxnk3ayqaYdNnxxJaBoy9llorT4GdM60zk 6pRn4FQ4Lqsr5R/+V9A86OIjx/Ub99YKTErahL9RmiSfUXnT2OLKHcmJQgrhNrioNqsG G9QICbIxkUyo7+RXr5Af02hKrZnGMbj9tK+5YUILKRwkLN5gIv/rjsHSlYoDW3NAdg6O jO5DnJRLquy1OwnpQPDizcj+9QOJ2B9X+8dbwKwucC7zATY76lPdodZYBN8N5xe+N3ey D1E9CQsfevoGdD4s9bEC098vfigBKVErO2zlpvzhMAUgqMikNLlO4vM4h9lRjHFfisl9 oWkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=2r80qbszvU6jBT+csI7xjz9erJcqYEzjVONob5JYS1M=; b=ZOIANzh3RZdUCSyC4xqaNKgD+K1Cj7zf7Zlzf7Q88koCnYWHn9ilsENHvbtqd84inN j+O7lQ0eeCHslnriq1UuQ9tzZVcn6zHPy38WnGsml/9BQoyDWlMHwH1pxUUxHWMJ0KeA tTs/GwDw7rDYeZF67rbCsh9h6lMZyRTv6bGU0/Bu34KZzLGfocGpdbVkPUyYv4EXYmWq ZfBJlyPVsnDHYb7cNQvW8KOmQVqUWZvdUKqnIlTPhery48poHMGgZNak/o61Asfl6/Ix fcDjf6n2BC9Pv9eOj11W/z+6d9Z7rTIxHQh5VzDA34LRIowhJWB5SyJ5uFojorCFIyvJ bs3A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id b25-v6sor2265800ejo.11.2018.10.25.01.24.11 for (Google Transport Security); Thu, 25 Oct 2018 01:24:11 -0700 (PDT) Received-SPF: pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of mstsxfx@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org X-Google-Smtp-Source: AJdET5fGcYKyZv8ex2yrOwj4TS8MCBfyJLYB62vWBp5fl5xOFbqzY28fUJCZStZa0CZ+iC8c1ECYdQ== X-Received: by 2002:a17:906:66cd:: with SMTP id k13-v6mr628496ejp.152.1540455851270; Thu, 25 Oct 2018 01:24:11 -0700 (PDT) Received: from tiehlicka.suse.cz (prg-ext-pat.suse.com. [213.151.95.130]) by smtp.gmail.com with ESMTPSA id m24-v6sm2628277edd.31.2018.10.25.01.24.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 25 Oct 2018 01:24:10 -0700 (PDT) From: Michal Hocko To: Cc: Tetsuo Handa , Roman Gushchin , David Rientjes , Andrew Morton , LKML Subject: [RFC PATCH v2 0/3] oom: rework oom_reaper vs. exit_mmap handoff Date: Thu, 25 Oct 2018 10:24:00 +0200 Message-Id: <20181025082403.3806-1-mhocko@kernel.org> X-Mailer: git-send-email 2.19.1 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The previous version of this RFC has been posted here [1]. I have fixed few issues spotted during the review and by 0day bot. I have also reworked patch 2 to be ratio rather than an absolute number based. With this series applied the locking protocol between the oom_reaper and the exit path is as follows. All parts which cannot race should use the exclusive lock on the exit path. Once the exit path has passed the moment when no blocking locks are taken then it clears mm->mmap under the exclusive lock. oom_reaper checks for this and sets MMF_OOM_SKIP only if the exit path is not guaranteed to finish the job. This is patch 3 so see the changelog for all the details. I would really appreciate if David could give this a try and see how this behaves in workloads where the oom_reaper falls flat now. I have been playing with sparsely allocated memory with a high pte/real memory ratio and large mlocked processes and it worked reasonably well. There is still some room for tuning here of course. We can change the number of retries for the oom_reaper as well as the threshold when the keep retrying. Michal Hocko (3): mm, oom: rework mmap_exit vs. oom_reaper synchronization mm, oom: keep retrying the oom_reap operation as long as there is substantial memory left mm, oom: hand over MMF_OOM_SKIP to exit path if it is guranteed to finish Diffstat: include/linux/oom.h | 2 -- mm/internal.h | 3 +++ mm/memory.c | 28 ++++++++++++++-------- mm/mmap.c | 69 +++++++++++++++++++++++++++++++++-------------------- mm/oom_kill.c | 45 ++++++++++++++++++++++++---------- 5 files changed, 97 insertions(+), 50 deletions(-) [1] http://lkml.kernel.org/r/20180910125513.311-1-mhocko@kernel.org Nacked-by: Tetsuo Handa