From patchwork Wed May 26 22:25:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Gushchin X-Patchwork-Id: 12282839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 640D0C4708A for ; Wed, 26 May 2021 22:26:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AEB89613CA for ; Wed, 26 May 2021 22:26:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AEB89613CA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=fb.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4D4856B006E; Wed, 26 May 2021 18:26:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 434F06B0070; Wed, 26 May 2021 18:26:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 262C96B0071; Wed, 26 May 2021 18:26:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0075.hostedemail.com [216.40.44.75]) by kanga.kvack.org (Postfix) with ESMTP id DDAC96B006E for ; Wed, 26 May 2021 18:26:06 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 6FCB8B7A4 for ; Wed, 26 May 2021 22:26:06 +0000 (UTC) X-FDA: 78184816332.24.825940E Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf20.hostedemail.com (Postfix) with ESMTP id 9586E2D1 for ; Wed, 26 May 2021 22:25:56 +0000 (UTC) Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 14QME13o016337 for ; Wed, 26 May 2021 15:26:05 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding : content-type; s=facebook; bh=m2Usb8pBnaJ5JSG5ZmqsO2PX6YX/Fcc3JtrbDbrn2J0=; b=KdozIGjxtdV16VDlebccLJllHB/zniP9jY3DhAld5mU9GLWGdhf3kZaoO5kOGPJRY8kh jNbnoEJrtEhRXaPGxoMaan0dLQiIpnFSPcI6Lfh/2PqW+J4sKIdSBlGt9OfUtcsKDWoA aSqcZSNH0A4aJQB/Vdp+AG/T94NDmp7SzKY= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 38ss3qjq0x-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Wed, 26 May 2021 15:26:04 -0700 Received: from intmgw003.48.prn1.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Wed, 26 May 2021 15:26:02 -0700 Received: by devvm3388.prn0.facebook.com (Postfix, from userid 111017) id A2E5A7B6ABB3; Wed, 26 May 2021 15:25:58 -0700 (PDT) From: Roman Gushchin To: Jan Kara , Tejun Heo CC: , , , Alexander Viro , Dennis Zhou , Dave Chinner , , Roman Gushchin Subject: [PATCH v5 0/2] cgroup, blkcg: prevent dirty inodes to pin dying memory cgroups Date: Wed, 26 May 2021 15:25:55 -0700 Message-ID: <20210526222557.3118114-1-guro@fb.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: WHWUwaovBCD8sKWp6HJcBafN00SfEGxY X-Proofpoint-ORIG-GUID: WHWUwaovBCD8sKWp6HJcBafN00SfEGxY X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.761 definitions=2021-05-26_12:2021-05-26,2021-05-26 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 spamscore=0 mlxscore=0 lowpriorityscore=0 phishscore=0 adultscore=0 suspectscore=0 bulkscore=0 mlxlogscore=520 impostorscore=0 clxscore=1015 priorityscore=1501 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2105260151 X-FB-Internal: deliver Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=KdozIGjx; dmarc=pass (policy=reject) header.from=fb.com; spf=pass (imf20.hostedemail.com: domain of "prvs=4780f431ea=guro@fb.com" designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=4780f431ea=guro@fb.com" X-Stat-Signature: 4w6oqctmzcj1uzksehs6ekikze8fj8iw X-Rspamd-Queue-Id: 9586E2D1 X-Rspamd-Server: rspam02 X-HE-Tag: 1622067956-414455 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When an inode is getting dirty for the first time it's associated with a wb structure (see __inode_attach_wb()). It can later be switched to another wb (if e.g. some other cgroup is writing a lot of data to the same inode), but otherwise stays attached to the original wb until being reclaimed. The problem is that the wb structure holds a reference to the original memory and blkcg cgroups. So if an inode has been dirty once and later is actively used in read-only mode, it has a good chance to pin down the original memory and blkcg cgroups forewer. This is often the case with services bringing data for other services, e.g. updating some rpm packages. In the real life it becomes a problem due to a large size of the memcg structure, which can easily be 1000x larger than an inode. Also a really large number of dying cgroups can raise different scalability issues, e.g. making the memory reclaim costly and less effective. To solve the problem inodes should be eventually detached from the corresponding writeback structure. It's inefficient to do it after every writeback completion. Instead it can be done whenever the original memory cgroup is offlined and writeback structure is getting killed. Scanning over a (potentially long) list of inodes and detach them from the writeback structure can take quite some time. To avoid scanning all inodes, attached inodes are kept on a new list (b_attached). To make it less noticeable to a user, the scanning is performed from a work context. Big thanks to Jan Kara and Dennis Zhou for their ideas and contribution to the previous iterations of this patch. v5: - switch inodes to bdi->wb instead of zeroing inode->i_wb - split the single patch into two - only cgwbs maintain lists of attached inodes - added cond_resched() - fixed !CONFIG_CGROUP_WRITEBACK handling - extended list of prohibited inodes flag - other small fixes Roman Gushchin (2): writeback, cgroup: keep list of inodes attached to bdi_writeback writeback, cgroup: release dying cgwbs by switching attached inodes fs/fs-writeback.c | 101 +++++++++++++++++++++++++------ include/linux/backing-dev-defs.h | 2 + include/linux/backing-dev.h | 7 +++ include/linux/writeback.h | 2 + mm/backing-dev.c | 63 ++++++++++++++++++- 5 files changed, 156 insertions(+), 19 deletions(-)