From patchwork Fri Jul 7 04:46:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhongkun He X-Patchwork-Id: 13304389 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B59D0EB64D9 for ; Fri, 7 Jul 2023 04:52:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 523A88D0005; Fri, 7 Jul 2023 00:52:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4AD578D0001; Fri, 7 Jul 2023 00:52:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 374F78D0005; Fri, 7 Jul 2023 00:52:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 241938D0001 for ; Fri, 7 Jul 2023 00:52:04 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id ECDC31C8FF1 for ; Fri, 7 Jul 2023 04:52:03 +0000 (UTC) X-FDA: 80983593726.11.6B3F6D0 Received: from mail-yb1-f171.google.com (mail-yb1-f171.google.com [209.85.219.171]) by imf15.hostedemail.com (Postfix) with ESMTP id 32BF0A0010 for ; Fri, 7 Jul 2023 04:52:02 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=CHZ1ledA; spf=pass (imf15.hostedemail.com: domain of hezhongkun.hzk@bytedance.com designates 209.85.219.171 as permitted sender) smtp.mailfrom=hezhongkun.hzk@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688705522; a=rsa-sha256; cv=none; b=LkZ1a+6yM22r2lyHfBbBIgK5+odFDLQsGjRF6wfI2cOGrCGzhbMPEA9U3MJ+ZTTdePwmc0 guXVWfUVBWh1UG/dXLYNVTsv7+SYv3IQYuhMS8NwiFrY6LHIWeOqcJnOAMJiOi8cONKASy izLbepRDRdObWu+97gILIUNPXA0cmVc= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=CHZ1ledA; spf=pass (imf15.hostedemail.com: domain of hezhongkun.hzk@bytedance.com designates 209.85.219.171 as permitted sender) smtp.mailfrom=hezhongkun.hzk@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688705522; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=FOcqYStjaEQ6e41x7ko6U0Q7unE1KNRPYn3wRQMvZDI=; b=p6xNUgRlmatzz68DVp28R1OD8ADctmYKHxHIE1Y/kmy3rSSLAIhskgqRV3KJH3I7JZbWB+ a5SMzgZnlgyqqj5THliKc6NQyWroN+bAh3+DtqzwjnqF2txnhbysPn6pcnwih5ME6C9+Or BjUrt6wtnWS+4SJ8/MmOwtf5eO0HMAE= Received: by mail-yb1-f171.google.com with SMTP id 3f1490d57ef6-bfe6ea01ff5so1592068276.3 for ; Thu, 06 Jul 2023 21:52:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1688705521; x=1691297521; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=FOcqYStjaEQ6e41x7ko6U0Q7unE1KNRPYn3wRQMvZDI=; b=CHZ1ledA9NwOG/pSJSHv/D+w66wHvrAztOFQOqH479zhDguVShOXbLJovgowm66V7+ klOY9hA4elIXXZbhXyFhcMkhZg6lXL/+gUGkHeX77QXCsBCP3B31wiSwgoakmMY1hKYr FYdZEUpDmS+1i6VPtY+eS3Peq1HVVjUWy9lRvRCq69FAyzYwkTLwnbI3PnJPHewlz367 4oIie2F1IfD3tYhwmtKttNxXHO6fEenTBtE5hJSXY3Cwzaoh+WG3CvNxuA+/r1XO/hsY 7NEkwiRU+VU48LhNB59xbu0htXAoLhAwlRXEsagOVlq2XL6kSe7ebB49GrBkas0T6JFi EwUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688705521; x=1691297521; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=FOcqYStjaEQ6e41x7ko6U0Q7unE1KNRPYn3wRQMvZDI=; b=guxC4UMCqgAgurpStPTmLQQALjQrrIGAaQxyxEcLU8HMQWnbCGwblPV8m+BdPpjIKN t3WOouQMFpUeti7WoP138Ox9p02XLyO72NZbYkl5jH9UvfOGnO4LrQCLeKvTWVCTB68s WauzPTCFHMJ+P659/W97iJJXs5mkwRwnKCNqZ8xXwXkI3Fq7a24IhtBN3vtpK56Ix1rO h2VNXqwj2g0A8jzqtWKBi5HymRj85kKWB+nlIYp/DqDVkkP0SDjVXTUwyQ0omdz6hHjE cWwb0NNfJ1jyvoq+ZVN69He7LzaW5mXE8WcHfuHpxqdNaAT/BE7PG2QJ1wa5D/751MAS GXrQ== X-Gm-Message-State: ABy/qLamdNvdM7WM2z+r94VYXoxcoPjc0qHvyyCstZUDZC4WQLgJfCPM Mzb9gtDYxhCm2VT95fWkamv5iK3yO9GmmVvvcKE= X-Google-Smtp-Source: APBJJlHb8cX+0KMLn+D06cRMztmkT8KQkZHyFAw/d3ujFt0BRPy1YxSFDgU25z1DlZoH715my6Kvhg== X-Received: by 2002:a17:902:7c0a:b0:1b4:65c3:ca68 with SMTP id x10-20020a1709027c0a00b001b465c3ca68mr405138pll.66.1688705217824; Thu, 06 Jul 2023 21:46:57 -0700 (PDT) Received: from Tower.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id bc8-20020a170902930800b001ac95be5081sm2193423plb.307.2023.07.06.21.46.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 06 Jul 2023 21:46:57 -0700 (PDT) From: Zhongkun He To: minchan@kernel.org, senozhatsky@chromium.org, mhocko@suse.com Cc: david@redhat.com, yosryahmed@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Zhongkun He Subject: [RFC PATCH 1/2] memcg: Add support for zram object charge Date: Fri, 7 Jul 2023 12:46:49 +0800 Message-Id: <20230707044649.1169155-1-hezhongkun.hzk@bytedance.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 32BF0A0010 X-Stat-Signature: k6yqy86sb1ksyfaef5m1kmg9qnkrtuge X-Rspam-User: X-HE-Tag: 1688705522-457095 X-HE-Meta: U2FsdGVkX1/oCmnDxAca97UTHXOBcJ2Ya8G3FOwn1DwRFPYwKCZEHC5Pjjl3u6ISMF04ukuS8YB70qvVS7Uin4fJcV5hZtkAL8EMgac7kGCVFoJXUJYjqc3TQBl10r1jKaile1PipQr1cuQSzfisIfUYanMigeL/uw/+Lg+sstPO229Q6dyOnR2CE7CWufLj1/1vTUkCqynEYRTzItRswEdIzM9dz/1wlFgnwuCuVGk5lbgr+YeR0JNaO5lbwkDw1pld8s7/KErdlcjnmB9h3LjvFbBIECoRn9NFDM478Y1a+fwctnM+9q3NXbEgqoXYzknej3iPPYcFKwj96uinSOBmI35gAKKNYokAiEQPGRwa8dgZhNyFU+zVNH+/RuE7oaDV2ub9FhRqM7kO4riblOrS6A4B5ejvLzAtR9eVAiHDLlC0VMt+H8p6Nvkd3a3MFX6b1lcyvLknwmAHE8+XvEKy6Q8wLfgjDsGFY6JhP65vp2lxzyFYTA/F4xiH9tOx3+ijwyo9Ligld7uN7eUWRXnZ0/NZ1UXPgKV2U4rTfQLicXK5MjMio3Zor8zkudDXhRtrPQHASH6Aod8hb/sc2q/eJkYZyHOOox4ks6/PMm/izi6naNxgl2p2lKFkqbj+I9AFBgvbjQry7FFMPc/YAS4B7qLG+Q3ClA7mcEMi89dvD/oe6XiGtYsPm6662ljh0nlhjKDeJNP9xExz8od2VQIjz8f9zd54yrjm0KWy1Fye1w4fU1spGClHgAYrWrl1uL0TZJynLx20x+70UsWEQGr84NQa4eYU55W+2fzaPRoBrcgi6xsF2PI+N1HP9dudjPOfOPohw1qVDu8glhrX91Je5KvV/+uVSoiGZlv9rNgra34Nsp2U2cFaLixAvDN5thrbEwqdfmPJ3RzbzIQ5XfoyudJ/2OJgsiT9pyaNETxTh8QSqRbInOb2K1FS0bObzgtZXYOZjHK7KU7FtxE 2FjWvnnP I/bKPKP7EV4FN+L6rIQ7yukRXgpUIiMwZMOdXah28APFruNbD5BYNQHSE17tKbSb1eFgSxpvKgmHmKzSUtOsVQT4wkAgC1ImXCcTsoo8HjROaMheaRwsUdvWlvj/URgRvBjpWXyN7PD5oydPzPjASfMJu7deHQ3ausOum+ynf+3ra99uh9xYduNMV8vAv2pSW8rqqHLkNxZR3pZQNYVNlvlsm5ovhGyCRlPS6fDYj0vPL9oaaX/kZtD1e35poEhR3yCWbxsBxSV6Rx2LCNxAtdu98YhSqFWskGq+wZ0cTMtzCVxAQLSwaVAxlnkUmswx8IxjO0qdTjuc8c45EpyyzkrIC7poh+9hhbFxRIvVQyBDzZUeBGiL8BErKv9QIFUfng5KA7vZgXJZjv2vts5lC5gsjI0rNoldhaze1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The compressed RAM is currently charged to kernel, not to any memory cgroup, which is not satisfy our usage scenario. if the memory of a task is limited by memcgroup, it will swap out the memory to zram swap device when the memory is insufficient. In that case, the memory limit will have no effect. So, it should makes sense to charge the compressed RAM to the page's memory cgroup. As we know, zram can be used in two ways, direct and indirect, this patchset can charge memory in both cases. Direct zram usage by process within a cgroup will fail to charge if there is no memory. Indirect zram usage by process within a cgroup via swap in PF_MEMALLOC context, will charge successfully. Signed-off-by: Zhongkun He --- include/linux/memcontrol.h | 10 ++++++++++ mm/memcontrol.c | 23 +++++++++++++++++++++++ 2 files changed, 33 insertions(+) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 5818af8eca5a..716cc08a6527 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1819,6 +1819,8 @@ static inline void count_objcg_event(struct obj_cgroup *objcg, rcu_read_unlock(); } +int obj_cgroup_charge_zram(struct obj_cgroup *objcg, size_t size); +void obj_cgroup_uncharge_zram(struct obj_cgroup *objcg, size_t size); #else static inline bool mem_cgroup_kmem_disabled(void) { @@ -1880,6 +1882,14 @@ static inline void count_objcg_event(struct obj_cgroup *objcg, { } +int obj_cgroup_charge_zram(struct obj_cgroup *objcg, size_t size) +{ + return 0; +} + +void obj_cgroup_uncharge_zram(struct obj_cgroup *objcg, size_t size) +{ +} #endif /* CONFIG_MEMCG_KMEM */ #if defined(CONFIG_MEMCG_KMEM) && defined(CONFIG_ZSWAP) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index e8ca4bdcb03c..e1d2c084c132 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3059,6 +3059,7 @@ struct obj_cgroup *get_obj_cgroup_from_page(struct page *page) } return objcg; } +EXPORT_SYMBOL(get_obj_cgroup_from_page); static void memcg_account_kmem(struct mem_cgroup *memcg, int nr_pages) { @@ -3409,6 +3410,28 @@ void obj_cgroup_uncharge(struct obj_cgroup *objcg, size_t size) refill_obj_stock(objcg, size, true); } +int obj_cgroup_charge_zram(struct obj_cgroup *objcg, size_t size) +{ + if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) + return 0; + + /* + * Indirect zram usage in PF_MEMALLOC, charging must succeed. + * Direct zram usage, charging may failed. + */ + return obj_cgroup_charge(objcg, GFP_KERNEL, size); +} +EXPORT_SYMBOL(obj_cgroup_charge_zram); + +void obj_cgroup_uncharge_zram(struct obj_cgroup *objcg, size_t size) +{ + if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) + return; + + obj_cgroup_uncharge(objcg, size); +} +EXPORT_SYMBOL(obj_cgroup_uncharge_zram); + #endif /* CONFIG_MEMCG_KMEM */ /* From patchwork Fri Jul 7 04:47:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhongkun He X-Patchwork-Id: 13304388 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DDCCEB64D9 for ; Fri, 7 Jul 2023 04:47:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 091E68D0005; Fri, 7 Jul 2023 00:47:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 041778D0001; Fri, 7 Jul 2023 00:47:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E4C998D0005; Fri, 7 Jul 2023 00:47:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D68D88D0001 for ; Fri, 7 Jul 2023 00:47:18 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A8ADF1402FA for ; Fri, 7 Jul 2023 04:47:18 +0000 (UTC) X-FDA: 80983581756.06.46C7F69 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf20.hostedemail.com (Postfix) with ESMTP id DCC971C000B for ; Fri, 7 Jul 2023 04:47:16 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=i9R4JehL; spf=pass (imf20.hostedemail.com: domain of hezhongkun.hzk@bytedance.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=hezhongkun.hzk@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688705236; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=OEV7S2qQa9wjcpqCPZF1/IRLU/XLUimMDUCsPYYW7kI=; b=2OFos/Zfz1VA6gBZ819dHcYdakoImZzjY4Iks/O5vHuEyiZQ1tR14MwwSOsoRC1dK0F4+T Dh4PYKyGcrNe8wa2NsWgcgAnVeBAGnORewiaz6PXud3Wa+I8TkK0W0yUAgbvD8NVXgvO3h Fy2cKrhNEOTyPjJEr04iFIPcXAOsGpQ= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=i9R4JehL; spf=pass (imf20.hostedemail.com: domain of hezhongkun.hzk@bytedance.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=hezhongkun.hzk@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688705236; a=rsa-sha256; cv=none; b=M+qNxFbAh9EsGSJsjE+fc6WCPZcWDM/2qlTcJLnDEVrJAG2sQdomCBlBMoqCZuCDWiGvkX dvECNApSpAiqyIcgefSCSpiOxYEyWig5C+7ZazDhRjr8Mds3K71hhKchyOmCyZJQzJIsk8 77Uj0iRpI2snDcJCOQaAHBCIzyrZ448= Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1b8b2cb7384so7967115ad.2 for ; Thu, 06 Jul 2023 21:47:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1688705236; x=1691297236; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=OEV7S2qQa9wjcpqCPZF1/IRLU/XLUimMDUCsPYYW7kI=; b=i9R4JehL62yeUI0J/4ssJQxvz0puzqqchTK/dmDx0jOYqynb15e1FVB9HaIbdAnBap I+9q1hHV+sLd0FkRPQrxa357li+iU7oRQ3ZKljZNWV3srIwdrp0BycDD7H+epgpwzfRQ YVWBEAsXX8PF0UhPux74/+tZUH3QnrxpBO4WdqdABk7EeoVbQxDpUwN7GkVD8PfbmKbi jrqdFSt13I8trb9p3hzK1fwPq4uIr0nK3BpF2FMwkbqRd5wXqzHhSxMUm3PxZ/vdQo/j 5S8xAvzry2rY9pLS05xsu74C9bbszUTiarWjDUT6mckjYUbOG3yUlgSfPTRsN+uftCxC r1qQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688705236; x=1691297236; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OEV7S2qQa9wjcpqCPZF1/IRLU/XLUimMDUCsPYYW7kI=; b=AF6NwSNIhw8jCZ1P3CwaRR7KVloJBSSVcZfklub44NWXKJZ/L10BZZtRrwryMwz95t zK6qm0WoWOWAGnh9N8uzD4UsV7qFd4Ei95xGzotgax4Wl+ara+nuIKSRQfG9eTBQTIMP /vbz5dPKvgrsZIoSRPyBv8gIHdQ4a7QZOqV8WlU2f8v+oQL+EJQ9tM4p3/v60AiZl8jn JUvvnpxVfAjzMFCw+Uvw8pTH0UDKronZkFIor9BjaRT01pUpKvIXDdu3gnmtFx3wzsG/ f/ELbQVbO5oTBM1RG/d6ES1PuYmLZFSoOHYSARXBFcPOB0dcavrclzYb2NWNyYoSHKFL grWA== X-Gm-Message-State: ABy/qLYO6C7HH9cCO71AXFJ8tiybgJDd3xF5VpVniIcnSy9yo5qSbOOO FHr3N6F4oHbU7W7XEK/UGnET7w== X-Google-Smtp-Source: APBJJlEtaEiw8usJCj2cjDIpbdnmQoBl+moxhFrnsASFMEpmDxsNajjhnvllwpDNFcF/3sV84z1rXQ== X-Received: by 2002:a17:902:7405:b0:1b8:b827:aa8e with SMTP id g5-20020a170902740500b001b8b827aa8emr3111867pll.11.1688705235754; Thu, 06 Jul 2023 21:47:15 -0700 (PDT) Received: from Tower.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id i11-20020a170902eb4b00b001b86deadfddsm2181858pli.264.2023.07.06.21.47.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 06 Jul 2023 21:47:15 -0700 (PDT) From: Zhongkun He To: minchan@kernel.org, senozhatsky@chromium.org, mhocko@suse.com Cc: david@redhat.com, yosryahmed@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Zhongkun He Subject: [RFC PATCH 2/2] zram: charge the compressed RAM to the page's memcgroup Date: Fri, 7 Jul 2023 12:47:07 +0800 Message-Id: <20230707044707.1169285-1-hezhongkun.hzk@bytedance.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Rspamd-Queue-Id: DCC971C000B X-Rspam-User: X-Stat-Signature: m3ft9ap67a6pzuuj883ertyg8mhzxmj7 X-Rspamd-Server: rspam01 X-HE-Tag: 1688705236-506105 X-HE-Meta: U2FsdGVkX19y91una1VWCKBfrCB6zxYwfdJSlAkWpU5enujrzx4L92KWYOpDNUyc6K2D+cQmLotGFQV581143DfWxMcwKtINaX7DPKXXVm50wUc45U4y73clopjbkRYZvx3dsRuA93fcidNYKwadG2PxYeYeqFGSmjEVGLC00c7RLCCQwy0qpVbFIPDlC0qzSZifSW9yAM02K1+kGb42vgwmG0ZgsLp8znx1/0mXbXD2wcEyYtNN/M8L12Z5qsTpZzGiG23ey5cYOEtEU8T1E9DUI53kSLvQqpQvGuHJugb7lffESMLjEOyMn76jPQkpAOh4vNjpl2oRUYYTpcbG752KJCn/iF4AcOZyIizjg8AckdHoSvv2SorsEi2SoDSELQFNN4eibo8SBls27+BFbw3+ZSWr1qPwJwl892VyBted0ppEopI8jyUNwWICNOQzntTGtfczeUD/6uRFXiai1R8/p3Lb1ovWQ/Jnew7v0XMcSfX2eXY+XQPuJBDd7zLFWFdeQMr5rauGBtVx9d270Lk6X5ldixbM1e/m17AuHfs8c9IDf02BW/cqjs8TOfhJ/gkv0IRVSGbw4V3u61mt0qHIXAsDVf528+QHUZ0x06TDWxXVr3SwZMbvtK0pBxPn/VNccoFuIuJgqkdPzDyW7YTxO5y2yih22gG3Q/v0cB2IZLWXy1Md+JFlzPLRj9GJ4zBcPWx7WNTeKVQBgs8hT5L1AzqWwUsjYRe8eKT2ozfTVX84ux+qpkTqhA1NzxdtLZzMEcpnAizz/eDkt4a5aDKebmxfFO7I5MWhRF5Z2+pEXF0Ccr3RAi0yeMPfXKsSLSdk20I/irFfEZE+xS3smGckUkppdhKMTYfFxj+5CLfDubxockEDd5SJ3XyKlUTcJIBL7vIsGWcD/PBk+3tp0JBCKsCyPURLwtBY89Ow8hIecCV6w0EOlxsqTOoTFvDDqNOHUEFvh2j+tQjd2ck IAMTfy2C Czm/4EpGz/bwZFTK5jtRqSo1BEQvtj2/w3mvbewYxkLe1ietk4yWF1WAXnrgQQ2xQydIwiwXIzPqTl1a6FH33rNGrU9WqrcH44n2U7j0YBhkqDd2mXJsXE+IHLvgHUIkEcXKIIxCxIc06fzookZ3gC0PwSdV3k9PoIC7hHmGD0+dSn4XgVhZC9StVLKLRC4XmMD9+TZ45ctAuohM6Wk0/W/ZckC6hf4Fo2SmMh+TuP9ripswtnoTljBFOakYRMOAqEQ54nOtVnbQYaubgT7gtzjwOqKiES/Z+Q3oBAi02kow0amUhyDq8jeseD768s8b3nJBBx1/L4gXmP9S7+Bf83m2sV0eiVVhMJ5M3eA/Jf50tyWgMn8ha/lfM1Vn82wXNijHcAajLpIj8JdaBEyuQqCDjKwp2IE7J03BH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The compressed RAM is currently charged to kernel, not to any memory cgroup. This patch can charge the pages regardless of direct or indirect zram usage. Direct zram usage by process within a cgroup will fail to charge if there is no memory. Indirect zram usage by process within a cgroup via swap in PF_MEMALLOC context, will charge successfully. Signed-off-by: Zhongkun He --- drivers/block/zram/zram_drv.c | 43 +++++++++++++++++++++++++++++++++++ drivers/block/zram/zram_drv.h | 1 + 2 files changed, 44 insertions(+) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 5676e6dd5b16..770c7d58afec 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "zram_drv.h" @@ -135,6 +136,18 @@ static void zram_set_obj_size(struct zram *zram, zram->table[index].flags = (flags << ZRAM_FLAG_SHIFT) | size; } +static inline void zram_set_obj_cgroup(struct zram *zram, u32 index, + struct obj_cgroup *objcg) +{ + zram->table[index].objcg = objcg; +} + +static inline struct obj_cgroup *zram_get_obj_cgroup(struct zram *zram, + u32 index) +{ + return zram->table[index].objcg; +} + static inline bool zram_allocated(struct zram *zram, u32 index) { return zram_get_obj_size(zram, index) || @@ -1256,6 +1269,7 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize) static void zram_free_page(struct zram *zram, size_t index) { unsigned long handle; + struct obj_cgroup *objcg; #ifdef CONFIG_ZRAM_MEMORY_TRACKING zram->table[index].ac_time = 0; @@ -1289,6 +1303,13 @@ static void zram_free_page(struct zram *zram, size_t index) goto out; } + objcg = zram_get_obj_cgroup(zram, index); + if (objcg) { + obj_cgroup_uncharge_zram(objcg, zram_get_obj_size(zram, index)); + obj_cgroup_put(objcg); + zram_set_obj_cgroup(zram, index, NULL); + } + handle = zram_get_handle(zram, index); if (!handle) return; @@ -1419,6 +1440,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) struct zcomp_strm *zstrm; unsigned long element = 0; enum zram_pageflags flags = 0; + struct obj_cgroup *objcg = NULL; mem = kmap_atomic(page); if (page_same_filled(mem, &element)) { @@ -1494,6 +1516,14 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return -ENOMEM; } + objcg = get_obj_cgroup_from_page(page); + if (objcg && obj_cgroup_charge_zram(objcg, comp_len)) { + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zs_free(zram->mem_pool, handle); + obj_cgroup_put(objcg); + return -ENOMEM; + } + dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); src = zstrm->buffer; @@ -1526,6 +1556,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) } else { zram_set_handle(zram, index, handle); zram_set_obj_size(zram, index, comp_len); + zram_set_obj_cgroup(zram, index, objcg); } zram_slot_unlock(zram, index); @@ -1581,6 +1612,8 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, unsigned int comp_len_new; unsigned int class_index_old; unsigned int class_index_new; + struct obj_cgroup *objcg = NULL; + unsigned int noreclaim_flag; u32 num_recomps = 0; void *src, *dst; int ret; @@ -1692,11 +1725,21 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, zs_unmap_object(zram->mem_pool, handle_new); + /* + * Recompress will reclaim some memory, so we set the reclaim + * flag in order to charge comp_len_new successfully. + */ + noreclaim_flag = memalloc_noreclaim_save(); + objcg = zram_get_obj_cgroup(zram, index); + obj_cgroup_get(objcg); zram_free_page(zram, index); + obj_cgroup_charge_zram(objcg, GFP_KERNEL, comp_len_new); + zram_set_obj_cgroup(zram, index, objcg); zram_set_handle(zram, index, handle_new); zram_set_obj_size(zram, index, comp_len_new); zram_set_priority(zram, index, prio); + memalloc_noreclaim_restore(noreclaim_flag); atomic64_add(comp_len_new, &zram->stats.compr_data_size); atomic64_inc(&zram->stats.pages_stored); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index ca7a15bd4845..959d721d5474 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -72,6 +72,7 @@ struct zram_table_entry { #ifdef CONFIG_ZRAM_MEMORY_TRACKING ktime_t ac_time; #endif + struct obj_cgroup *objcg; }; struct zram_stats {