From patchwork Mon Jul 24 06:22:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhongkun He X-Patchwork-Id: 13323469 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E50EC001B0 for ; Mon, 24 Jul 2023 06:22:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 04E7A6B0074; Mon, 24 Jul 2023 02:22:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F40E46B0078; Mon, 24 Jul 2023 02:22:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE4086B007B; Mon, 24 Jul 2023 02:22:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C8ED86B0074 for ; Mon, 24 Jul 2023 02:22:30 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 9682F1608DE for ; Mon, 24 Jul 2023 06:22:30 +0000 (UTC) X-FDA: 81045511260.12.7A01B6D Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf17.hostedemail.com (Postfix) with ESMTP id D1FCD40009 for ; Mon, 24 Jul 2023 06:22:28 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="g8XFmi/X"; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf17.hostedemail.com: domain of hezhongkun.hzk@bytedance.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=hezhongkun.hzk@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690179748; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=vQ0+FHFKuAIc1z56nGRfE+KN8lTLE329HHeSqqWKPM0=; b=zBydM8NSxCp4S+80EAl0/W/R9PGHA45lv0Y8yjgs9WWMpTED014nHsRhMb8e28YWSzqM5t +xfjYJyvkrXI3eTIQKJnOoVgGgStrXRQdY/85/DujbyopZKfzAWU9W8a7gRzHaaPF98fXR FHEuL6EKz39ZphPxl0EVt7vYQSdGtDo= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="g8XFmi/X"; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf17.hostedemail.com: domain of hezhongkun.hzk@bytedance.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=hezhongkun.hzk@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690179748; a=rsa-sha256; cv=none; b=BXyzCy2eHT/gHfMt9XzYjL35VuxnQKfyBaEU9izFIbsaEWO8r8WdRYxqoHPCWTsOxwXj1O NcAw2m9nrDQfC0Clsd3aHDYMIuLdva4bsS6p8+92h5/c0tBjidT0rM75KENwsR7tZ9oKOq Dm7V3tPVx/QTj8dTXB5SULvqGhRx7P0= Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-1bba2318546so5261185ad.1 for ; Sun, 23 Jul 2023 23:22:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1690179747; x=1690784547; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=vQ0+FHFKuAIc1z56nGRfE+KN8lTLE329HHeSqqWKPM0=; b=g8XFmi/Xu7Q/IPaOlyMdCeZDLC2qDIUrES87VDacv5L4RIvIl2r0gW/KhtEcTINWF6 85uL6rrA8LQjGriLE4rjl7XVCGU8czhLCTXmr4/dBUFZmwaukf40MyoppTsVSXGPnMED yuLs+xCiBxSTLrm8Wp+vlyQRp022Dqdc5p/LMHi5gv8ZOTY7L8QxjhIb1B56LA/FH6ml b4vSr/xbX35wV5C08h/v8anQ2+KiEZecKkvJfOPAvGWtQd/3dA75o4GonVIh8w3yJFiC QQonyK5SW6Ifk7GmdSnw1GrPioh0+m5Mf///5dFHYd8C1LlCS2KyorB9zGkd8VXm/aPc Ge3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690179747; x=1690784547; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=vQ0+FHFKuAIc1z56nGRfE+KN8lTLE329HHeSqqWKPM0=; b=iNpgfTH2Nm5i4grcd/Ic2Y/mU8RXQsmAerrnHCptxmOQ7b+m3dpsZukI4lJdcRJS1F SSOBBNREAkvzCP2oCRXIbvfSW5p2Jvnk4DYI3oXukJAVzn8WytonXOABQcp0Jtm82wmj yodZdmd5O5DcxSEj5CZltOPvKhAOCNUUSjLcBxsg3XtM6O3tA6+LDM8Y0Ay3TjpHy8hv HxA5ev2bfuiD3UKJf9cNYTwlxu7NdHIZi8MTfBnnvy90LhG4q2ipMtGFvnsxEIhs89t8 SubGYcnmEky1JVjrIQlp+k6OWHSfwpXkTrMgD1tW9fuGp96A910oSIRI2A28yRUhaPdF C8ag== X-Gm-Message-State: ABy/qLa4X/zcxm6CvXQNTKeU4Gj6TeGanzkapI7jEIjGnaoTOUMDGW+Y 7aSpSi768O/SEasJ2aL9N3knYw== X-Google-Smtp-Source: APBJJlEolK+aUii1/yBS9Re9OkeH47TWX8ggnozCaLtCJpFpuy5aQEWLtvsSzNsXA7eaGPyusMtcLQ== X-Received: by 2002:a17:902:d2cd:b0:1b8:a389:43ef with SMTP id n13-20020a170902d2cd00b001b8a38943efmr12250391plc.24.1690179747454; Sun, 23 Jul 2023 23:22:27 -0700 (PDT) Received: from Tower.bytedance.net ([203.208.167.146]) by smtp.gmail.com with ESMTPSA id ix20-20020a170902f81400b001b80760fd04sm1443236plb.112.2023.07.23.23.22.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jul 2023 23:22:26 -0700 (PDT) From: Zhongkun He To: minchan@kernel.org, senozhatsky@chromium.org, mhocko@suse.com Cc: david@redhat.com, yosryahmed@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Zhongkun He Subject: [RFC PATCH v2 2/2] zram: charge the compressed RAM to the page's memcgroup Date: Mon, 24 Jul 2023 14:22:19 +0800 Message-Id: <20230724062219.2244240-1-hezhongkun.hzk@bytedance.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: D1FCD40009 X-Stat-Signature: s8utzikq51x4m17in3jgwwhpk9ikwgkk X-Rspam-User: X-HE-Tag: 1690179748-434170 X-HE-Meta: U2FsdGVkX18N58Bk2Uzps1jKUjc46Ok+tZM71EGveBeQyV4XZcviTTWtgeagwLgYSFks4fcpAlcLYY2vt26BHnMBlyErKloCRdg6C2rxvjiXk1qeK8Iw2IRVbqf4sa3WAKNSAWnbZIoRSBBCi4sV8kCwm5zXCZ4rIJ/xJ7k6K2sPFoe5aAr0yFVnYz1xxT/R51VhIbTnzRbNawrAhdhMT+rzakytSSYhrCmak6OBaXWEAo7zeZyNECOZEQ14ivVUJArqn13jX7/1Vh2s7GzrDUgUfBIQv0gQwkpi93YYUmnqhHoann2g3NgjD9USIUopc7607OPgZktn7G5QI5mjL3ivpUx2M/qzg3Hhh2UFpD0CGv/s4XNH/2r2asFsr6MoBmhDQtvg4fkjviMXa7eLEjHXXMyLMu/ytoHd3Wom9a1/pTjwiiJXSaOb9+/Qfj6HblPJbC4b1UfVQJ2vnov3M+Azou9DAqIEaKhEdZF3s2iWB9T13j5mba0vnGv1dexd850gkMc1yKEAczwE/GjpVH5/c2+MdHOXdvAdTbxRuSVw+U9K/w3VcolYu6M/bxMmsL4AFNllk/Htg5eYcm4wD0VZLIEF9qqjEqoIDwJdSsATiM+70Ao7hHvKMTc4hXXqAX9Q2uE67QtMxgHWH/AHzdL729cp8+yHZ+fQq6wn96z/iw/x+Fs2wNlOpO3xYTMVrQagzToV7NO8lHbkm33mN3mYfxptjygXJX1N719DUBzwFCiY75IbZkN7M+WXc04mQE9ANubABDpakvGN7gOMgoVKg44XuTRrvUAXwi6Ph/UeCvBtYW/rLoQUrrQM06cqz+u6Q+iMyzeURAipmD/WDEoD9xH8upWYtgVnMS9mfmEXxx6gZ4cmU+Fm1Bat1oF0mwv9rT2F7nr7MMmGEIOBIRL8bXdEmOUvBAhTrfq75A/gTwAxfVwOXjNG9n8yud/Jpx1aGA8fsHgXGkA3BIo yiwgIjEj yT2nyTv7Bwcm59irQMC1hvbE4iSQA4FQMrya5TPOB6cW/e6d6KjSkvDPa9ALKYvW/HUftvzbFeDRSjQ2qKGVP82m9sn/GTkt6tA1NYqLZOTer3O+UOckId4MRWM2zFJC8YDLbFuqxYE3ClPVGAQAjnH1v2+8rj8S63qCKIk10X342gatfVY9uT79ELEbLYisf++bcBwIWe4Li6ih6sGWu/WvbEOkjcLsdWRj1w6kqVa4Rpq/13NiNtfGpYLfzAMGKp/0DplYehNFw6SBH+pj5ap1sfYQfSAmoGnrKn6SE80Bu5zmUKautzyRRDV4/lmhh+S87k3hQ78g3QS9z4jhoeY75ofIj8QwgHbuGGmQm4yBgdWoKwkMy9vL6AqDcnJvO6+hiPYkDMxSDgCfdt4irxc6bXxY2KMZzwItG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The compressed RAM is currently charged to kernel, not to any memory cgroup. This patch can charge the pages regardless of direct or indirect zram usage. Direct zram usage by process within a cgroup will fail to charge if there is no memory. Indirect zram usage by process within a cgroup via swap in PF_MEMALLOC context, wqwill charge successfully. This allows some limit overrun, but not enough to matter in practice.Charge compressed page once, mean a page will be freed.the size of compressed page is less than or equal to the page to be freed. The numbers of excess depend on the compression ratio only. The maximum amount will not exceed 400KB, and will be smaller than the hard limit finally, So not an unbounded way. Signed-off-by: Zhongkun He --- drivers/block/zram/zram_drv.c | 46 +++++++++++++++++++++++++++++++++++ drivers/block/zram/zram_drv.h | 1 + 2 files changed, 47 insertions(+) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 5676e6dd5b16..1e685a430c95 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "zram_drv.h" @@ -135,6 +136,18 @@ static void zram_set_obj_size(struct zram *zram, zram->table[index].flags = (flags << ZRAM_FLAG_SHIFT) | size; } +static inline void zram_set_obj_cgroup(struct zram *zram, u32 index, + struct obj_cgroup *objcg) +{ + zram->table[index].objcg = objcg; +} + +static inline struct obj_cgroup *zram_get_obj_cgroup(struct zram *zram, + u32 index) +{ + return zram->table[index].objcg; +} + static inline bool zram_allocated(struct zram *zram, u32 index) { return zram_get_obj_size(zram, index) || @@ -1256,6 +1269,7 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize) static void zram_free_page(struct zram *zram, size_t index) { unsigned long handle; + struct obj_cgroup *objcg; #ifdef CONFIG_ZRAM_MEMORY_TRACKING zram->table[index].ac_time = 0; @@ -1289,6 +1303,13 @@ static void zram_free_page(struct zram *zram, size_t index) goto out; } + objcg = zram_get_obj_cgroup(zram, index); + if (objcg) { + obj_cgroup_uncharge_zram(objcg, zram_get_obj_size(zram, index)); + obj_cgroup_put(objcg); + zram_set_obj_cgroup(zram, index, NULL); + } + handle = zram_get_handle(zram, index); if (!handle) return; @@ -1419,6 +1440,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) struct zcomp_strm *zstrm; unsigned long element = 0; enum zram_pageflags flags = 0; + struct obj_cgroup *objcg; mem = kmap_atomic(page); if (page_same_filled(mem, &element)) { @@ -1494,6 +1516,14 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return -ENOMEM; } + objcg = get_obj_cgroup_from_page(page); + if (objcg && obj_cgroup_charge_zram(objcg, comp_len)) { + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zs_free(zram->mem_pool, handle); + obj_cgroup_put(objcg); + return -ENOMEM; + } + dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); src = zstrm->buffer; @@ -1526,6 +1556,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) } else { zram_set_handle(zram, index, handle); zram_set_obj_size(zram, index, comp_len); + zram_set_obj_cgroup(zram, index, objcg); } zram_slot_unlock(zram, index); @@ -1575,6 +1606,7 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, u32 threshold, u32 prio, u32 prio_max) { struct zcomp_strm *zstrm = NULL; + struct obj_cgroup *objcg; unsigned long handle_old; unsigned long handle_new; unsigned int comp_len_old; @@ -1669,6 +1701,17 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, if (threshold && comp_len_new >= threshold) return 0; + objcg = zram_get_obj_cgroup(zram, index); + if (objcg) { + obj_cgroup_get(objcg); + if (obj_cgroup_charge_zram(objcg, GFP_KERNEL, comp_len_new)) { + zcomp_stream_put(zram->comps[prio]); + obj_cgroup_put(objcg); + return -ENOMEM; + } + } + /* * No direct reclaim (slow path) for handle allocation and no * re-compression attempt (unlike in zram_write_bvec()) since @@ -1683,6 +1726,8 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, __GFP_MOVABLE); if (IS_ERR_VALUE(handle_new)) { zcomp_stream_put(zram->comps[prio]); + obj_cgroup_uncharge_zram(objcg, comp_len_new); + obj_cgroup_put(objcg); return PTR_ERR((void *)handle_new); } @@ -1696,6 +1741,7 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, zram_set_handle(zram, index, handle_new); zram_set_obj_size(zram, index, comp_len_new); zram_set_priority(zram, index, prio); + zram_set_obj_cgroup(zram, index, objcg); atomic64_add(comp_len_new, &zram->stats.compr_data_size); atomic64_inc(&zram->stats.pages_stored); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index ca7a15bd4845..959d721d5474 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -72,6 +72,7 @@ struct zram_table_entry { #ifdef CONFIG_ZRAM_MEMORY_TRACKING ktime_t ac_time; #endif + struct obj_cgroup *objcg; }; struct zram_stats {