From patchwork Thu Dec 19 14:21:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vitaly Wool X-Patchwork-Id: 11303721 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 68EE76C1 for ; Thu, 19 Dec 2019 14:21:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 27491222C2 for ; Thu, 19 Dec 2019 14:21:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="WiB/mqAZ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 27491222C2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3FBB38E0172; Thu, 19 Dec 2019 09:21:55 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 382E48E00F5; Thu, 19 Dec 2019 09:21:55 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24B1C8E0172; Thu, 19 Dec 2019 09:21:55 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0176.hostedemail.com [216.40.44.176]) by kanga.kvack.org (Postfix) with ESMTP id 0B34E8E00F5 for ; Thu, 19 Dec 2019 09:21:55 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id B63AA181AC9CC for ; Thu, 19 Dec 2019 14:21:54 +0000 (UTC) X-FDA: 76282104948.09.title11_2db4a2a9c1d43 X-Spam-Summary: 2,0,0,1d84a5dfc8cfc974,d41d8cd98f00b204,vitalywool@gmail.com,::akpm@linux-foundation.org:ddstreet@ieee.org:minchan@kernel.org:sergey.senozhatsky.work@gmail.com:linux-kernel@vger.kernel.org:vbabka@suse.cz:shakeelb@google.com:henrywolfeburns@gmail.com:tytso@thunk.org,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1277:1311:1313:1314:1345:1359:1437:1515:1516:1518:1535:1543:1593:1594:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3355:3369:3865:3866:3867:3868:3871:4117:4250:4321:4605:5007:6119:6261:6653:7875:7903:8603:9413:10004:10400:11026:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12683:12760:12895:12986:13161:13229:13439:13869:13972:14096:14097:14181:14394:14659:14687:14721:21080:21444:21451:21627:21666:21990:30005:30034:30054:30070,0,RBL:209.85.208.195:@gmail.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom _rules:0 X-HE-Tag: title11_2db4a2a9c1d43 X-Filterd-Recvd-Size: 6938 Received: from mail-lj1-f195.google.com (mail-lj1-f195.google.com [209.85.208.195]) by imf11.hostedemail.com (Postfix) with ESMTP for ; Thu, 19 Dec 2019 14:21:53 +0000 (UTC) Received: by mail-lj1-f195.google.com with SMTP id p8so6470920ljg.0 for ; Thu, 19 Dec 2019 06:21:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gofwvjfHv3OpfgHnS5zABByZ2BwnD9/VLkGxBh/tVfU=; b=WiB/mqAZ5AgK4mWB/04pDrsu5J3ScbjcrXxRkJwcLKm91NhUe8IvvUTONor6F0A3gH 19dDsLf3/hz8GHot/TCP/m0hsK93Oot9SLDUsghv+t7Dgwu6++UWgSFHiVVPUUVq35S6 EgTddRAERBaqN9C1MwvwsAugoFKdC7Z5KPePDxnXBYP6J9WnTFOddR58Cl2KbsOhyc97 /V6L2So+t3zvjo5NzniUssdOWQ0WJ6Y6pwxKk5HvpyOBfBRNCoMlD2uuw7ciXjXmzEeT uILplQ0AxEos6IXc7g7SRVII1SPd87EcX/migVeLZoNhX1RJhhcK3r+vZ7S2SN+lmE0n PHDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gofwvjfHv3OpfgHnS5zABByZ2BwnD9/VLkGxBh/tVfU=; b=aqWxjLhjACYBZoHPQ0ae9Oq3n6j2Xyyh4bakWUgHgf8mPvH3tHLMerpWs4tkAGQ/rc WjCmUeOIf6H4yy9Upi/rsrqizfog02JhvBs8grL+GCbCVYgP/+WplYU+pkWFGKiz/M3n UdQ7mBx75hL1qq2u4dwHXboc3Leg7T+eVme38utqLZFf2lu2iOT+lz0WUT0905gJMpJv 112NGBS7MyLpr23HOraLw3xFxAym2//LoclCAgNtPjBy/yQe9LqSBCYdqgQFUu6MAJrP 85wJNXVFvjiz4+kTK7/P1I4Rp/S7TBn+5iys3H2HGIzhZ4PbBCKU15kV+StvyAeGYQu3 Fx+g== X-Gm-Message-State: APjAAAWAUfuj3czeVHcNb4Va2R8bKdT7nVwQM/drcl2UUOXrmYQ7qFEB GPCBpFcQaq52EzKRR/0XhJcrx14Qt21cnQ== X-Google-Smtp-Source: APXvYqzf/gl2762BuLOYeikmN5C5VC3CQ+H9Afb70IeusMMpvUcm6hTtOZduBDnKaZt8uHwja2fH7A== X-Received: by 2002:a2e:86c4:: with SMTP id n4mr6068346ljj.97.1576765312270; Thu, 19 Dec 2019 06:21:52 -0800 (PST) Received: from assa ([109.252.14.238]) by smtp.gmail.com with ESMTPSA id 140sm2728532lfk.78.2019.12.19.06.21.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Dec 2019 06:21:51 -0800 (PST) Date: Thu, 19 Dec 2019 15:21:50 +0100 From: Vitaly Wool To: Linux-MM , Andrew Morton , Dan Streetman , Minchan Kim Cc: Sergey Senozhatsky , LKML , Vlastimil Babka , Shakeel Butt , Henry Burns , Theodore Ts'o Subject: [PATCHv2 1/3] zpool: add compaction api Message-Id: <20191219152150.1ed1900008a6ec867779f82e@gmail.com> In-Reply-To: <20191219151928.ad4ccf732b64b7f8a26116db@gmail.com> References: <20191219151928.ad4ccf732b64b7f8a26116db@gmail.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; x86_64-pc-linux-gnu) Mime-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch adds the following functions to the zpool API: - zpool_compact() - zpool_get_num_compacted() - zpool_huge_class_size() The first one triggers compaction for the underlying allocator, the second retrieves the number of pages migrated due to compaction for the whole time of this pool's existence and the third one returns the huge class size. This API extension is done to align zpool API with zsmalloc API. Signed-off-by: Vitaly Wool --- include/linux/zpool.h | 14 +++++++++++++- mm/zpool.c | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+), 1 deletion(-) diff --git a/include/linux/zpool.h b/include/linux/zpool.h index 51bf43076165..31f0c1360569 100644 --- a/include/linux/zpool.h +++ b/include/linux/zpool.h @@ -61,8 +61,13 @@ void *zpool_map_handle(struct zpool *pool, unsigned long handle, void zpool_unmap_handle(struct zpool *pool, unsigned long handle); +unsigned long zpool_compact(struct zpool *pool); + +unsigned long zpool_get_num_compacted(struct zpool *pool); + u64 zpool_get_total_size(struct zpool *pool); +size_t zpool_huge_class_size(struct zpool *zpool); /** * struct zpool_driver - driver implementation for zpool @@ -75,7 +80,10 @@ u64 zpool_get_total_size(struct zpool *pool); * @shrink: shrink the pool. * @map: map a handle. * @unmap: unmap a handle. - * @total_size: get total size of a pool. + * @compact: try to run compaction over a pool + * @get_num_compacted: get amount of compacted pages for a pool + * @total_size: get total size of a pool + * @huge_class_size: huge class threshold for pool pages. * * This is created by a zpool implementation and registered * with zpool. @@ -104,7 +112,11 @@ struct zpool_driver { enum zpool_mapmode mm); void (*unmap)(void *pool, unsigned long handle); + unsigned long (*compact)(void *pool); + unsigned long (*get_num_compacted)(void *pool); + u64 (*total_size)(void *pool); + size_t (*huge_class_size)(void *pool); }; void zpool_register_driver(struct zpool_driver *driver); diff --git a/mm/zpool.c b/mm/zpool.c index 863669212070..55e69213c2eb 100644 --- a/mm/zpool.c +++ b/mm/zpool.c @@ -362,6 +362,30 @@ void zpool_unmap_handle(struct zpool *zpool, unsigned long handle) zpool->driver->unmap(zpool->pool, handle); } + /** + * zpool_compact() - try to run compaction over zpool + * @pool The zpool to compact + * + * Returns: the number of migrated pages + */ +unsigned long zpool_compact(struct zpool *zpool) +{ + return zpool->driver->compact ? zpool->driver->compact(zpool->pool) : 0; +} + + +/** + * zpool_get_num_compacted() - get the number of migrated/compacted pages + * @pool The zpool to get compaction statistic for + * + * Returns: the total number of migrated pages for the pool + */ +unsigned long zpool_get_num_compacted(struct zpool *zpool) +{ + return zpool->driver->get_num_compacted ? + zpool->driver->get_num_compacted(zpool->pool) : 0; +} + /** * zpool_get_total_size() - The total size of the pool * @zpool: The zpool to check @@ -375,6 +399,18 @@ u64 zpool_get_total_size(struct zpool *zpool) return zpool->driver->total_size(zpool->pool); } +/** + * zpool_huge_class_size() - get size for the "huge" class + * @pool The zpool to check + * + * Returns: size of the huge class + */ +size_t zpool_huge_class_size(struct zpool *zpool) +{ + return zpool->driver->huge_class_size ? + zpool->driver->huge_class_size(zpool->pool) : 0; +} + /** * zpool_evictable() - Test if zpool is potentially evictable * @zpool: The zpool to test From patchwork Thu Dec 19 14:26:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vitaly Wool X-Patchwork-Id: 11303723 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 08392138C for ; Thu, 19 Dec 2019 14:26:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C872E24672 for ; Thu, 19 Dec 2019 14:26:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UG2Rag9z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C872E24672 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0E98C8E0174; Thu, 19 Dec 2019 09:26:12 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0C1888E00F5; Thu, 19 Dec 2019 09:26:12 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F3FEE8E0174; Thu, 19 Dec 2019 09:26:11 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0077.hostedemail.com [216.40.44.77]) by kanga.kvack.org (Postfix) with ESMTP id DE1228E00F5 for ; Thu, 19 Dec 2019 09:26:11 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 5D28D824999B for ; Thu, 19 Dec 2019 14:26:11 +0000 (UTC) X-FDA: 76282115742.06.cow63_5312da1685e51 X-Spam-Summary: 2,0,0,adcfffdf27f03a7f,d41d8cd98f00b204,vitalywool@gmail.com,::akpm@linux-foundation.org:ddstreet@ieee.org:minchan@kernel.org:sergey.senozhatsky.work@gmail.com:linux-kernel@vger.kernel.org:vbabka@suse.cz:shakeelb@google.com:henrywolfeburns@gmail.com:tytso@thunk.org,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1277:1311:1313:1314:1345:1359:1437:1515:1516:1518:1534:1541:1593:1594:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3352:3369:3867:4250:4321:5007:6261:6642:6653:9413:10004:10400:11026:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12760:12895:12986:13069:13311:13357:13439:14096:14097:14181:14394:14659:14687:14721:21080:21212:21444:21451:21627:21666:21990:30054,0,RBL:209.85.208.194:@gmail.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: cow63_5312da1685e51 X-Filterd-Recvd-Size: 4694 Received: from mail-lj1-f194.google.com (mail-lj1-f194.google.com [209.85.208.194]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Thu, 19 Dec 2019 14:26:10 +0000 (UTC) Received: by mail-lj1-f194.google.com with SMTP id u1so6435714ljk.7 for ; Thu, 19 Dec 2019 06:26:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eYZf2PAZk6wJlce5o8wLQfqz+k895+yefprdeiTmKqI=; b=UG2Rag9zPQHoaQ48g84ISujxsEL3CqlKjU167VCafzjKPq2OPtDP05/RKbX+aRaJBK APiv1iiZcjwGZTm+NVk2vrUzQ4YLNxb5rMznUGV+krOQdXLjZ+xUj0LUN8C/AUaSkfJ3 lKeKpUPFNJN/pzwh3MvnV8mzR6XmKkuniKawcKtmCnvylPrt2Yzs9S77wwzQ27mWXdb+ yl0i6MMzFQhg4sNQRHJoK1h9nMrKUVF+TPIOPDbJ1r7M8eShQkbnae0/lES/OdBvjIFS 08z2nH846N9EuofyWjdcAX1QLeqdAosv8iJQhcW6cE5fBFcLg1qw6mIPCZOV0WMK+wk/ P2Vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eYZf2PAZk6wJlce5o8wLQfqz+k895+yefprdeiTmKqI=; b=olviMXlHxnSQS+ykeBzVa8pEzCdXpqJOE7hoTZwYmQw3jMN4TVON3ukNyzpvFUfZ3Z HDI6YvHb+7nyk7PLGZskymetsKYbXbbeqr4o1GojuKMJt3TliNnyXs5CqkLXCO1k0aZX z7DnZOhXqyfaXoOUxIWBNVZHDgqlTJwL9rjHg/YF/vnSFUplJr71lwQvKAmkDL/umVW8 tkoYE2KpcLwSfTY0WX7j33/t0uSa50+4CEchPt0VwlNjP70E7Bx5gImLq9U8WiHDLQGb 4HiVOUT+K9zUyAbYkqDjRuc8yeSL703xEJ1ZVduk0ARRT+khG4e6QkAT+HJ5Ss+M8fUQ iM/A== X-Gm-Message-State: APjAAAXEuASJ7PZ2UqL1qe+qDAli/AqdZlS0LxsiQDd9Xi8ufzaZE/Gj Xxvj1d7wh1Id71KidTlteWdUqlrtuCvhng== X-Google-Smtp-Source: APXvYqwoDSKjOq8wwxmkSZnXq/HfNtdJzzk7qU8k6Ew6E7DXu96p7VMBGYDJB60KDr7YWng8euHmIA== X-Received: by 2002:a2e:800b:: with SMTP id j11mr5690943ljg.126.1576765569240; Thu, 19 Dec 2019 06:26:09 -0800 (PST) Received: from assa ([109.252.14.238]) by smtp.gmail.com with ESMTPSA id y1sm611437ljm.12.2019.12.19.06.26.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Dec 2019 06:26:08 -0800 (PST) Date: Thu, 19 Dec 2019 15:26:07 +0100 From: Vitaly Wool To: Linux-MM , Andrew Morton , Dan Streetman , Minchan Kim Cc: Sergey Senozhatsky , LKML , Vlastimil Babka , Shakeel Butt , Henry Burns , Theodore Ts'o Subject: [PATCHv2 2/3] zsmalloc: add compaction and huge class callbacks Message-Id: <20191219152607.03b458b910625d95f388f4d1@gmail.com> In-Reply-To: <20191219151928.ad4ccf732b64b7f8a26116db@gmail.com> References: <20191219151928.ad4ccf732b64b7f8a26116db@gmail.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; x86_64-pc-linux-gnu) Mime-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add compaction callbacks for zpool compaction API extension. Add huge_class_size callback too to be fully aligned. Signed-off-by: Vitaly Wool --- mm/zsmalloc.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 2b2b9aae8a3c..43f43272b998 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -437,11 +437,29 @@ static void zs_zpool_unmap(void *pool, unsigned long handle) zs_unmap_object(pool, handle); } +static unsigned long zs_zpool_compact(void *pool) +{ + return zs_compact(pool); +} + +static unsigned long zs_zpool_get_compacted(void *pool) +{ + struct zs_pool_stats stats; + + zs_pool_stats(pool, &stats); + return stats.pages_compacted; +} + static u64 zs_zpool_total_size(void *pool) { return zs_get_total_pages(pool) << PAGE_SHIFT; } +static size_t zs_zpool_huge_class_size(void *pool) +{ + return zs_huge_class_size(pool); +} + static struct zpool_driver zs_zpool_driver = { .type = "zsmalloc", .owner = THIS_MODULE, @@ -453,6 +471,9 @@ static struct zpool_driver zs_zpool_driver = { .map = zs_zpool_map, .unmap = zs_zpool_unmap, .total_size = zs_zpool_total_size, + .compact = zs_zpool_compact, + .get_num_compacted = zs_zpool_get_compacted, + .huge_class_size = zs_zpool_huge_class_size, }; MODULE_ALIAS("zpool-zsmalloc"); From patchwork Thu Dec 19 14:27:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vitaly Wool X-Patchwork-Id: 11303729 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C9AE613A4 for ; Thu, 19 Dec 2019 14:27:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7AEE724672 for ; Thu, 19 Dec 2019 14:27:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ohpdRq1K" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7AEE724672 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AD3478E0175; Thu, 19 Dec 2019 09:27:58 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A83B38E00F5; Thu, 19 Dec 2019 09:27:58 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 972EF8E0175; Thu, 19 Dec 2019 09:27:58 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0002.hostedemail.com [216.40.44.2]) by kanga.kvack.org (Postfix) with ESMTP id 802D78E00F5 for ; Thu, 19 Dec 2019 09:27:58 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 3AF422472 for ; Thu, 19 Dec 2019 14:27:58 +0000 (UTC) X-FDA: 76282120236.15.wren47_6293e46c0bd34 X-Spam-Summary: 50,0,0,632e13bdc5ccbece,d41d8cd98f00b204,vitalywool@gmail.com,::akpm@linux-foundation.org:ddstreet@ieee.org:minchan@kernel.org:sergey.senozhatsky.work@gmail.com:linux-kernel@vger.kernel.org:vbabka@suse.cz:shakeelb@google.com:henrywolfeburns@gmail.com:tytso@thunk.org,RULES_HIT:1:41:69:355:379:541:800:960:966:967:973:988:989:1260:1277:1311:1313:1314:1345:1359:1437:1515:1516:1518:1593:1594:1605:1730:1747:1777:1792:2196:2199:2393:2525:2561:2564:2637:2682:2685:2693:2859:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:5007:6119:6261:6653:7550:7653:7875:7903:8603:9025:9036:9040:9413:9592:10004:11026:11232:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12683:12740:12760:12895:12986:13137:13150:13230:13231:13439:13972:14394:14659:14687:21080:21444:21451:21627:21666:21788:21795:21990:30003:30029:30051:30054:30070,0,RBL:209.85 .167.68: X-HE-Tag: wren47_6293e46c0bd34 X-Filterd-Recvd-Size: 14603 Received: from mail-lf1-f68.google.com (mail-lf1-f68.google.com [209.85.167.68]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Thu, 19 Dec 2019 14:27:57 +0000 (UTC) Received: by mail-lf1-f68.google.com with SMTP id l18so4518002lfc.1 for ; Thu, 19 Dec 2019 06:27:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=i6l4Uk50/oOL+noZcGuNL6lOSRLFGNzbE/1V5CB31ww=; b=ohpdRq1KxvYwDxSyHLjzosO3l8bonluKTIhNc1I3nHbfz9VEgqSKXgWFpY/u7pYJ4y /NEFUEFd1oSpvynygS/xqLBM+Eu5z1diGzg1hM+zt6YYEkiOQpnssyn+OzTm7bmfF24L zDOS7EJOm+AymdkE+K9O5dwnTWvANcwz7Cg6WIN9BjRLJunmKPNV0TVpwwx1CivcNV05 48Gi2eNcNWT/sUMMUaPN1YXaCoNdCpaeIN0zN/TWc+p9M4vfI+blktIv8FV+c1m4r089 4ahG8Ac0O5khHOCz2qTiukHq0k8jYyOzKSe7QgRV9TufxVXRIsagoHlS+xIMXqV6ixpC JI7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=i6l4Uk50/oOL+noZcGuNL6lOSRLFGNzbE/1V5CB31ww=; b=ebAmvuSnBprf3P9xGoGSYAGLxETYzqR8Y5+n0yFN1sH1QZ4Ch+AEdV1vlmrhuFEbF2 GFDvLexILQyjmeDvit6w7xIZFR3w2lQT9CJV5awGynBkdrIMXFjaZc2AojZ6ve8ihj/w KGaH+MEoe03lxdz27UCYLvnT8mj2SbD+mp4OTOspOLb+XXENl+g6BH+Ghkx14rCFMuJz Fr0Qv+qU0sxKVgeoAbsyOzesENyN37/d/zyIWr2Bbb6Y9PTLsWZEkCR40XsYoEpRwkJ3 eUPRhnWwbABvvsJDACOCB4roqA4LZg302WDTLcrjS2xMNoeDfLuHbjnf0Py9o6yYq1Q6 8VHA== X-Gm-Message-State: APjAAAXkLO2aeCXcGYaCrTQ0HRGiRHRR2vIu2sikLXGtQAU6QDrBpXMR Xhr/VN7M5GhljgVNaTYm/qkLVOZ/fAgamw== X-Google-Smtp-Source: APXvYqzoT884kgnRIRU9V8Sq7UVdq/29Qc6jrSLd/4jjgnrwznFdrLGJHcj3JwXwfhOf5x38i8G42Q== X-Received: by 2002:a05:6512:48c:: with SMTP id v12mr5684318lfq.56.1576765675232; Thu, 19 Dec 2019 06:27:55 -0800 (PST) Received: from assa (109-252-14-238.nat.spd-mgts.ru. [109.252.14.238]) by smtp.gmail.com with ESMTPSA id p12sm2668497lfc.43.2019.12.19.06.27.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Dec 2019 06:27:54 -0800 (PST) Date: Thu, 19 Dec 2019 15:27:53 +0100 From: Vitaly Wool To: Linux-MM , Andrew Morton , Dan Streetman , Minchan Kim Cc: Sergey Senozhatsky , LKML , Vlastimil Babka , Shakeel Butt , Henry Burns , Theodore Ts'o Subject: [PATCHv2 3/3] zram: use common zpool interface Message-Id: <20191219152753.b30262676a6a3261490a49ab@gmail.com> In-Reply-To: <20191219151928.ad4ccf732b64b7f8a26116db@gmail.com> References: <20191219151928.ad4ccf732b64b7f8a26116db@gmail.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; x86_64-pc-linux-gnu) Mime-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch modifies ZRAM use a common "small allocator" API (zpool). This is done to address increasing demand to deploy ZRAM in systems where zsmalloc (which is currently the only available ZRAM allocator backend) is not a perfect match or is not applicable at all. An example of a system of the first type is an embedded system using ZRAM block device as a swap where quick application launch is critical for good user experience since z3fold is substantially faster on read than zsmalloc [1]. A system of the second type is, for instance, the one with hardware on-the-fly RAM compression/decompression where the saved RAM space could be used for ZRAM but would require a special allocator. These cases wre also discussed in detail at LPC'2019 [2]. zpool-registered backend can be selected using ZRAM sysfs interface on a per-device basis. 'zsmalloc' is taken by default. This patch is transparent with regard to the existing ZRAM functionality and does not change its default behavior. [1] https://lkml.org/lkml/2019/10/21/743 [2] https://linuxplumbersconf.org/event/4/contributions/551/ Signed-off-by: Vitaly Wool --- drivers/block/zram/Kconfig | 3 +- drivers/block/zram/zram_drv.c | 100 ++++++++++++++++++++++++---------- drivers/block/zram/zram_drv.h | 6 +- 3 files changed, 76 insertions(+), 33 deletions(-) diff --git a/drivers/block/zram/Kconfig b/drivers/block/zram/Kconfig index fe7a4b7d30cf..7248d5aa3468 100644 --- a/drivers/block/zram/Kconfig +++ b/drivers/block/zram/Kconfig @@ -1,8 +1,9 @@ # SPDX-License-Identifier: GPL-2.0 config ZRAM tristate "Compressed RAM block device support" - depends on BLOCK && SYSFS && ZSMALLOC && CRYPTO + depends on BLOCK && SYSFS && CRYPTO select CRYPTO_LZO + select ZPOOL help Creates virtual block devices called /dev/zramX (X = 0, 1, ...). Pages written to these disks are compressed and stored in memory diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 4285e75e52c3..189c326cbeee 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -43,6 +43,8 @@ static DEFINE_MUTEX(zram_index_mutex); static int zram_major; static const char *default_compressor = "lzo-rle"; +static const char *default_allocator = "zsmalloc"; + /* Module params (documentation at end) */ static unsigned int num_devices = 1; /* @@ -245,6 +247,14 @@ static ssize_t disksize_show(struct device *dev, return scnprintf(buf, PAGE_SIZE, "%llu\n", zram->disksize); } +static ssize_t allocator_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct zram *zram = dev_to_zram(dev); + + return scnprintf(buf, PAGE_SIZE, "%s\n", zram->allocator); +} + static ssize_t mem_limit_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t len) { @@ -277,7 +287,7 @@ static ssize_t mem_used_max_store(struct device *dev, down_read(&zram->init_lock); if (init_done(zram)) { atomic_long_set(&zram->stats.max_used_pages, - zs_get_total_pages(zram->mem_pool)); + zpool_get_total_size(zram->mem_pool) >> PAGE_SHIFT); } up_read(&zram->init_lock); @@ -1021,7 +1031,7 @@ static ssize_t compact_store(struct device *dev, return -EINVAL; } - zs_compact(zram->mem_pool); + zpool_compact(zram->mem_pool); up_read(&zram->init_lock); return len; @@ -1049,17 +1059,14 @@ static ssize_t mm_stat_show(struct device *dev, struct device_attribute *attr, char *buf) { struct zram *zram = dev_to_zram(dev); - struct zs_pool_stats pool_stats; u64 orig_size, mem_used = 0; - long max_used; + long max_used, num_compacted = 0; ssize_t ret; - memset(&pool_stats, 0x00, sizeof(struct zs_pool_stats)); - down_read(&zram->init_lock); if (init_done(zram)) { - mem_used = zs_get_total_pages(zram->mem_pool); - zs_pool_stats(zram->mem_pool, &pool_stats); + mem_used = zpool_get_total_size(zram->mem_pool); + num_compacted = zpool_get_num_compacted(zram->mem_pool); } orig_size = atomic64_read(&zram->stats.pages_stored); @@ -1069,11 +1076,11 @@ static ssize_t mm_stat_show(struct device *dev, "%8llu %8llu %8llu %8lu %8ld %8llu %8lu %8llu\n", orig_size << PAGE_SHIFT, (u64)atomic64_read(&zram->stats.compr_data_size), - mem_used << PAGE_SHIFT, + mem_used, zram->limit_pages << PAGE_SHIFT, max_used << PAGE_SHIFT, (u64)atomic64_read(&zram->stats.same_pages), - pool_stats.pages_compacted, + num_compacted, (u64)atomic64_read(&zram->stats.huge_pages)); up_read(&zram->init_lock); @@ -1134,7 +1141,7 @@ static void zram_meta_free(struct zram *zram, u64 disksize) for (index = 0; index < num_pages; index++) zram_free_page(zram, index); - zs_destroy_pool(zram->mem_pool); + zpool_destroy_pool(zram->mem_pool); vfree(zram->table); } @@ -1147,14 +1154,18 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize) if (!zram->table) return false; - zram->mem_pool = zs_create_pool(zram->disk->disk_name); + zram->mem_pool = zpool_create_pool(zram->allocator, + zram->disk->disk_name, + GFP_NOIO, NULL); if (!zram->mem_pool) { + pr_err("%s: not enough memory or wrong allocator '%s'\n", + __func__, zram->allocator); vfree(zram->table); return false; } if (!huge_class_size) - huge_class_size = zs_huge_class_size(zram->mem_pool); + huge_class_size = zpool_huge_class_size(zram->mem_pool); return true; } @@ -1198,7 +1209,7 @@ static void zram_free_page(struct zram *zram, size_t index) if (!handle) return; - zs_free(zram->mem_pool, handle); + zpool_free(zram->mem_pool, handle); atomic64_sub(zram_get_obj_size(zram, index), &zram->stats.compr_data_size); @@ -1247,7 +1258,7 @@ static int __zram_bvec_read(struct zram *zram, struct page *page, u32 index, size = zram_get_obj_size(zram, index); - src = zs_map_object(zram->mem_pool, handle, ZS_MM_RO); + src = zpool_map_handle(zram->mem_pool, handle, ZPOOL_MM_RO); if (size == PAGE_SIZE) { dst = kmap_atomic(page); memcpy(dst, src, PAGE_SIZE); @@ -1261,7 +1272,7 @@ static int __zram_bvec_read(struct zram *zram, struct page *page, u32 index, kunmap_atomic(dst); zcomp_stream_put(zram->comp); } - zs_unmap_object(zram->mem_pool, handle); + zpool_unmap_handle(zram->mem_pool, handle); zram_slot_unlock(zram, index); /* Should NEVER happen. Return bio error if it does. */ @@ -1336,7 +1347,7 @@ static int __zram_bvec_write(struct zram *zram, struct bio_vec *bvec, if (unlikely(ret)) { zcomp_stream_put(zram->comp); pr_err("Compression failed! err=%d\n", ret); - zs_free(zram->mem_pool, handle); + zpool_free(zram->mem_pool, handle); return ret; } @@ -1355,33 +1366,34 @@ static int __zram_bvec_write(struct zram *zram, struct bio_vec *bvec, * if we have a 'non-null' handle here then we are coming * from the slow path and handle has already been allocated. */ - if (!handle) - handle = zs_malloc(zram->mem_pool, comp_len, + if (handle == 0) + ret = zpool_malloc(zram->mem_pool, comp_len, __GFP_KSWAPD_RECLAIM | __GFP_NOWARN | __GFP_HIGHMEM | - __GFP_MOVABLE); - if (!handle) { + __GFP_MOVABLE, + &handle); + if (ret) { zcomp_stream_put(zram->comp); atomic64_inc(&zram->stats.writestall); - handle = zs_malloc(zram->mem_pool, comp_len, - GFP_NOIO | __GFP_HIGHMEM | - __GFP_MOVABLE); - if (handle) + ret = zpool_malloc(zram->mem_pool, comp_len, + GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE, + &handle); + if (ret == 0) goto compress_again; return -ENOMEM; } - alloced_pages = zs_get_total_pages(zram->mem_pool); + alloced_pages = zpool_get_total_size(zram->mem_pool) >> PAGE_SHIFT; update_used_max(zram, alloced_pages); if (zram->limit_pages && alloced_pages > zram->limit_pages) { zcomp_stream_put(zram->comp); - zs_free(zram->mem_pool, handle); + zpool_free(zram->mem_pool, handle); return -ENOMEM; } - dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); + dst = zpool_map_handle(zram->mem_pool, handle, ZPOOL_MM_WO); src = zstrm->buffer; if (comp_len == PAGE_SIZE) @@ -1391,7 +1403,7 @@ static int __zram_bvec_write(struct zram *zram, struct bio_vec *bvec, kunmap_atomic(src); zcomp_stream_put(zram->comp); - zs_unmap_object(zram->mem_pool, handle); + zpool_unmap_handle(zram->mem_pool, handle); atomic64_add(comp_len, &zram->stats.compr_data_size); out: /* @@ -1752,6 +1764,32 @@ static ssize_t disksize_store(struct device *dev, return err; } +static ssize_t allocator_store(struct device *dev, + struct device_attribute *attr, const char *buf, size_t len) +{ + struct zram *zram = dev_to_zram(dev); + int ret; + + down_write(&zram->init_lock); + if (init_done(zram)) { + pr_info("Cannot change disksize for initialized device\n"); + ret = -EBUSY; + goto out; + } + + if (len > sizeof(zram->allocator)) { + pr_info("Allocator backend name too long\n"); + ret = -EINVAL; + goto out; + } + strlcpy(zram->allocator, buf, len); + ret = len; + +out: + up_write(&zram->init_lock); + return ret; +} + static ssize_t reset_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t len) { @@ -1834,6 +1872,7 @@ static DEVICE_ATTR_WO(writeback); static DEVICE_ATTR_RW(writeback_limit); static DEVICE_ATTR_RW(writeback_limit_enable); #endif +static DEVICE_ATTR_RW(allocator); static struct attribute *zram_disk_attrs[] = { &dev_attr_disksize.attr, @@ -1857,6 +1896,7 @@ static struct attribute *zram_disk_attrs[] = { &dev_attr_bd_stat.attr, #endif &dev_attr_debug_stat.attr, + &dev_attr_allocator.attr, NULL, }; @@ -1954,7 +1994,7 @@ static int zram_add(void) device_add_disk(NULL, zram->disk, zram_disk_attr_groups); strlcpy(zram->compressor, default_compressor, sizeof(zram->compressor)); - + strlcpy(zram->allocator, default_allocator, sizeof(zram->allocator)); zram_debugfs_register(zram); pr_info("Added device: %s\n", zram->disk->disk_name); return device_id; diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index f2fd46daa760..8a649a553a7a 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -16,7 +16,7 @@ #define _ZRAM_DRV_H_ #include -#include +#include #include #include "zcomp.h" @@ -28,6 +28,7 @@ #define ZRAM_SECTOR_PER_LOGICAL_BLOCK \ (1 << (ZRAM_LOGICAL_BLOCK_SHIFT - SECTOR_SHIFT)) +#define ZRAM_MAX_ALLOCATOR_NAME 64 /* * The lower ZRAM_FLAG_SHIFT bits of table.flags is for @@ -91,7 +92,7 @@ struct zram_stats { struct zram { struct zram_table_entry *table; - struct zs_pool *mem_pool; + struct zpool *mem_pool; struct zcomp *comp; struct gendisk *disk; /* Prevent concurrent execution of device init */ @@ -108,6 +109,7 @@ struct zram { */ u64 disksize; /* bytes */ char compressor[CRYPTO_MAX_ALG_NAME]; + char allocator[ZRAM_MAX_ALLOCATOR_NAME]; /* * zram is claimed so open request will be failed */