From patchwork Mon Jun 12 09:38:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Domenico Cerasuolo X-Patchwork-Id: 13275953 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65B68C7EE45 for ; Mon, 12 Jun 2023 09:38:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE2B18E0003; Mon, 12 Jun 2023 05:38:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E6BE06B0075; Mon, 12 Jun 2023 05:38:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C4B378E0003; Mon, 12 Jun 2023 05:38:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B61B86B0074 for ; Mon, 12 Jun 2023 05:38:26 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8D5CCAF11A for ; Mon, 12 Jun 2023 09:38:26 +0000 (UTC) X-FDA: 80893595412.17.0D4A96A Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) by imf25.hostedemail.com (Postfix) with ESMTP id 9A492A0005 for ; Mon, 12 Jun 2023 09:38:24 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=BS47izYk; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf25.hostedemail.com: domain of cerasuolodomenico@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=cerasuolodomenico@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686562704; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BYQpKJ28RkPZW3NObwb2r+lw4fKc8lZ4o5qJebIhoW0=; b=BEHtVOdEfWF/ckaAgVYstzIwPnDUMhTI9DrGddcyYZ9b1R6YMZqGGmenk7jSP1dmfbfSy6 fCfFvQzFDXdWORWxGI8kMxjJbpF5A18DTpkL8TjFEXxaGZZ2NpU/awBoVJ2j842iaOCOQO FqYoKDv6z1+bHX4Nbep0w2qob5qOUBE= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=BS47izYk; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf25.hostedemail.com: domain of cerasuolodomenico@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=cerasuolodomenico@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686562704; a=rsa-sha256; cv=none; b=kVOzVZpyGXdrFzoWe//JcpSCoA8L5+YpEXtP2f04TPGIsX3TIQNSjay6T182X6a7P46MT4 P+f8z4rwIjEaDFIzC91RcisfFTTg7MQDb6OIzDzPk/gPZLWfFf16K+RYZ2rYsa8kmo8+FO W398TP2RERwiNDWv800NYqvfYIm+uWM= Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-97668583210so611803066b.1 for ; Mon, 12 Jun 2023 02:38:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686562703; x=1689154703; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BYQpKJ28RkPZW3NObwb2r+lw4fKc8lZ4o5qJebIhoW0=; b=BS47izYkA4or+JYwN3wR0jWIAr7/ME4QNRglwncxoAQO9o+6LY1zTB3kY03hwJHju2 WMBjUOjW/qkJNlH28JdJ+xMVSh3pUuJFN7OzEc0McBsreP74hbwgjYAzIoGkOnD0l5R+ r6QSkkJ2vh72n0/AFNniae1QDv+V4M4vYBiYrsQlOafLteiOGGVzFeSu2+DZXGjsCezo vZkT2lVVm5g49PS8kRftdY2RJiwWYCJJN6sI2bNJ/eySFQ2SQIXXzmm2FAJNYtOdFqxO 6oq2UZjsDZ+09NG2OC5ZxQn26q5+eO6dFGX0+U+ujkJT9Lajq6pG9FJYc1FbfqSwS/t0 N3Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686562703; x=1689154703; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BYQpKJ28RkPZW3NObwb2r+lw4fKc8lZ4o5qJebIhoW0=; b=S3EHKNZman1ectvt0dkeSmNW41n06iVJKHqcTmNEegoCJnFQt+B+Y7dknGxetkCC8N 02TOlyFFWqqEWrs71UzXecwdcB1wwQnH0UaqbOgv9Cv6XSWkZllm/VZaDVTJN/vbcNYF bQg/5xw/Pm7C+i7xq9DOLCiVwHMY54Q2DwhuYSqQulJmdlkafsuFbgqFeQ2kGM7DwmHl CLBnvD3OUKL/NMP68y5D9uYqxWJuBOpXf9rWYNQ41EKr7OWT0PSsfkab1cEt18f1GF8x IzR+bVYRrvDLkRRvDioX4QhhoG3fvLTpwrkzfKGCFIJNV6SPoMNqNEyfRej5jFTC665B vzVA== X-Gm-Message-State: AC+VfDz6/J/D0DLCoJrUjS3qObMa7H4o2Ecw+OW+kRQmN+xqtq0GVYKG jR1H3AfmoFkhNJ8qkrB3KaE= X-Google-Smtp-Source: ACHHUZ6Ig9GsZfwV+Y9+P73Dv4hJNyxOlnWH4bLFoYv6LKiG4MBKz3X8HRd4297KFDPeb/M14sM/3w== X-Received: by 2002:a17:907:6da5:b0:974:4a33:83a7 with SMTP id sb37-20020a1709076da500b009744a3383a7mr8294900ejc.12.1686562702987; Mon, 12 Jun 2023 02:38:22 -0700 (PDT) Received: from lelloman-5950.homenet.telecomitalia.it (host-212-171-43-8.retail.telecomitalia.it. [212.171.43.8]) by smtp.gmail.com with ESMTPSA id ce23-20020a170906b25700b0097887b68c17sm4951358ejb.98.2023.06.12.02.38.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jun 2023 02:38:22 -0700 (PDT) From: Domenico Cerasuolo To: vitaly.wool@konsulko.com, minchan@kernel.org, senozhatsky@chromium.org, yosryahmed@google.com, linux-mm@kvack.org Cc: ddstreet@ieee.org, sjenning@redhat.com, nphamcs@gmail.com, hannes@cmpxchg.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, Domenico Cerasuolo Subject: [PATCH v3 2/7] mm: zswap: remove page reclaim logic from zbud Date: Mon, 12 Jun 2023 11:38:10 +0200 Message-Id: <20230612093815.133504-3-cerasuolodomenico@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230612093815.133504-1-cerasuolodomenico@gmail.com> References: <20230612093815.133504-1-cerasuolodomenico@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 9A492A0005 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: oeb1syzzo7rhsph7694ki9inxxxoowka X-HE-Tag: 1686562704-787476 X-HE-Meta: U2FsdGVkX1+NXcBWxrJTfEHZiNogOUoD1Z0lOoxlHCG6WsRcxcy2/tamZHx9pRCCjYS8s2rWOTGUAGz8VtNJr6Ol0jLCPY6Fq0CztkkqtoJk+11uR9RNuwdbiT/pnU/XU4e0umnQdmgJ9blPAZZD8kyzYz+LD7iAVwZVtzxfSnJyjXTwy6yQDTUl9Qonei8LzJtqY+SYqE+n3Ilc3j9lDe3+QZT6e6tYNvgcWyKxsfSmEu9H8meYWv/E5dukes4UCyUon31+0keKdkmzN5TYfX8fHB/92G3Oytiio1oqy1wtGyNGvJrSEocS05YmvZU9YlwiKunn/5w8NuIPHTkMC0e0r/yH26WiqvyEKs9hgwGUbGao9c7ANP+pmGYlVbydMpnb15xvQXpXPnLnba/YePocpGd2GXIUT1U68LB7ZbxWuv0AEapnsRgc1wU+RfkVmrSF7MjE9iZeGhV/kIVNU+4jrcZwyLu2ipscP8Xwr+BLYqwoHmM0Ztc+mdzHj1Z6gEUuozQu1mG7OR3wLWClNhlvVdqCYM577voMpTM3N6cIuZ6HT3OorT6pV5HhRfcOq2YfuX2ZdzUYbH1GLZICl/QPXrnJpYfRWtwCnlPsyYZwWNWEl1nCmkdtG650t+G7aaV+hd1RsZbEhconm1frs/ZVCsMZbPJvfK/cNco0z3rondIcNWzTLc5n7jOEQhSc81H3TpHgTIgqZgTp39QtFPBRVkFbyiyXALL6wKBMMTxnN7YBWHS++XmyYjzvEo7QvCxmsNKWS4vG4caTb7X3RNJ7EqlXgJmyOAQ1fKUxgeg6PMANMBDncnOGtQLKHJsl9dhFx8LhuvKqMMOAF4VBwWKAB23fFxRlZfDNSIOlg7eI1I0p7YteYt2b7eiwT88s2STNNIXFYlcbDpwwtcWNslzVBiBvd/8Dw4mhw2OHt3lCoE92Bauan8UGQ59ReCOxxafUcCrND8ZsItVXKER Viyvvra4 NoYxH9+fF02afuKVsae5UvtaIK3dw44IxitkmM3ahCeZO/wFX+ISjK6kOZe0nsWyVbMvGS/36tsvx1ILdFGD01sp8A64NWOa2ziRwXR8xzGj/IjK4lInxMuylotIkyHJvfal1L2Q/ZhPIsogPn6/1D/vLfYBw1YGHaWw2xlm4uJsnqmzsa3Nf1sFAmoVSWVVRM649t1V+KaySVl52tDGIK63DtyfPCV/3Z3pJSLw2OIqEkCc+YeK6/LNQkDgjkLOQtr7apmnRPDJWRenoX4WZrm4SINVPYIOWQ2VN56icb40sKUWiTEPEzoZUsKR6ilV0yMWLyiGvwJv9LiJl73qM7oyhU/qhcoMpYqF9So4ylwQ0BxVaDcTi4kOgF2+OsnoUzx2z9XSeRVONY7+uS9lOeB2eQw07YIyMmVakeE8F+Tu4h07BJ3wTLMOfQSPzBpgY9CJVK+IRFKTrCz6lhk8PNOgfC1EXPLD65fPk/UZlI+EEDy/jyoJWXq511u7/KsN00jGK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Switch zbud to the new generic zswap LRU and remove its custom implementation. Acked-by: Johannes Weiner Signed-off-by: Domenico Cerasuolo --- mm/zbud.c | 163 +----------------------------------------------------- 1 file changed, 1 insertion(+), 162 deletions(-) diff --git a/mm/zbud.c b/mm/zbud.c index 3acd26193920..9d35fd4091ed 100644 --- a/mm/zbud.c +++ b/mm/zbud.c @@ -83,11 +83,7 @@ struct zbud_pool; * its free region. * @buddied: list tracking the zbud pages that contain two buddies; * these zbud pages are full - * @lru: list tracking the zbud pages in LRU order by most recently - * added buddy. * @pages_nr: number of zbud pages in the pool. - * @zpool: zpool driver - * @zpool_ops: zpool operations structure with an evict callback * * This structure is allocated at pool creation time and maintains metadata * pertaining to a particular zbud pool. @@ -102,26 +98,20 @@ struct zbud_pool { struct list_head buddied; struct list_head unbuddied[NCHUNKS]; }; - struct list_head lru; u64 pages_nr; - struct zpool *zpool; - const struct zpool_ops *zpool_ops; }; /* * struct zbud_header - zbud page metadata occupying the first chunk of each * zbud page. * @buddy: links the zbud page into the unbuddied/buddied lists in the pool - * @lru: links the zbud page into the lru list in the pool * @first_chunks: the size of the first buddy in chunks, 0 if free * @last_chunks: the size of the last buddy in chunks, 0 if free */ struct zbud_header { struct list_head buddy; - struct list_head lru; unsigned int first_chunks; unsigned int last_chunks; - bool under_reclaim; }; /***************** @@ -149,8 +139,6 @@ static struct zbud_header *init_zbud_page(struct page *page) zhdr->first_chunks = 0; zhdr->last_chunks = 0; INIT_LIST_HEAD(&zhdr->buddy); - INIT_LIST_HEAD(&zhdr->lru); - zhdr->under_reclaim = false; return zhdr; } @@ -221,7 +209,6 @@ static struct zbud_pool *zbud_create_pool(gfp_t gfp) for_each_unbuddied_list(i, 0) INIT_LIST_HEAD(&pool->unbuddied[i]); INIT_LIST_HEAD(&pool->buddied); - INIT_LIST_HEAD(&pool->lru); pool->pages_nr = 0; return pool; } @@ -310,11 +297,6 @@ static int zbud_alloc(struct zbud_pool *pool, size_t size, gfp_t gfp, list_add(&zhdr->buddy, &pool->buddied); } - /* Add/move zbud page to beginning of LRU */ - if (!list_empty(&zhdr->lru)) - list_del(&zhdr->lru); - list_add(&zhdr->lru, &pool->lru); - *handle = encode_handle(zhdr, bud); spin_unlock(&pool->lock); @@ -325,11 +307,6 @@ static int zbud_alloc(struct zbud_pool *pool, size_t size, gfp_t gfp, * zbud_free() - frees the allocation associated with the given handle * @pool: pool in which the allocation resided * @handle: handle associated with the allocation returned by zbud_alloc() - * - * In the case that the zbud page in which the allocation resides is under - * reclaim, as indicated by the PG_reclaim flag being set, this function - * only sets the first|last_chunks to 0. The page is actually freed - * once both buddies are evicted (see zbud_reclaim_page() below). */ static void zbud_free(struct zbud_pool *pool, unsigned long handle) { @@ -345,18 +322,11 @@ static void zbud_free(struct zbud_pool *pool, unsigned long handle) else zhdr->first_chunks = 0; - if (zhdr->under_reclaim) { - /* zbud page is under reclaim, reclaim will free */ - spin_unlock(&pool->lock); - return; - } - /* Remove from existing buddy list */ list_del(&zhdr->buddy); if (zhdr->first_chunks == 0 && zhdr->last_chunks == 0) { /* zbud page is empty, free */ - list_del(&zhdr->lru); free_zbud_page(zhdr); pool->pages_nr--; } else { @@ -368,110 +338,6 @@ static void zbud_free(struct zbud_pool *pool, unsigned long handle) spin_unlock(&pool->lock); } -/** - * zbud_reclaim_page() - evicts allocations from a pool page and frees it - * @pool: pool from which a page will attempt to be evicted - * @retries: number of pages on the LRU list for which eviction will - * be attempted before failing - * - * zbud reclaim is different from normal system reclaim in that the reclaim is - * done from the bottom, up. This is because only the bottom layer, zbud, has - * information on how the allocations are organized within each zbud page. This - * has the potential to create interesting locking situations between zbud and - * the user, however. - * - * To avoid these, this is how zbud_reclaim_page() should be called: - * - * The user detects a page should be reclaimed and calls zbud_reclaim_page(). - * zbud_reclaim_page() will remove a zbud page from the pool LRU list and call - * the user-defined eviction handler with the pool and handle as arguments. - * - * If the handle can not be evicted, the eviction handler should return - * non-zero. zbud_reclaim_page() will add the zbud page back to the - * appropriate list and try the next zbud page on the LRU up to - * a user defined number of retries. - * - * If the handle is successfully evicted, the eviction handler should - * return 0 _and_ should have called zbud_free() on the handle. zbud_free() - * contains logic to delay freeing the page if the page is under reclaim, - * as indicated by the setting of the PG_reclaim flag on the underlying page. - * - * If all buddies in the zbud page are successfully evicted, then the - * zbud page can be freed. - * - * Returns: 0 if page is successfully freed, otherwise -EINVAL if there are - * no pages to evict or an eviction handler is not registered, -EAGAIN if - * the retry limit was hit. - */ -static int zbud_reclaim_page(struct zbud_pool *pool, unsigned int retries) -{ - int i, ret, freechunks; - struct zbud_header *zhdr; - unsigned long first_handle = 0, last_handle = 0; - - spin_lock(&pool->lock); - if (list_empty(&pool->lru)) { - spin_unlock(&pool->lock); - return -EINVAL; - } - for (i = 0; i < retries; i++) { - zhdr = list_last_entry(&pool->lru, struct zbud_header, lru); - list_del(&zhdr->lru); - list_del(&zhdr->buddy); - /* Protect zbud page against free */ - zhdr->under_reclaim = true; - /* - * We need encode the handles before unlocking, since we can - * race with free that will set (first|last)_chunks to 0 - */ - first_handle = 0; - last_handle = 0; - if (zhdr->first_chunks) - first_handle = encode_handle(zhdr, FIRST); - if (zhdr->last_chunks) - last_handle = encode_handle(zhdr, LAST); - spin_unlock(&pool->lock); - - /* Issue the eviction callback(s) */ - if (first_handle) { - ret = pool->zpool_ops->evict(pool->zpool, first_handle); - if (ret) - goto next; - } - if (last_handle) { - ret = pool->zpool_ops->evict(pool->zpool, last_handle); - if (ret) - goto next; - } -next: - spin_lock(&pool->lock); - zhdr->under_reclaim = false; - if (zhdr->first_chunks == 0 && zhdr->last_chunks == 0) { - /* - * Both buddies are now free, free the zbud page and - * return success. - */ - free_zbud_page(zhdr); - pool->pages_nr--; - spin_unlock(&pool->lock); - return 0; - } else if (zhdr->first_chunks == 0 || - zhdr->last_chunks == 0) { - /* add to unbuddied list */ - freechunks = num_free_chunks(zhdr); - list_add(&zhdr->buddy, &pool->unbuddied[freechunks]); - } else { - /* add to buddied list */ - list_add(&zhdr->buddy, &pool->buddied); - } - - /* add to beginning of LRU */ - list_add(&zhdr->lru, &pool->lru); - } - spin_unlock(&pool->lock); - return -EAGAIN; -} - /** * zbud_map() - maps the allocation associated with the given handle * @pool: pool in which the allocation resides @@ -518,14 +384,7 @@ static void *zbud_zpool_create(const char *name, gfp_t gfp, const struct zpool_ops *zpool_ops, struct zpool *zpool) { - struct zbud_pool *pool; - - pool = zbud_create_pool(gfp); - if (pool) { - pool->zpool = zpool; - pool->zpool_ops = zpool_ops; - } - return pool; + return zbud_create_pool(gfp); } static void zbud_zpool_destroy(void *pool) @@ -543,25 +402,6 @@ static void zbud_zpool_free(void *pool, unsigned long handle) zbud_free(pool, handle); } -static int zbud_zpool_shrink(void *pool, unsigned int pages, - unsigned int *reclaimed) -{ - unsigned int total = 0; - int ret = -EINVAL; - - while (total < pages) { - ret = zbud_reclaim_page(pool, 8); - if (ret < 0) - break; - total++; - } - - if (reclaimed) - *reclaimed = total; - - return ret; -} - static void *zbud_zpool_map(void *pool, unsigned long handle, enum zpool_mapmode mm) { @@ -585,7 +425,6 @@ static struct zpool_driver zbud_zpool_driver = { .destroy = zbud_zpool_destroy, .malloc = zbud_zpool_malloc, .free = zbud_zpool_free, - .shrink = zbud_zpool_shrink, .map = zbud_zpool_map, .unmap = zbud_zpool_unmap, .total_size = zbud_zpool_total_size,