From patchwork Sat Jun 24 05:12:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13291598 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52A67EB64DD for ; Sat, 24 Jun 2023 05:31:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7C71D8D0003; Sat, 24 Jun 2023 01:31:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 778358D0001; Sat, 24 Jun 2023 01:31:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 616708D0003; Sat, 24 Jun 2023 01:31:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 5242C8D0001 for ; Sat, 24 Jun 2023 01:31:32 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 12AD51C932A for ; Sat, 24 Jun 2023 05:31:32 +0000 (UTC) X-FDA: 80936518824.16.45435E2 Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by imf16.hostedemail.com (Postfix) with ESMTP id 3884A180009 for ; Sat, 24 Jun 2023 05:31:30 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=LANh1sDQ; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf16.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.215.175 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687584690; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=A39gdokmxlm4U38P/Zql/B/dp1bRrgyGGMBd2ncXVBo=; b=l6XCiYGnapvVv+++ObG9elI4kphyBDRho+4xEwZuT6AoBM1fiWuAfQa8D/NUtmIpI4p4dX HicSH21Yov3vme2wrvajAJ62hvnO4RtzIMIWTK2fC46AznhOzyjyt3kT7SaL0t0jXgb4EH bv1U2ctQ99jDcxJBosiUPkDhBVwkMHs= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=LANh1sDQ; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf16.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.215.175 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687584690; a=rsa-sha256; cv=none; b=WBDUMDGOk+ICLZMKYJ6f3Nkf0U/e/+SKUciDq6FZNqjxcWEMFtPeAOTIwN0Ljv9+OJTs4x +OHhJbKLXT9bZeHKjVwXWboU2w2I4OU2jaIfLgQWYklVtMxnwLzhaVXGaPeQhs3rDnGdgx tFPBiPazeiZ8g9KEHc44QKRD6r+0FO4= Received: by mail-pg1-f175.google.com with SMTP id 41be03b00d2f7-54f85f8b961so1108563a12.3 for ; Fri, 23 Jun 2023 22:31:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1687584689; x=1690176689; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=A39gdokmxlm4U38P/Zql/B/dp1bRrgyGGMBd2ncXVBo=; b=LANh1sDQ0Cqew/nk2fWgtBz4HwaykMfnm7u6tguZ9kC8E+QeXfexEIVbVk2ZI1cxKA iQQ1M/FSjdcMGc/TD9zjcogPnBiT5Dm3/lo/jXUz2HCzhl0UochpEpS3Afhg5MXLPkzb UbOHYYKYV/58KxdqAgSLYHZjfLyRODgzI9/7s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687584689; x=1690176689; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=A39gdokmxlm4U38P/Zql/B/dp1bRrgyGGMBd2ncXVBo=; b=jh3v5O4OtzOzcKemxOD5LWvf6EvZXnuh/wRxM+mgyj0JkY+Vjt8xuCV7IuJt0Ta13T tfalQ3cPZ983SqKHy5g3VDTYZU9nbBmuu86yqpAoo/i5m2qzY/lzn6Oh07WpMBYDGRFq ynKPaHWiMKlf3RHJ/vV9tSDFChLFfjM9b31/dNB3gUiWgUbzJHSH9xp5xcwawN/qXJMi fURF8973JnhuShnNFLG89/uMmgmSJsRCRFIA6g1eBBhQh3LEGs1ToT1drf1lsjKuQESb 37f5MdbjZpsOHVmE7hWjSyXjDsrqkZRhStNz/HPlUnGcT/JgEJAJD5yU/u1B3yBrAaIq b+SA== X-Gm-Message-State: AC+VfDxCNYK1DyME6L25BfOn4GIifGJZmy7hFgqE1PduwS74Unp8vfTD UcHK88OrXyW8Ux8zyWMLwobcUg== X-Google-Smtp-Source: ACHHUZ52NiJUM2vh+TDHLpF+yhX0gxBj2JOBI8yeSeM/qrXW8datRm1xeFixMs2KJoNyN7fYpdbnyg== X-Received: by 2002:a17:90b:147:b0:25c:571:8670 with SMTP id em7-20020a17090b014700b0025c05718670mr22087582pjb.46.1687584688858; Fri, 23 Jun 2023 22:31:28 -0700 (PDT) Received: from tigerii.tok.corp.google.com ([2401:fa00:8f:203:d7fc:8a97:4a18:6a18]) by smtp.gmail.com with ESMTPSA id 16-20020a17090a0c1000b002563e8a0225sm567069pjs.48.2023.06.23.22.31.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Jun 2023 22:31:28 -0700 (PDT) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky , Alexey Romanov Subject: [PATCHv2 1/3] zsmalloc: do not scan for allocated objects in empty zspage Date: Sat, 24 Jun 2023 14:12:14 +0900 Message-ID: <20230624053120.643409-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog In-Reply-To: <20230624053120.643409-1-senozhatsky@chromium.org> References: <20230624053120.643409-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 3884A180009 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: yihs9j4ujkgx6xrdcdyf41i9gw94yy8w X-HE-Tag: 1687584690-566819 X-HE-Meta: U2FsdGVkX1+fs6LAM4ln0A7Tvqb9cmK2/Ql4XS7f61XiiEhKtBmG9j46n0bj5iIH6COQnvQqmQepPOlM7+wJWawTxEvaKGjHnIy2v69PsV9SHmXWinEvUefcJHjRNXrmeZrc7Js55HcBzfixgTrEF2wtoGC2ZfBqWsMpER59CeBR+4Aot1OcYVdknoCXwocagm6sf8BS1vOmoTevaRCgEI2rUor3R0PrVQ3fBKOxI+AKtHXbQt8BxyN4E3vEVXdZ6TAU3Z4eOYiNLOx444wB1jC+4gksH81UyO5XrUDbFXWPf3NWvrjFWRoaa8ijZ+Kvn9TMYtPfBCUFFaGoeS8ZX+5dlLDKEpT3UnT1MC0gWi2+nC0meg/T+HN9Ou7Zkk2Dx6btIbW7EVOjEafzi82iyJIFC/kjVlgvI5nQfHu6eHFEcZZ8H5/LQ4ww17QV8MVeWkgFlCWxAw2LeEim2b94w7mww8cpfYfCYICAbRo1LiCdjxuh3pOxXOmtyDoiN7jvLVHph5M3UqVGEfk5UETvZqZVEs7Gyu0g3hrl9jxzYQmBOzWW4NELJAJstXkPmnPM7MQlz3EwFbji5a3hO9tfv6R4DH1sJsRPBzmG/tky50rNEDH33IMcVossE/6WCWE+cmoyFyn++5sOFtSNMnN8uyUwvxI15EkHvz1O7ZCI479qoppkMPeWI7+spNTnUb6GOanhGM2uz42CMZIBOBpJO6mKcXA+W7eKsBl1A571A9lKRyyiu5M8EdrBZcrl44J0RZiql/Aic6dnUKlpfoMAx31muIoGwLwPkwvMlEs7V7LanSamOO8pHF/hbN8E+lMoxRo5MkujgccHCFfAz6ua2xGOCO34bYUifycjSKhPnoDA0xohy7ubYIpOE6GH/kvz6W+WhH/OQy9MynHGf03daXjC/425OLLRAZhn2zmbCN5KtbiczL0Nf87cHIofe0GwKYCLO1GXqRwBfu1VT9q JbXFET2C IO9Cg9uHv+McsVmPhYTlcVf83sEsgCuXtOVW8M1+OiKCyogzDrNe3Wq6ohHv4BLAY/GVNSW//hQ2+srq5lY9wDa5i0zXG72cF4V6UZia0bbwEn8ZtyLqjZt346N5E2RFTp+uv9inBlBYdVjxc2RAROZkh+E5oTKim5Y3KNcWiqLIvW8ky/fJbUEH05kKy6SjuWs7R1T10i+tI919ls8gM9deJP8oe8IAo1XmIEPttI0pYbpfM1dzrv9F/HMpXXoR9/2ZWcR1yza7b0INEdJGsoeIKI4OCeuvPHanbJG7Cm5DeE4DA8DCjRJkgDiSfmgvNEgKPY15a3uBaOwXILHlely2xU35gQ8Qb191h8IwbK3DSWRuTMi6LRgRWMWZYVVuNYAUfCIGaBKnjMBGZOTTPB2FrwrzdtPNbJsyiNSjGXCaDW1nP3fsLBocfLBHmKfG2r37y1qSVHTiTRbYcAKdXlHoQFw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: zspage migration can terminate as soon as it moves the last allocated object from the source zspage. Add a simple helper zspage_empty() that tests zspage ->inuse on each migration iteration. Suggested-by: Alexey Romanov Signed-off-by: Sergey Senozhatsky Reviewed-by: Alexey Romanov Acked-by: Minchan Kim --- mm/zsmalloc.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 3f057970504e..5d60eaedc3b7 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -1147,6 +1147,11 @@ static bool zspage_full(struct size_class *class, struct zspage *zspage) return get_zspage_inuse(zspage) == class->objs_per_zspage; } +static bool zspage_empty(struct zspage *zspage) +{ + return get_zspage_inuse(zspage) == 0; +} + /** * zs_lookup_class_index() - Returns index of the zsmalloc &size_class * that hold objects of the provided size. @@ -1625,6 +1630,10 @@ static void migrate_zspage(struct zs_pool *pool, struct size_class *class, obj_idx++; record_obj(handle, free_obj); obj_free(class->size, used_obj); + + /* Stop if there are no more objects to migrate */ + if (zspage_empty(get_zspage(s_page))) + break; } /* Remember last position in this iteration */