From patchwork Wed Dec 18 06:34:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13913084 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D844FE77187 for ; Wed, 18 Dec 2024 06:35:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 718746B0085; Wed, 18 Dec 2024 01:35:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6C80B6B0088; Wed, 18 Dec 2024 01:35:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56A086B0089; Wed, 18 Dec 2024 01:35:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 35E5D6B0085 for ; Wed, 18 Dec 2024 01:35:27 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id A225180BF6 for ; Wed, 18 Dec 2024 06:35:26 +0000 (UTC) X-FDA: 82907117328.26.A9AD1A9 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf05.hostedemail.com (Postfix) with ESMTP id 9C2C310000F for ; Wed, 18 Dec 2024 06:34:24 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=ktac5ihi; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf05.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.179 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734503710; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YM5tbrHoFcnr7Yi5hBwjzOD0QAW+fJr+gF9sDLsbf7Q=; b=Hw8AcSzSHn7droUXAC/LHTsAfQ4Y4neEXNamapdMP1Ole67+w6FnU1WUz/oVF6/EheKDFd wHSOq0WVDV30l+f44S75ZIBhj0Jog1Dc3+JUk89v3WuTG7NXkZajGHh9efeWQINu+i5EaD I4lFLIrIauZDdkJthemdm2d2eIbLfEQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734503710; a=rsa-sha256; cv=none; b=k1EgZvqdMPwhqmzaN3vWd6f6MbRcw423JxO1GRYyWyQnjqA4+TMPuTLsdmZ0D3n5AU99PA TSQ+AWJeuMkJ83QnAbPYx1ajR2Qeh1U7ttm50XXbWvOw86l3ad/K9VjGUYQDw+a7W69Sgt bZxtGCLS7Lp/QF3On1fgdL1mnsGEHRE= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=ktac5ihi; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf05.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.179 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-218c8aca5f1so19808275ad.0 for ; Tue, 17 Dec 2024 22:35:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1734503723; x=1735108523; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YM5tbrHoFcnr7Yi5hBwjzOD0QAW+fJr+gF9sDLsbf7Q=; b=ktac5ihin0oxfHeRrA//FqOuEuZ/gemxvcSDmgBldZp6fpycO44/nOIKkxo6vttngp Ex1Gf9W1q19dN1M/8LXDx5zMHL8T2MCyNtpjP86DUuwkn/au3NJbanLtPnZPo9cEfEBH OKHuiiLcWb/G439/KQUgUnyM2R8FeEefgRzYA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734503723; x=1735108523; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YM5tbrHoFcnr7Yi5hBwjzOD0QAW+fJr+gF9sDLsbf7Q=; b=mkPc3f4OI0LhabkfKkXS0f3vJOdk4yxitFTQPeAA8YQS7STTvL8XHEzmfoEhwv/8ct FfsBTTO6r7emYRd4vdwTrECvCA3mNeHuykjW3o+cHQbNwWBVW1RL0KVqaQN3WkST0NpZ m7pCFEPRDsT9A4juz/HeKUKuxIwA+PGf1dCbxtOQETAWvbQu6qkjADLOdiPdBOi1/juf EHgJ984YUY0oDvFY5TWkCn7i7ilqngyvk+M9FNTnnqHN1tllRhf/PHfBjz5G7M3tCIl2 oHh5Uvw/O2Djw2dzCc+g0cvgVMi8KlJHg6ugV3n1EsHYJU7/wmR4pyJaBYBlx7XKf6XE 76Ag== X-Forwarded-Encrypted: i=1; AJvYcCX08MKZJKMEo59Y5CQs1dP5FBpz2wWhaM5ru5zEQ2EEJRB5uYEApeWSyS6hIuxpat83dDMEeDoXlQ==@kvack.org X-Gm-Message-State: AOJu0YzX6hYfM+EfrahSSnFatlhgYZc6ULAXujk4U23jeRcRzNh0pjl0 MiS/DdHcnvCP9AR1v7CGlbrj3VbIVFd/4pQTpBAKguBuCRIsavjyHbGgQL7fzw== X-Gm-Gg: ASbGncuS4F97jmTMpOaetA+JEn3BtfnhbxBv26TN3v8thuIspDZ/uRx1OcbV0B++0Br mUw8JfdtoUOkMggkSVh1ULtARLeYLMnYOPfeVs6wbm9RilSKCicZkA/+kxQdJ7kpf7ZbAUSpdmo IvYlGRSppx4v+DgAD19Z6NE3qbz6wD9L183//ATbSDW7FFdfv2zyf6iKsXRGpA6aPRNEnI7ECjp TnuBs2BfXvMGu2KpUilTl1FR+CtuIqAQ+kjQD6dZFdGGzBtzF+077o7IXk= X-Google-Smtp-Source: AGHT+IEzRX0s0ipw+MkmY6rxb4hZkmahH0+HA2BPVeSSTzwnMahAGsE7gUMp3e22K+tr163jhvT0PQ== X-Received: by 2002:a17:902:dac6:b0:216:7cd8:e964 with SMTP id d9443c01a7336-218d710417emr25045545ad.22.1734503723387; Tue, 17 Dec 2024 22:35:23 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:3bcc:36cc:b9fe:9379]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-218a1dcb8e9sm69394115ad.83.2024.12.17.22.35.21 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Dec 2024 22:35:23 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 1/7] zram: free slot memory early during write Date: Wed, 18 Dec 2024 15:34:18 +0900 Message-ID: <20241218063513.297475-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog In-Reply-To: <20241218063513.297475-1-senozhatsky@chromium.org> References: <20241218063513.297475-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Stat-Signature: yqimt35rrhd1ycqsh6ndz4pajarf7q3d X-Rspamd-Queue-Id: 9C2C310000F X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1734503664-878623 X-HE-Meta: U2FsdGVkX1+SebtzIMswoHSJYPNBZQV+V5egtTSzCe+AgikUa0SMW9LOAHKl4Fszapwi3WEhK6W3YW/uDV45MUMhCuHoJIjd1hIxDxSfFVdOurnwVvRnGxD1quOD6ZBtCFMlJvdAWLcFz8eK49VLyQuHJtjyB9y+qY1GHeUVEKwft+wDrq2GkAUHDNMl49XyjEjixxW9ueIV7B8ZXlTTYsK2p0CPc0bQ7oIbvnpGORWvHlgQ0lVNSYyblSqh1aoPrQtt8R1899IxtT5+yTQMouqYiTchqf1K48VIfCWLvP//4MJ589UoW/+QNuuc8R3npx2j9tDHPIvkBfH2H46koUDAKFJudkQADA0M+ncbpzde6A0VRV+XsNKhuseP2XMiDtuV95hXUW9UjiwZCUvnHMrulXpc43sE9Fx9f7d0bacgIRXWuGjsN41Vo1g1TBeQ5Z4JbG/nX6MivVbaM7IOn/seEetvJNh5oWTF970SqkN5ko8iKiOES+sVL6BsPC9iXFGIwv7RaaZcNkNKeYpzd+NcqHy1pmwkby/LgzrjbGerJbOw/RVCr5viAuirmnbtf2f9ZuQxwSto2loxyKyePjNaniHwBQpQvC/FNBNrcRSkg5EI740a9z5arwXx99cpmtXEOvJPcn/M3j3cnApvSh2F/PpfW/iFiXVuCaA31j+I5VYwDCoH2sVieXyZ+VZgUU3g5IAM2ECwN2mNRwshz0bKKiQpVhve9MozjtVRTPXRA6M6yGl+JZGjCkoIm2TZUU6VNZRroFBmIcAGT8GpHahPZDterbx6A84nYYQmmIbgABEDEQp+jugMR8ijYIA3M6jG8/t8CwoMH2xy6yc/7bnV/rwQo2MkdqjaBCsEk3cIDYM2zPxIqUBhF+9EFhjcsZ1yfaW3RmLmsVp54hR8oeTgwuU454HmtiurGe2ZIEafiYle0R7+o4z63W8TQUIlyMi2SPf5bwmfi8avW5k C+9OpMHO 1CFVOu/BuPPk+ny8yYj0NIy5djEN2pGnUSNij3xrbFzrKFWBK1JQcbGTyiwg/BTBsbBUq0DtkE/gDobooh3iuRhuEwcDU6IVQKJWGD3VJL/gUwx9qIAaxUTeOMSXjJ1zF8OVRQixAhPcMUtfyKwTCHsGd2RiNA69lL8s1gV3JX0hd4t9kCZeVGkWVeZnQoox2XbN1++P9POcbDe7/5ADgSKiE3na0AGGhhQo58RZpNi6vqvNbE9YAfLHsmTam4j+DXbRLFZJurr/2C7M= X-Bogosity: Ham, tests=bogofilter, spamicity=0.329346, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In the current implementation entry's previously allocated memory is released in the very last moment, when we already have allocated a new memory for new data. This, basically, temporarily increases memory usage for no good reason. For example, consider the case when both old (stale) and new entry data are incompressible so such entry will temporarily use two physical pages - one for stale (old) data and one for new data. We can release old memory as soon as we get a write request for entry. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 45df5eeabc5e..cda0f3fd4058 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1648,6 +1648,11 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) unsigned long element = 0; enum zram_pageflags flags = 0; + /* First, free memory allocated to this slot (if any) */ + zram_slot_lock(zram, index); + zram_free_page(zram, index); + zram_slot_unlock(zram, index); + mem = kmap_local_page(page); if (page_same_filled(mem, &element)) { kunmap_local(mem); @@ -1736,13 +1741,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) zs_unmap_object(zram->mem_pool, handle); atomic64_add(comp_len, &zram->stats.compr_data_size); out: - /* - * Free memory associated with this sector - * before overwriting unused sectors. - */ zram_slot_lock(zram, index); - zram_free_page(zram, index); - if (comp_len == PAGE_SIZE) { zram_set_flag(zram, index, ZRAM_HUGE); atomic64_inc(&zram->stats.huge_pages); From patchwork Wed Dec 18 06:34:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13913085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DB31E77183 for ; Wed, 18 Dec 2024 06:35:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 210DA6B0089; Wed, 18 Dec 2024 01:35:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1C0466B008A; Wed, 18 Dec 2024 01:35:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 013776B008C; Wed, 18 Dec 2024 01:35:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D31EE6B0089 for ; Wed, 18 Dec 2024 01:35:30 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 50A3C160BBA for ; Wed, 18 Dec 2024 06:35:30 +0000 (UTC) X-FDA: 82907117412.23.4876ABC Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf14.hostedemail.com (Postfix) with ESMTP id A4CBD100005 for ; Wed, 18 Dec 2024 06:34:53 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=CqM4iqg0; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734503713; a=rsa-sha256; cv=none; b=zGz71MXElCOSqvERhVNQoX8RnRYx4aK8cTmR4YnTO0WMjLY3Nzzhy+71m62mWStv37ljOA sOqZLmMc/hSO2OYdHvVKpmmbSwwnWGm2TKOW9tu24VxZrEiio32nItpN3ezw9hPOVCXwBY trot6EkUdESAEFfwRMIl1nFwZdumVWo= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=CqM4iqg0; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf14.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734503713; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1lsNLWZe1xx8LIy2DkJ6EwI7DqhKpPjMJ1gyizR1Gbo=; b=xu57j3Rpmrih9H61YM4ZXnV31Ti0APYDB2100A7+RZ+PkyvVBRZAE8/NX3/vK8PlMaZu8A FeaKKMF7BxgHXpcFTM2EtFYHJa/7Geux8W3qgqJ3SvaHvoz9Fxj5XRktS2M6DC2MEE3UoR GGwyITDFLmZZQCwAJU2yU79UvJcRzOg= Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-218c8aca5f1so19808685ad.0 for ; Tue, 17 Dec 2024 22:35:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1734503727; x=1735108527; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1lsNLWZe1xx8LIy2DkJ6EwI7DqhKpPjMJ1gyizR1Gbo=; b=CqM4iqg0kJeZb+5zTR1X7KIfDKaruduWIMsvbbHJsrej2GjXDP6e1b2RcAK4YJRKvi iAxQHNv5p4670LWXB92ZuoUbDXbFOTUGsrfvN1GkgEaNVHoesBent5hI2q+tLIWe1Wby e9xyx3hMzbCxUuvIPlwNsY2dQ4nS2LOViglu4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734503727; x=1735108527; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1lsNLWZe1xx8LIy2DkJ6EwI7DqhKpPjMJ1gyizR1Gbo=; b=BvPXbCB7MGA994Z0vEsVR79rFK3UUraz90vJSExfyJ4mgE33yYrnAtpOa+Kf9xqonD mi8LCLCZpC86S91I7l9SoTOODDWeAO/kQUp6wUFFu47KHy6z7hHZXRzq0tAsxI+oMqCa Ju+bO4hrFQlB6E2iE6Kam8iZIXaImkVPVuavHSYsLmPoMYWzv3zeenwanBDMQi9hqt0O /06F5UfEOuX9U+1O27/PrbQ6PuIw3tICwRcHpsSGa6bZVgZ1s6UjwBHsZFKLuUyRbrET 9wIs0kWBozpYn1SKrEBdPPd9nKtwAL6wrZRDhraf7IP6iENwXKJkzuJfVynCh06r5qbQ Trkg== X-Forwarded-Encrypted: i=1; AJvYcCW5MLFwp8fXS2sf2nMQhRxpEhWu944tD3dbHS+BTr8vdCpeQD0oloOUaJVVa2MGX8+ACd+qrK6dMA==@kvack.org X-Gm-Message-State: AOJu0YzyVbj8cl9X1K5zTnnpBqaktO14lPMzZTndMdWNJXc++mERmGml 1GODxuNTxLZaXxGzXV/jD/iWIcTmtmIwwB7dX34e1KWCHi4T1+XDF1IugqbFOg== X-Gm-Gg: ASbGncuaX/aO9CExJX2RduyX9HGGJCmCtExoAhNLx74+qK/DCgbpRNBM7QV/Y0aFPjM q43qz4gaifD3QLlwLPbeqKULgf8C+zbWzso4XFFeEYWHDOmtOebhlNrGneQr8GsTSJ/71ir91r2 YDbvBnRle6Z2pddrXHNIh6HCdc3oYwuz4LU5s1dwFsBx6t4ZiKNOUtTAnef4It/+FcomjQYbK7p NpWEjbREgD1TvobS5EnenG9HMcr573v7ZzjAEJVS7feK2QLtd0V7vVprOg= X-Google-Smtp-Source: AGHT+IGtvXF84vSzKEt9nvcRn17lafPxKZcVbWSXwCigI3N+2LoaMAbFgPOkqbnj99UeaG1XnEyJIA== X-Received: by 2002:a17:903:2447:b0:216:725c:a12c with SMTP id d9443c01a7336-218d6fd7926mr23537805ad.9.1734503727216; Tue, 17 Dec 2024 22:35:27 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:3bcc:36cc:b9fe:9379]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2f2ed62cddbsm656772a91.15.2024.12.17.22.35.25 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Dec 2024 22:35:26 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 2/7] zram: remove entry element member Date: Wed, 18 Dec 2024 15:34:19 +0900 Message-ID: <20241218063513.297475-3-senozhatsky@chromium.org> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog In-Reply-To: <20241218063513.297475-1-senozhatsky@chromium.org> References: <20241218063513.297475-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: A4CBD100005 X-Stat-Signature: p7oka54wggsbot96gsbo96xj4w9fq8cs X-HE-Tag: 1734503693-309683 X-HE-Meta: U2FsdGVkX1+frescl5lR5jHJX9lcZgXChbkBQs7Zm/TChxdIWiABNyX3EqSjHXZ5vHpm/1YUvJ1L2AHGeEkapDEmAAv3FOctqs98b63jqXO++bXvbcvCM1F47sdO/cKgmugF1j4d4DgQvWLuR0UdWK7HdT7f0Y4FhK9XIblH47LbPjgxk5kllBDSlTltgT9AVzaUSmaSL+cLB6bKFYAjVfOAZ7ZCIkM4fDHDWWkaVrod40+RGbqMx5aMvn/qFGsKrIq4FnQsgR6elF1t407juanhiwRFGhEfEzfZAW1jabWYyDcbn6uZb1If95SMtuRBuy6JLkVWY0VGTerJ+IRrpDFwSG3SusJdO5Rrvi4VqZijClqz2TR3Bu5ghHjtRU4uNu/exZS+wcidWYL0V+sZBRSlW7xcHmzaitE7Yxaha8Uc3LRjdn8aqBPkFwKd01SR38/JTWwOe6yWY0eiFxT63amlN55rki5ZP/71LqMsnLojmJBTtpVe4KCHMvhORAcJZsVdMwaXrUMFKeD7ZbQ1RbfW2jAGjw1YGqwmPKSjg79i8Y9GW7fpZxIpKAOSxPx9yfxmyTN9sRruNicpgiVKaXCmcHwP3Vlu16mOXiawDB/HxpbEi2vI2qVVf3jl7XXALAN5pvbxtLE91yPuqGsWeZOMOW7Ps7IX0uw4Z2AXtZ1zC80rxNlJsvIscegAR6xSV45iyjGrFqmwN2aHZ5mkMMevqeHg4pYQm5zeQcjDAkiREhqc7IADAy9jUStjfUOeJk1Q29+T96H1E6QvS50nZJF82BKxUtW4im4rwqEjmWEteI/O22UvQ9qz1Th9dbpkY92SESlSKJbNclrw0hvmN5DsDsiZUH+xesaritI5XV5Gk3oyRd6Oi89jp4HafwmPL5XCvDyBxPXCWHFFW1SUtR7brKaHvANhPcVTIIKyTdzC+5TpJTmUhcIzhW8PFQ2EMQPdNqeebw+d0Fk2iCc vONngslv 9XL4VvddeW2Yz0OdJ7nxVVna+hO/6RWsAFIc9dlRMaSGdQe/TfRXVlGyqdU6r833u4UBOTeHoiQVdDOYFfLrY4f2JlleNjJehk/Ddg1HEPbGm2cYsgmlx22pGwGlUPOeIs5RAPm5M52SYvKc+Xo19v6V2MaPmn070wBFvt5kgNkSIZXeGf+X+k463iX23ySFpfLirAxudwXWdIWQw7NKkhQD3ThxjwPM+QKiOHUCxWTG75ohK3cxMvWgocEhMYFgX4ojgJt4xHZJM/Ng= X-Bogosity: Ham, tests=bogofilter, spamicity=0.039701, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Element is in the same anon union as handle and hence holds the same value, which makes code below sort of confusing handle = zram_get_handle() if (!handle) element = zram_get_element() Element doesn't really simplify the code, let's just remove it. We already re-purpose handle to store the block id a written back page. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 23 +++++------------------ drivers/block/zram/zram_drv.h | 5 +---- 2 files changed, 6 insertions(+), 22 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index cda0f3fd4058..8c71ddd17024 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -112,17 +112,6 @@ static void zram_clear_flag(struct zram *zram, u32 index, zram->table[index].flags &= ~BIT(flag); } -static inline void zram_set_element(struct zram *zram, u32 index, - unsigned long element) -{ - zram->table[index].element = element; -} - -static unsigned long zram_get_element(struct zram *zram, u32 index) -{ - return zram->table[index].element; -} - static size_t zram_get_obj_size(struct zram *zram, u32 index) { return zram->table[index].flags & (BIT(ZRAM_FLAG_SHIFT) - 1); @@ -879,7 +868,7 @@ static ssize_t writeback_store(struct device *dev, zram_free_page(zram, index); zram_set_flag(zram, index, ZRAM_WB); - zram_set_element(zram, index, blk_idx); + zram_set_handle(zram, index, blk_idx); blk_idx = 0; atomic64_inc(&zram->stats.pages_stored); spin_lock(&zram->wb_limit_lock); @@ -1504,7 +1493,7 @@ static void zram_free_page(struct zram *zram, size_t index) if (zram_test_flag(zram, index, ZRAM_WB)) { zram_clear_flag(zram, index, ZRAM_WB); - free_block_bdev(zram, zram_get_element(zram, index)); + free_block_bdev(zram, zram_get_handle(zram, index)); goto out; } @@ -1548,12 +1537,10 @@ static int zram_read_from_zspool(struct zram *zram, struct page *page, handle = zram_get_handle(zram, index); if (!handle || zram_test_flag(zram, index, ZRAM_SAME)) { - unsigned long value; void *mem; - value = handle ? zram_get_element(zram, index) : 0; mem = kmap_local_page(page); - zram_fill_page(mem, PAGE_SIZE, value); + zram_fill_page(mem, PAGE_SIZE, handle); kunmap_local(mem); return 0; } @@ -1599,7 +1586,7 @@ static int zram_read_page(struct zram *zram, struct page *page, u32 index, */ zram_slot_unlock(zram, index); - ret = read_from_bdev(zram, page, zram_get_element(zram, index), + ret = read_from_bdev(zram, page, zram_get_handle(zram, index), parent); } @@ -1750,7 +1737,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (flags) { zram_set_flag(zram, index, flags); - zram_set_element(zram, index, element); + zram_set_handle(zram, index, element); } else { zram_set_handle(zram, index, handle); zram_set_obj_size(zram, index, comp_len); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 134be414e210..db78d7c01b9a 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -62,10 +62,7 @@ enum zram_pageflags { /* Allocated for each disk page */ struct zram_table_entry { - union { - unsigned long handle; - unsigned long element; - }; + unsigned long handle; unsigned int flags; spinlock_t lock; #ifdef CONFIG_ZRAM_TRACK_ENTRY_ACTIME From patchwork Wed Dec 18 06:34:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13913086 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64F09E77187 for ; Wed, 18 Dec 2024 06:35:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EB9236B008C; Wed, 18 Dec 2024 01:35:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E69926B0092; Wed, 18 Dec 2024 01:35:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D31746B0093; Wed, 18 Dec 2024 01:35:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B50706B008C for ; Wed, 18 Dec 2024 01:35:34 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 63E24C0B94 for ; Wed, 18 Dec 2024 06:35:34 +0000 (UTC) X-FDA: 82907116740.09.BBAD09D Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf03.hostedemail.com (Postfix) with ESMTP id 43B8320008 for ; Wed, 18 Dec 2024 06:35:18 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=fN0mqwms; spf=pass (imf03.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.53 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734503697; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=K01D9WjQ4IDKLASlrrrpT/PTwcowLsveSV75IjoNXbE=; b=F54MOh8WbvT7TQk4sTpKq4CjegOlI/rKnwKTcD5zciP+B0BQAI6bMnX/H1MSL92a6/kez0 VZy4rtd6+Ze90aaFR0iQv8SSBy7BlpI4Gtnm8W1PPzKLA5ZePxeLoiE97d8vScwCkMc+YH RDHZNBFZi3VdD+ldZL/USxaphascT0g= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734503697; a=rsa-sha256; cv=none; b=Hf7VU7kiVcASZ0xpdUi/VLth3VSbqS9SFIIRacFTlCl+abUtFkcc4TaOs7KQc400QhrQme 50aJVMelWDIfAyuPMg2kP9BWNBoQD1rigUiiUZBPaVg9jevEGdByffYeGXMpnN35lc1dGS kUfX6o/rUUypqDx8r+pmerSX4e3n0Qw= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=fN0mqwms; spf=pass (imf03.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.53 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pj1-f53.google.com with SMTP id 98e67ed59e1d1-2ee51f8c47dso4243416a91.1 for ; Tue, 17 Dec 2024 22:35:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1734503731; x=1735108531; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=K01D9WjQ4IDKLASlrrrpT/PTwcowLsveSV75IjoNXbE=; b=fN0mqwms+8GnMJ5W3jDzL4UnBCYh3x33WpQL7Q4uBQooBjy2ObRQA+tIVaxHSLVhvX VqBiXsXNNZ3NeTdOz66tNWMIJQGfDYDBvZWL3P9roTFY3/Tlcl6JNqy985H4nsqCa890 exyopAl7WNUqwQ9NtolS1BV+QlSzoWj7/W14E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734503731; x=1735108531; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=K01D9WjQ4IDKLASlrrrpT/PTwcowLsveSV75IjoNXbE=; b=R4MuyK7/uV0NFuqtdyFQ1X74nymrG9WAeu4ezR6jUYG76Nn4Y64R3szttOlCE4xiJ1 vL1L0eDRqZn9YYpvsZFEmL/QCLiVDZMAhayWktL1ottai1ijNy5lxYyMuaZ3i6cH9MFV uvvm8G+RybrLg0pxsxLkxI/BXO1aGuXw2NEVhSmActMB4ExuwazP97Uq8hdyEL9m15KO zRmxfqwwrU5DpGGIwe4ySjVCrQ3rpdxhxXyeu3+1CxoEUyp3dLhsOmYmfwcbiaZSJcWL eEqNINbxv0UD7PUJS+4zN66anzpO6UJBUxIm5WDesVfoM/5tDGq7zccQ20vN4P/RTo3R kX4Q== X-Forwarded-Encrypted: i=1; AJvYcCX4XwHuHU1wmDWwM1rIqWPPyP+SKkMZBCbbf43EjzA56LYJb1Dkpc1iMJvek6y3m7QKlbpZYa8zvA==@kvack.org X-Gm-Message-State: AOJu0YzKQ3zIp4oJhgrLmJZPfD0ZQTHMhfXDNQVY1uIG1OH4fQdWMFaJ 0aK9WJuOFcJY4MZpcmUT658EYHpkh1l/96a02q1CXfSlKRDk0g/QQVUv3iiB6w== X-Gm-Gg: ASbGnctE8X6e/9l+iHa85HkXUa4hRWl21XF2Q25Qt30hddE5zdK2SbdJw+NV658FYVb ov+3cPmae210AhD//I4hPGLz3EbPfTrOvTarZvdeaU1mbxepzaVbvy7SviKZ2+L31aesjcSs3Q0 lUFLmphNQvAdfRLP5j8yaz2stPaeC9cgVbOpVpK/T8XmKthrvPUjqyDDo9EPwda5KHLEJEl2n9y L4Y9zrOQIaqQymk3Y94vkVWC2gabZTIU9BrwkQZ6qCP2IDBPr0zsdJqLBQ= X-Google-Smtp-Source: AGHT+IFK5WlahiR5aGL0BTCtzr0kzih6dT4LaJYbmSnHINvLQ21PzUVu87JBr7NRGop1fXLHbUC5yg== X-Received: by 2002:a17:90b:2703:b0:2ee:ed1c:e451 with SMTP id 98e67ed59e1d1-2f2e91db281mr2864827a91.15.1734503731353; Tue, 17 Dec 2024 22:35:31 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:3bcc:36cc:b9fe:9379]) by smtp.gmail.com with UTF8SMTPSA id 98e67ed59e1d1-2f2ed52c3bdsm587770a91.10.2024.12.17.22.35.29 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Dec 2024 22:35:30 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 3/7] zram: factor out ZRAM_SAME write Date: Wed, 18 Dec 2024 15:34:20 +0900 Message-ID: <20241218063513.297475-4-senozhatsky@chromium.org> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog In-Reply-To: <20241218063513.297475-1-senozhatsky@chromium.org> References: <20241218063513.297475-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 43B8320008 X-Rspam-User: X-Rspamd-Server: rspam07 X-Stat-Signature: p4jng5sxredggg1m9wr4m3pwb39oqmob X-HE-Tag: 1734503718-485882 X-HE-Meta: U2FsdGVkX1/n172lR/xDZtMp9wQSWgWwlLVqwOdpCsK9ehEb1xNc7knEgvi6PDp1Lx9mWwjXgKmTrfykhzTE9X0wclyB1TFXQRZ9VTfue9/4VjO/UweyWk8X5Cdd+OxSxGgXzdMYqMj80s6vPP+j90UC67mJDdevHUr9+FOLw3ghxkmrjeBTWH2Qn74/8iYTV4vyvcApG7/cWREQZi89leGtST/PLQ3GyrTjmrjuuRDU5uSYkCg/9B4pBDu/YkWABBNnoYGG2dbhKmS5zUMx3KHF4McQPJ5AknFPnnhoC5kuyNoM+gD8FrrumDBqPqI6ESWvT9oecgQm+aug0qm57HDXao7U+QUF5xW55OVod7J1Coan6hc7dPYFXxPBf0u4FOvdbaSyFsA98fE8u0hH38IW37UQV3j/s+TVYajctYyw5AAAIDneX/fsaa192bman48sQ78Xk/Az8FpT7mTjs1IFxZljLEA+Xz3QTj9MwmmIaF/3L5DzD2/Fsq/eW2LfgcQl+H6jFLt0zGZi9cL28IohkB5yJw7MFDcQ8lHxQRANA/QV2RbSRxs+xkJaVdS9I6A0pjGhwutDXxr7hG+dTqqczqF6YrUhuNXEXhvmngqf/NQm9YS5IRdu2aM8eYHbgi58dWxJle27j+xfAbzSt75CI9of2UuLdgHExa5DaQLosEZshHyRPLp96B1/fHV76WNJZRdFDHlqA7lX+OEE7M3bHBgI/rrAsSe2oc3iPu1vrf9B/rxpGwIw4nVt2s4EjRqbG+IzIgeE7kDWzLWorLKOXphc2g9FaIHGRaIQJXYTfACYH4HJYtFoz6glMrRuL24mLu9mrYkzzYpiSK/Jg00GLOLBbqIqGEMv9kzcwR34lx8ZHCsByb/uoAmMq//yY4XrSSKcHE38e8ZEPKba8rRcdFXLPjbHZGoByaZ5x1bTgQxz+t6vKhaalABuqJRCgZVQE1v/dWYVgmsfbd3 ocrl9Z4k VKQzDze/y2PHiuC495VEVJLgtg58YY53r8F2uyjwynRg7R/fbXFn1FlferuxaG+fUyNqHC1Q8W/FYZBafM5piI86domHw1GTUzeXiKgmy1h0TT1/fFFWlVmH/ogj2ZFyN5uNno971pfmTWzk3bQt1tTd55N/3JvsEyi+31NPiCOGjKQSlFuvd7Snhk2+FmBDFdFWgu5ZZuv9pOfsfwbeiiEkc/E7GimJJlwF1S6ZUJ8OSzehiL6uwX0mt1WnCZrGRgBzdzx5ZiVaeHvjSt1/upiTTiDdZEr57mLeD3TkM4b9Pu/4Qcd+cpGszyA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.033798, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Handling of ZRAM_SAME now uses a goto to the final stages of zram_write_page() plus it introduces a branch and flags variable, which is not making the code any simpler. In reality, we can handle ZRAM_SAME immediately when we detect such pages and remove a goto and a branch. Factor out ZRAM_SAME handling into a separate routine to simplify zram_write_page(). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 37 ++++++++++++++++++++--------------- 1 file changed, 21 insertions(+), 16 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 8c71ddd17024..89f3aaa23329 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1624,6 +1624,20 @@ static int zram_bvec_read(struct zram *zram, struct bio_vec *bvec, return zram_read_page(zram, bvec->bv_page, index, bio); } +static int write_same_filled_page(struct zram *zram, unsigned long fill, + u32 index) +{ + zram_slot_lock(zram, index); + zram_set_flag(zram, index, ZRAM_SAME); + zram_set_handle(zram, index, fill); + zram_slot_unlock(zram, index); + + atomic64_inc(&zram->stats.same_pages); + atomic64_inc(&zram->stats.pages_stored); + + return 0; +} + static int zram_write_page(struct zram *zram, struct page *page, u32 index) { int ret = 0; @@ -1633,7 +1647,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) void *src, *dst, *mem; struct zcomp_strm *zstrm; unsigned long element = 0; - enum zram_pageflags flags = 0; + bool same_filled; /* First, free memory allocated to this slot (if any) */ zram_slot_lock(zram, index); @@ -1641,14 +1655,10 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) zram_slot_unlock(zram, index); mem = kmap_local_page(page); - if (page_same_filled(mem, &element)) { - kunmap_local(mem); - /* Free memory associated with this sector now. */ - flags = ZRAM_SAME; - atomic64_inc(&zram->stats.same_pages); - goto out; - } + same_filled = page_same_filled(mem, &element); kunmap_local(mem); + if (same_filled) + return write_same_filled_page(zram, element, index); compress_again: zstrm = zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); @@ -1727,7 +1737,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); zs_unmap_object(zram->mem_pool, handle); atomic64_add(comp_len, &zram->stats.compr_data_size); -out: + zram_slot_lock(zram, index); if (comp_len == PAGE_SIZE) { zram_set_flag(zram, index, ZRAM_HUGE); @@ -1735,13 +1745,8 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) atomic64_inc(&zram->stats.huge_pages_since); } - if (flags) { - zram_set_flag(zram, index, flags); - zram_set_handle(zram, index, element); - } else { - zram_set_handle(zram, index, handle); - zram_set_obj_size(zram, index, comp_len); - } + zram_set_handle(zram, index, handle); + zram_set_obj_size(zram, index, comp_len); zram_slot_unlock(zram, index); /* Update stats */ From patchwork Wed Dec 18 06:34:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13913087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6B9BE77187 for ; Wed, 18 Dec 2024 06:35:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6F1546B0093; Wed, 18 Dec 2024 01:35:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A2B66B0095; Wed, 18 Dec 2024 01:35:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4F4096B0096; Wed, 18 Dec 2024 01:35:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2D9626B0093 for ; Wed, 18 Dec 2024 01:35:39 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A7308140C11 for ; Wed, 18 Dec 2024 06:35:38 +0000 (UTC) X-FDA: 82907118420.02.1F63D96 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by imf22.hostedemail.com (Postfix) with ESMTP id C6AECC0004 for ; Wed, 18 Dec 2024 06:35:03 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=MA7MSsnx; spf=pass (imf22.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.210.174 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734503702; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kcFgQDfUGNa4RpmE9ZvQmkln//OMWOItv9XoEUwGa60=; b=iHbFUREp2eqK3dVy0YeArXS65H6+nKky4t7uql8ea2CnxcJZp2N1/CdnY5PEJ+kB9jXm3R T8uZcO/kQkBVCudn7bfavnfZ5NmRtirCmNMKFZAWzR9JljGK5GefnbKK2Ygonejr7EelOU tfTDoZ/D/1tdb0XyEUR6c2Ultl1ODf4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734503702; a=rsa-sha256; cv=none; b=kOIBu/AOeUu+OOQdopceOn+WoTGoIIfKNm4Sd6QDKGNoKcJKWvYzJ2aa0gTeKeOT25Wmxg dko7A0eOpk++vkpPxRQs4W95SAYlKUBFhRkDxLvDk1AQZA43zbrpYYWYmklY67SCYAHu08 Aw4IOoMHvbUQxXhlkMF8FnudSWQZUTw= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=MA7MSsnx; spf=pass (imf22.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.210.174 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-728e1799d95so7169488b3a.2 for ; Tue, 17 Dec 2024 22:35:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1734503735; x=1735108535; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kcFgQDfUGNa4RpmE9ZvQmkln//OMWOItv9XoEUwGa60=; b=MA7MSsnxr+GEwGVOoxvMO0EVnGXmct8aR8V4GehHAyoON3C0OHj41x6l1aT27xIzLU 5JFkfh9xQJPmRAu71Ag7dqP5/h+wULJBbA2r02wXNh+jhQwiJI67+uLKOstH1r/W1gYV eOSYL4ZVhfHo6epg35q5x//EyodnFGXfQPlu4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734503735; x=1735108535; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kcFgQDfUGNa4RpmE9ZvQmkln//OMWOItv9XoEUwGa60=; b=e1kMdUGDk/1nY2i/iDxDCoxuwvAJwLIbbHFOw1z/8FYoN5M5NFAhuggxG8H7wt7RBN McAg2af2xIBSiFFZNQOZaRelEzj3Ei1Tu3aYUs7TRzklIyK+I4L1SMwwy+gD2X8lBuA8 URh2+zJWr3JLAJUOjp9KPr0IkfW8XyEnpte0DeIXVZjOwN6WGqxNH0/Oxu+4+oKFHOLD 95qEaOlAHrP15E1ptim0uTDvGNIqJ4WoqPTSJHIx/DptFYerOZvK8VIYqtc62FlSdETY on5i8c0+T899mUgwmrVIPIsl8oUkrBXzV/mRpXEKNyBA5awmPeJBy0KZoOO8v7a5bNfD ANMw== X-Forwarded-Encrypted: i=1; AJvYcCViM13rkXZ30Deb4D8b0vPQYY5nWsnqEpXu6zV0AxUmRB8nPRx7JdOlN4PcnIiQ+5Ap4O52bQDbhQ==@kvack.org X-Gm-Message-State: AOJu0YwIwlFhUeR4q6T/OTKm6ucbpTH6I49ssOczJ83vD3x/IayP3D5v XQ6eJaj5t7Y6yxcCbxhDbvuElncHuSzY+/pV11Rd4aP5gbVjK8Qy/RD7RCBRC4JMuXESob4o3sU = X-Gm-Gg: ASbGnctk8CB3TLqKeOcBu+RTgS3jk3RO6JnlCeABuqrW9rdtXmzxSCYF7/yw5T0JRGp tMdvWJcZPaersZFCipVUhW9t12Ak9nKF6AO6ldpfjuoQ7p3JYHioQcv5zIpSSDcckEMmUYZaQwD mjz2hzqiR2aZMNAoIUCnxlzfrIfjVUozPZkKdIPpj6gPPEdEkNyTWhBUoUKM7qDsnmQmbQVgciS Uy2P1Z5kDjc9xSIrAfx8ScMtkImES3eLOrrwRupfSazp3xlf1quqjTXLXU= X-Google-Smtp-Source: AGHT+IH4+dvzwEYz6RMzf3+gGHyop3pBmEqsMorp12eUh0yPbMZK6wxgVt40cdSmpIiL6/jj76oo3w== X-Received: by 2002:a05:6a20:3d88:b0:1e1:b023:6c98 with SMTP id adf61e73a8af0-1e5b482147emr2917670637.26.1734503735547; Tue, 17 Dec 2024 22:35:35 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:3bcc:36cc:b9fe:9379]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72918b77335sm7690904b3a.118.2024.12.17.22.35.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Dec 2024 22:35:35 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 4/7] zram: factor out ZRAM_HUGE write Date: Wed, 18 Dec 2024 15:34:21 +0900 Message-ID: <20241218063513.297475-5-senozhatsky@chromium.org> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog In-Reply-To: <20241218063513.297475-1-senozhatsky@chromium.org> References: <20241218063513.297475-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: C6AECC0004 X-Rspam-User: X-Rspamd-Server: rspam07 X-Stat-Signature: mtu1fn9a711fwfm39zuyc4krnjgt53w8 X-HE-Tag: 1734503703-661414 X-HE-Meta: U2FsdGVkX1+n7BmmB6HNor5FQwNsG7MS4/AtxHCXSBSwhJ8P/OTvxvh9g/y9281lWHJH4DUIqw/AB3JtrpJNymsoLhGZ2pzAasAThEZjrsMjdUlnd62kwDuy/wtwjT2+W7CWFIRs5z+I4HPil3sCR3/NxgZuBW+O+R99yNjBmkArRU5/TjUcdlcap4tIIztFXGfB7U7uqPz5KROwuD3I82cIgjzJVmj1CX41RQhJ2noPh9jjbbGknlwXimNAx5hCYFmuA3QBXsKWO+cCZsxZVDtHo1ek4vjEzHH+iDIZryauhrk/gdpTyCgy+XPD7VSbbhCqngg7O+kGwKOmKqMSO270SXwyJQHRQ0tRLYireMrHJqooGpB7KOGZL0ZvVUL957hsSN64iRF16l/kronnvLSXBVbHw1fyjgD8C84rXk5yPMgPPuwJhl/n1G+uEHi3GDEN9PqM/xbif3DTCTkvJQEjNOnH6e3lz9L/G6+u2myzMCMwEaBz2MapiwIl05V0Pp6nXct9bFpXvAUyHEn0pyL2AM0sdfpLYH3hKbGjwLjuSeNx4oi7pFvTVT+8aYjIABCh57E+EIPJlTH0D/wwI9DIRxu5fRHjAPxbMe/aJiG/1n8tt00+TcG1+4s8HKjWOEyfmuuSvAk74q0tnqJy7j0YUobkCtjYFttAc4oPMT2vY2iceQPvlb5WY7sODpGMr4gDOlwMMY+/4cfl4dx7ADkkG+UFS6Ry9400KAhwEAt1qXIP7d3JjF2hEZl7+mdDLVkwj3s+acl0UlO2unpnLtunMSAGLXCLybTMCChi8CV3LhFBXTjPSCgR8fgUVTbpxKqL34Hc7zNtzRsu4EJd7GkYccoZwFWsfevQokv7gCEiifcMs5naP+ijv3oMX/K0jXSjEeOQ4mP0aWxLJegkDhiMCc9hRX+ip0Rjr49JmdNXf/57WmhtuzUnZ/zaN+ioii2hmL6Q4fckSvFy5Vi F0cwSJAn qRQWHj4U2O3w4Tv8yFTvke3ddTOTmVql39s2ISFeXCeAqQWcK/xX8ae+aChGMyDLsNk+aGi7/LA5C//5960bgslc0NpyAh6SHcSAD/oOkdnJG7uMQohKUwsrzSs+CrgAa0kxARXcVP5yX5hMDxekoJBHggTgTpwO6HPC12aLMefAPFG18ZVv1mA07V7LTUK2KieV2EBTT2zUcUVwedADpCddbo7psDjuxMBzUa31Q9OxJtPV3PpbrqbcaFBsSlPMFRe/QKjTRni0CumrbvOynbAPXMJfiq+i/AOtECTTuL7b6VtL74FdP8Ui5i8VzO2mKUucR X-Bogosity: Ham, tests=bogofilter, spamicity=0.000038, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: zram_write_page() handles: ZRAM_SAME pages (which was already factored out) stores, regular page stores and ZRAM_HUGE pages stores. ZRAM_HUGE handling adds a significant amount of complexity. Instead, we can handle ZRAM_HUGE in a separate function. This allows us to simplify zs_handle allocations slow-path, as it now does not handle ZRAM_HUGE case. ZRAM_HUGE zs_handle allocation, on the other hand, can now drop __GFP_KSWAPD_RECLAIM because we handle ZRAM_HUGE in preemptible context (outside of local-lock scope). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 136 +++++++++++++++++++++------------- 1 file changed, 83 insertions(+), 53 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 89f3aaa23329..1339776bc6c5 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -132,6 +132,27 @@ static inline bool zram_allocated(struct zram *zram, u32 index) zram_test_flag(zram, index, ZRAM_WB); } +static inline void update_used_max(struct zram *zram, const unsigned long pages) +{ + unsigned long cur_max = atomic_long_read(&zram->stats.max_used_pages); + + do { + if (cur_max >= pages) + return; + } while (!atomic_long_try_cmpxchg(&zram->stats.max_used_pages, + &cur_max, pages)); +} + +static bool zram_can_store_page(struct zram *zram) +{ + unsigned long alloced_pages; + + alloced_pages = zs_get_total_pages(zram->mem_pool); + update_used_max(zram, alloced_pages); + + return !zram->limit_pages || alloced_pages <= zram->limit_pages; +} + #if PAGE_SIZE != 4096 static inline bool is_partial_io(struct bio_vec *bvec) { @@ -266,18 +287,6 @@ static struct zram_pp_slot *select_pp_slot(struct zram_pp_ctl *ctl) } #endif -static inline void update_used_max(struct zram *zram, - const unsigned long pages) -{ - unsigned long cur_max = atomic_long_read(&zram->stats.max_used_pages); - - do { - if (cur_max >= pages) - return; - } while (!atomic_long_try_cmpxchg(&zram->stats.max_used_pages, - &cur_max, pages)); -} - static inline void zram_fill_page(void *ptr, unsigned long len, unsigned long value) { @@ -1638,13 +1647,54 @@ static int write_same_filled_page(struct zram *zram, unsigned long fill, return 0; } +static int write_incompressible_page(struct zram *zram, struct page *page, + u32 index) +{ + unsigned long handle; + void *src, *dst; + + /* + * This function is called from preemptible context so we don't need + * to do optimistic and fallback to pessimistic handle allocation, + * like we do for compressible pages. + */ + handle = zs_malloc(zram->mem_pool, PAGE_SIZE, + GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); + if (IS_ERR_VALUE(handle)) + return PTR_ERR((void *)handle); + + if (!zram_can_store_page(zram)) { + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zs_free(zram->mem_pool, handle); + return -ENOMEM; + } + + dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); + src = kmap_local_page(page); + memcpy(dst, src, PAGE_SIZE); + kunmap_local(src); + zs_unmap_object(zram->mem_pool, handle); + + zram_slot_lock(zram, index); + zram_set_flag(zram, index, ZRAM_HUGE); + zram_set_handle(zram, index, handle); + zram_set_obj_size(zram, index, PAGE_SIZE); + zram_slot_unlock(zram, index); + + atomic64_add(PAGE_SIZE, &zram->stats.compr_data_size); + atomic64_inc(&zram->stats.huge_pages); + atomic64_inc(&zram->stats.huge_pages_since); + atomic64_inc(&zram->stats.pages_stored); + + return 0; +} + static int zram_write_page(struct zram *zram, struct page *page, u32 index) { int ret = 0; - unsigned long alloced_pages; unsigned long handle = -ENOMEM; unsigned int comp_len = 0; - void *src, *dst, *mem; + void *dst, *mem; struct zcomp_strm *zstrm; unsigned long element = 0; bool same_filled; @@ -1662,10 +1712,10 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) compress_again: zstrm = zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); - src = kmap_local_page(page); + mem = kmap_local_page(page); ret = zcomp_compress(zram->comps[ZRAM_PRIMARY_COMP], zstrm, - src, &comp_len); - kunmap_local(src); + mem, &comp_len); + kunmap_local(mem); if (unlikely(ret)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); @@ -1674,8 +1724,11 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return ret; } - if (comp_len >= huge_class_size) - comp_len = PAGE_SIZE; + if (comp_len >= huge_class_size) { + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + return write_incompressible_page(zram, page, index); + } + /* * handle allocation has 2 paths: * a) fast path is executed with preemption disabled (for @@ -1691,35 +1744,23 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) */ if (IS_ERR_VALUE(handle)) handle = zs_malloc(zram->mem_pool, comp_len, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); + __GFP_KSWAPD_RECLAIM | + __GFP_NOWARN | + __GFP_HIGHMEM | + __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); atomic64_inc(&zram->stats.writestall); handle = zs_malloc(zram->mem_pool, comp_len, - GFP_NOIO | __GFP_HIGHMEM | - __GFP_MOVABLE); + GFP_NOIO | __GFP_HIGHMEM | + __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) return PTR_ERR((void *)handle); - if (comp_len != PAGE_SIZE) - goto compress_again; - /* - * If the page is not compressible, you need to acquire the - * lock and execute the code below. The zcomp_stream_get() - * call is needed to disable the cpu hotplug and grab the - * zstrm buffer back. It is necessary that the dereferencing - * of the zstrm variable below occurs correctly. - */ - zstrm = zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); + goto compress_again; } - alloced_pages = zs_get_total_pages(zram->mem_pool); - update_used_max(zram, alloced_pages); - - if (zram->limit_pages && alloced_pages > zram->limit_pages) { + if (!zram_can_store_page(zram)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); zs_free(zram->mem_pool, handle); return -ENOMEM; @@ -1727,30 +1768,19 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); - src = zstrm->buffer; - if (comp_len == PAGE_SIZE) - src = kmap_local_page(page); - memcpy(dst, src, comp_len); - if (comp_len == PAGE_SIZE) - kunmap_local(src); - + memcpy(dst, zstrm->buffer, comp_len); zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); zs_unmap_object(zram->mem_pool, handle); - atomic64_add(comp_len, &zram->stats.compr_data_size); zram_slot_lock(zram, index); - if (comp_len == PAGE_SIZE) { - zram_set_flag(zram, index, ZRAM_HUGE); - atomic64_inc(&zram->stats.huge_pages); - atomic64_inc(&zram->stats.huge_pages_since); - } - zram_set_handle(zram, index, handle); zram_set_obj_size(zram, index, comp_len); zram_slot_unlock(zram, index); /* Update stats */ atomic64_inc(&zram->stats.pages_stored); + atomic64_add(comp_len, &zram->stats.compr_data_size); + return ret; } From patchwork Wed Dec 18 06:34:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13913088 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7892E77187 for ; Wed, 18 Dec 2024 06:35:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4A96F6B0096; Wed, 18 Dec 2024 01:35:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 457EA6B0098; Wed, 18 Dec 2024 01:35:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2F83F6B0099; Wed, 18 Dec 2024 01:35:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 076F06B0096 for ; Wed, 18 Dec 2024 01:35:43 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id B274A80BE5 for ; Wed, 18 Dec 2024 06:35:42 +0000 (UTC) X-FDA: 82907118630.14.484708D Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf23.hostedemail.com (Postfix) with ESMTP id DDE4D140010 for ; Wed, 18 Dec 2024 06:35:19 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Gvh0PhwP; spf=pass (imf23.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.210.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734503719; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8UlIfnWhj8akIUQw35UgTdzrn/T1WzL/Vbski8Obleg=; b=wwiF8pLnnWP2m2RD3CuhZixLO5wR6WSZR8ldrgEpd4pKT7R+p1HICVgxtAn7fBqWqrq2kV Vgd1Cp3cOaLNVvGbeW3HBh+ThgmdDlB8aTNMXANhhSqV/sDdnrS4dckpEeWiv6f7p8Di7g JBhxvAOrTrOh/HX5pxKNDUkT8/Toil4= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=Gvh0PhwP; spf=pass (imf23.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.210.169 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734503719; a=rsa-sha256; cv=none; b=vOANWh6QOstFEZdKL6ECjIgqvf4xl8O/Hn//vmAOd1wrbw5NbA58JADmTCVlPuSyVUMIVQ oh5JPgHe3WKZXUvNQa5z8EqcEk723c7gkbPENIDkofVxz4IFhTXBzYlj4wHBuwRgZixllp OAVaoz4/cHI188U2fHgMmLx737ACbSY= Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-728ea1573c0so5345057b3a.0 for ; Tue, 17 Dec 2024 22:35:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1734503739; x=1735108539; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8UlIfnWhj8akIUQw35UgTdzrn/T1WzL/Vbski8Obleg=; b=Gvh0PhwPEPSIb3+40GIE8KIxdiYKaRhlqoxtHxooXgHD2ZhRpV1FMzqbbNjFSuBOwN VkYXiVvIUtsk8N7wZJp4t64INzPm0rHU1BACdkgxP+RfRZK73SBxbDQekGAiEjfsgOvi +VNGVcz6UpAfeJtSVgww/IPP0IMFMyTKYka2c= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734503739; x=1735108539; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8UlIfnWhj8akIUQw35UgTdzrn/T1WzL/Vbski8Obleg=; b=T1Q7oY6HlCziuMbjuCYbYAjOzlVSh0dwSExjcZPCdR/4XiJ6Zb4lP/QgyLqmXwaFxn L8T8BJ0mZSqF9qBJMwqemZqnGHYVG1KPZLZTsnycVtuf4rJYpmHqch0dzuLr18PcN1nN ok6fozz1bQVfStsqPxXvEvvZOAWQKnpgx/41/QYcka0lyl1ZKant5TdNsV/NgdlOq4dg /D30hKkTz5JsQFvWPLLYI79Bm51ArvQL31iMLm3T2fp15+s5rCfvx779NUzrpEDnjMHR RVYZLr3ukc3vM/yQf23rhFxztt/SuZmP3D+jpXReMDlTOzzcox4+SbfDfKlrIhKzylEf FSwA== X-Forwarded-Encrypted: i=1; AJvYcCXGeD5LLo6kcfE412lYEAjO8lrswCGGGEUkdSRvsYVbfCtAq2Nsmuda1lIC19tlGjJBRJXkx0W2ag==@kvack.org X-Gm-Message-State: AOJu0YzEQn9+tCeGJPnVRDkG0U4fh7vTW6mZKL29229Ooi19Ra2p6riB mRBXWdbdXS1+QZfFTY8JEq6tFsHK7ITl/C+KK+k6bQv/kUqn6qGQvFe44+EJOQ== X-Gm-Gg: ASbGncsg+vON74v0Dbatf2APkTpwIgd+n41GfdB6a6OB8Nn+yMJ2RW/QKKU9Ddy3y0y O0rpvuri+DvMlsqfdjynlwNOViFRHjxI3e89+/ySG9dQzPOzTRHKB/TkNvJcD7z5zq+6K6N9Yuh OGYa0/Jbe36VC/uPbHIUcus4dV1W9LX5xrzEPkYskG30imd/oRe62vPTQm9ZegbKmfzaJyA1Ve5 1zRRV6NZaG3/QwIfJROKY/9P5FFaosHJY9dIaIkF6+OvBwDS+ClSPMV6nQ= X-Google-Smtp-Source: AGHT+IGwQ0wQAXBUGI22Q3J+XJvZdIgvdCXo+/PUjQeIZFjzJ+5utt6loeABpMh/C7JssxnEtaeHvw== X-Received: by 2002:a05:6a20:3948:b0:1e1:aef4:9ce8 with SMTP id adf61e73a8af0-1e5b48a15a5mr2391517637.28.1734503739653; Tue, 17 Dec 2024 22:35:39 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:3bcc:36cc:b9fe:9379]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72918bad760sm7760879b3a.152.2024.12.17.22.35.38 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Dec 2024 22:35:39 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 5/7] zram: factor out different page types read Date: Wed, 18 Dec 2024 15:34:22 +0900 Message-ID: <20241218063513.297475-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog In-Reply-To: <20241218063513.297475-1-senozhatsky@chromium.org> References: <20241218063513.297475-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Stat-Signature: 9o9ycmrttre8sj77yd9wykm8txtsqqis X-Rspamd-Queue-Id: DDE4D140010 X-Rspam-User: X-HE-Tag: 1734503719-16203 X-HE-Meta: U2FsdGVkX18MRchNNdWytUfAZSCzz6ecH9GxdWbBfRtmYDiKG66ScDoStT16er6m2hPCqD0tCgPg0jIQXNyX20HLGT4YPd7gf7iVrTxbLdFi3YADFjwchkCFCzzrP5YnQ90nLxha1KQX+K7HHX/NdcsDLh+stDBB6vXx1Pbvg148Wxk6U6X/FqAxb+2tnFzIOHfxJn7EP6cw3jXtFkSJ4EJ1Uvs8NfbGQmBvlBO+sjBVrOlePXmD7fhPVrwiOyq+NL96Uk91QgXl/3K1lvXcfrWMtseLypp1KDapv7G+0O6997L1LQ+on4/8wMRqNrpBeQSeVOZK9t3vpY/nCcsgAKcxj6Dvt7pQClwTwbD0qWfi7bS3BowSkr177c7E8GtQB0ZkvjfjzO4g4jrXiZhU2o/0GdBhzvruCW7aJItGD4cOmaVkmWGrCG2XvvjtmnZRW2NdQCbKlzdZXLZ2hea9iikrKmPP1ES6TWFHloVw+w56OL/PTSLs1oVwYhYoHINd13rkHhjYJdU4G3gnmjeEUb9pMHTtnc0RS/Ug5G10N9Z0fug3r+mRSkHebX9sUnupR2aHhkNcg/Z5f7wsg9vX6ubOUxtYWgq5/m0whV4AGWVAqIIskEjeF1v7hvosSIQhw4oBwOqzQkG5POkxgj1BGeL8lwc8DIlARzOym5RFKtChSZ6psVmjmNS/iPWZg6AusR2S2oV9ngJi2dpHYprCqA6ITUezZJ6hWxGnFsk8bFowH2qvoLfoTEav/IdiiEm5NBLihBUCrX7ixY+qz476NvDdMYAN0JKLafi3H73uqniadrirCNgmCwyKL/z+0jRhhutvkMLMkTnKqUTG+yagPOE6GqW/0djtJM59lmfbTydxclUEbNw1dBD2JeVRH3AvK8XO4I/EArFLtvrb709niVYexp83eCKRYqWUoD+UUesmUUslXcCMJnz5kI4BXk4NSUAgseOc+mvHxhjO0wf VbD32ALN Zf8QgTGcogiCshtuH9vs55BtyuRzBT2XJCq2JzFjXzRCy7WYSAr/ugUYWjdUZ6iY8NBc4xYq+wO84XeAzGC1fV3BOxAAggBNOfbnelRmd+USS+6j0jmyW57IkX04cYaqW16mP/ASE80e4CO5Hd0wZC+0Id/MkqjtwEEnAw0uWT23lsFRdGYrZTRFe7uc39R5HTmFNBlAqbfk5boJ/zWtX6KUuDCK7HcTTcLIlMFhAV11TLRUIlTwvPwooV/edWCa/8Vlnx/nubEHCDNI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.222401, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Similarly to write, split the page read code into ZRAM_HUGE read, ZRAM_SAME read and compressed page read to simplify the code. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 85 +++++++++++++++++++++-------------- 1 file changed, 52 insertions(+), 33 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 1339776bc6c5..465297d31bdf 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1530,54 +1530,73 @@ static void zram_free_page(struct zram *zram, size_t index) zram_set_obj_size(zram, index, 0); } -/* - * Reads (decompresses if needed) a page from zspool (zsmalloc). - * Corresponding ZRAM slot should be locked. - */ -static int zram_read_from_zspool(struct zram *zram, struct page *page, +static int read_same_filled_page(struct zram *zram, struct page *page, u32 index) { - struct zcomp_strm *zstrm; + void *mem; + + mem = kmap_local_page(page); + zram_fill_page(mem, PAGE_SIZE, zram_get_handle(zram, index)); + kunmap_local(mem); + return 0; +} + +static int read_incompressible_page(struct zram *zram, struct page *page, + u32 index) +{ unsigned long handle; - unsigned int size; void *src, *dst; - u32 prio; - int ret; handle = zram_get_handle(zram, index); - if (!handle || zram_test_flag(zram, index, ZRAM_SAME)) { - void *mem; + src = zs_map_object(zram->mem_pool, handle, ZS_MM_RO); + dst = kmap_local_page(page); + copy_page(dst, src); + kunmap_local(dst); + zs_unmap_object(zram->mem_pool, handle); - mem = kmap_local_page(page); - zram_fill_page(mem, PAGE_SIZE, handle); - kunmap_local(mem); - return 0; - } + return 0; +} - size = zram_get_obj_size(zram, index); +static int read_compressed_page(struct zram *zram, struct page *page, u32 index) +{ + struct zcomp_strm *zstrm; + unsigned long handle; + unsigned int size; + void *src, *dst; + int ret, prio; - if (size != PAGE_SIZE) { - prio = zram_get_priority(zram, index); - zstrm = zcomp_stream_get(zram->comps[prio]); - } + handle = zram_get_handle(zram, index); + size = zram_get_obj_size(zram, index); + prio = zram_get_priority(zram, index); + zstrm = zcomp_stream_get(zram->comps[prio]); src = zs_map_object(zram->mem_pool, handle, ZS_MM_RO); - if (size == PAGE_SIZE) { - dst = kmap_local_page(page); - copy_page(dst, src); - kunmap_local(dst); - ret = 0; - } else { - dst = kmap_local_page(page); - ret = zcomp_decompress(zram->comps[prio], zstrm, - src, size, dst); - kunmap_local(dst); - zcomp_stream_put(zram->comps[prio]); - } + dst = kmap_local_page(page); + ret = zcomp_decompress(zram->comps[prio], zstrm, src, size, dst); + kunmap_local(dst); zs_unmap_object(zram->mem_pool, handle); + zcomp_stream_put(zram->comps[prio]); + return ret; } +/* + * Reads (decompresses if needed) a page from zspool (zsmalloc). + * Corresponding ZRAM slot should be locked. + */ +static int zram_read_from_zspool(struct zram *zram, struct page *page, + u32 index) +{ + if (zram_test_flag(zram, index, ZRAM_SAME) || + !zram_get_handle(zram, index)) + return read_same_filled_page(zram, page, index); + + if (!zram_test_flag(zram, index, ZRAM_HUGE)) + return read_compressed_page(zram, page, index); + else + return read_incompressible_page(zram, page, index); +} + static int zram_read_page(struct zram *zram, struct page *page, u32 index, struct bio *parent) { From patchwork Wed Dec 18 06:34:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13913089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45FEBE77187 for ; Wed, 18 Dec 2024 06:35:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CBE856B0099; Wed, 18 Dec 2024 01:35:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C6E846B009A; Wed, 18 Dec 2024 01:35:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0FBC6B009B; Wed, 18 Dec 2024 01:35:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 910AB6B0099 for ; Wed, 18 Dec 2024 01:35:46 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 45DE580C1E for ; Wed, 18 Dec 2024 06:35:46 +0000 (UTC) X-FDA: 82907118378.05.44DB678 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf13.hostedemail.com (Postfix) with ESMTP id C188A20009 for ; Wed, 18 Dec 2024 06:35:13 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=SIuRQ8al; spf=pass (imf13.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.210.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734503729; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Jx9bIgiEIqkEeB+/C7YzsZikx7IBwIxJmTKfYS6wvFk=; b=XxbLzw6JQRkQFQD1QkAgXrdHEB+9zVhO/3NQ/reYiCSANdkCjNltnozvEck7b02WxuSrhz K7Mylujkj2SADcfggPKH4CliIdwTPCJgI6cnl3ls8iAyWo75bGxrsP8AXhoF4iP8XxRirc xBfec6O94h/03rL6I9MRKGb/sFA+U/E= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734503729; a=rsa-sha256; cv=none; b=cPKjL22fGlo228Ku65Mrlrqe2dqaU4XPovJfuHGNraa3adamO3CkClriEkfseN5RBe5/O5 X7GMBkskPgRVgqnheZAJr5fMCeP7tjavbVp2uyRiTG4OcNMmjQmPSnMCYBCragk5sEiKSj FPArDGU6tFIEJ1OwmXA40ttZVHaaq9I= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=SIuRQ8al; spf=pass (imf13.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.210.171 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-725ea1e19f0so5324440b3a.3 for ; Tue, 17 Dec 2024 22:35:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1734503743; x=1735108543; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Jx9bIgiEIqkEeB+/C7YzsZikx7IBwIxJmTKfYS6wvFk=; b=SIuRQ8alLjcJ7dl9CbMi0JGKVppllIoViecmAmEFjQuWktzaNWkvx1Dz/Y1IRvh6Ly qw0Vc2Ki/Y+D/qzs/5zmN0ZySynjCXCYyz34n449jLApnaYEH8vPJJW/QRKYa3C208gG HDJZxTiUcGF91j+e9dQqQkHHcafKXEvbNF+lA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734503743; x=1735108543; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jx9bIgiEIqkEeB+/C7YzsZikx7IBwIxJmTKfYS6wvFk=; b=NmHT8WedoC0PWmhEjIvE6Pl7YBanC7tc8tUTLp6MunWKRQQbkEGV/e8PfxBpScYO2V rMNrBqJ3v69AhRwcBAUux4hZFNij3iy1p5NlRW0+78elZzgEejiwODYtk7ROdRKLYQho 8D/mYJzEURfSdH4+RKKB/sCqaIDFmU4G4qPk6ymgL2Mxov5jT5n3RrI+xLTpUtLBvLn+ 4GUXfDHMxaE6R3wjOeCo50HoaJMa7D4XXYBALRTy9ANUyqxKBSYadlH6StAAiHnS9ldc R8IqWhQV7knK6MpMohukIlxgzoITN2c+igKIMNufpumq/dZcNW/DjIMvBDyUSvCR1bGk EEGA== X-Forwarded-Encrypted: i=1; AJvYcCUkow1fn0sZgAx2zPt3Kh/M/KgGEPTvlmC08xEdoqEYjyrY+Rb3rZciaV9C9tBrwQ8N0DulLVyWLw==@kvack.org X-Gm-Message-State: AOJu0YylHnBg7g6hy3i6+k9JNbdZGiXUo7oJGMeMYtjmpK66DRxKzXl9 S0o8QVRyiO72SC+w1pwygikqMXGY/Yfu+his8Kl0CDhosRTpotb8NtsWEV5J4w== X-Gm-Gg: ASbGncsEROmI6x+v6+j6Txpax7ti1+6mvbbLNvx2wCFc8SF1W2P+QHoA7lPQ7nu12A4 JXW97/PQR8siW2ss6EHqdBtVexdWHH9QXU1/jCh5yCLBDYCrEIZbnDqfvjhMUuOmQ03ePCtyfWe LJSepbR0b9ePeAa4kou7U22hwm3qgIemk5RjPr02ze89Erf5phbbenAXA+uqhvA8Jxhj6Uqdm7a RhzPIcEokLdRgODf7yDysETi+DLFiNpOD2/mIi4GCn9O4XuTjziY5Exckc= X-Google-Smtp-Source: AGHT+IFnXzThR3kYgLdl6KAfkV3e2DAw4nVMXA+afaVN1un8QDO+bjEptYoZDse8hnX1mt8BUslL8Q== X-Received: by 2002:aa7:888c:0:b0:725:f097:ed21 with SMTP id d2e1a72fcca58-72a8d23e809mr2464921b3a.15.1734503743201; Tue, 17 Dec 2024 22:35:43 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:3bcc:36cc:b9fe:9379]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-7291bca1069sm7446008b3a.114.2024.12.17.22.35.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Dec 2024 22:35:42 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 6/7] zram: use zram_read_from_zspool() in writeback Date: Wed, 18 Dec 2024 15:34:23 +0900 Message-ID: <20241218063513.297475-7-senozhatsky@chromium.org> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog In-Reply-To: <20241218063513.297475-1-senozhatsky@chromium.org> References: <20241218063513.297475-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: C188A20009 X-Stat-Signature: p5ctp7cyu9oauqmmcn93jed5hewaphem X-Rspam-User: X-HE-Tag: 1734503713-332113 X-HE-Meta: U2FsdGVkX1+3IlNedWgkPWUs4YXIi5zOC2FEAk+vWKSVU6Jqi3wHCEbD9z9UY9A3SEA7LRJq/fetdDPiPLdeq2M5MzY56efguAXfsqmPZAIa3WrepZnupy/HcMreOD1ebghyNhBn1dm1i0jds7JPq1Vbk+VDoFA2akc4i2qrz98ecdRueulXXAD+xs4IkYbN86eampBT3/OMUR//OttMPymofOA7D35K9gaKx1hyG4AC4+lcg9Wu8kcvqi4tAFIj7B813X+B55wzrVMGIwvu1ttHA4ZDdvpwg1HryRfWGosTXMb+1NeBHe7YaeBabkCJnAtaf+aQ0GMWMwC9Ry/NCaY4DtMMMsndHDV7o6NYFbZnTnHKD206oZtad9Kf3OXSMhC+wSgZcEEjZQdq1bX/WwK2wk/0AdMWFpvPT6bObjPLShjvdScf9lFmExG0X3+bTXf0fAhpO7r3VTwK78EcCyxpw7hKOGukhvsQhXBJdTEKI0mVaCA1gUrUnkvZDmx0lOY6fF7SPIkyi15pczPtyEZwxPg7rfk0n2k+oGVouItJDKoV7g8K7Vxq1ggzMOFsLOt1f8oEFl6Y4Wn/7SHbrM6cjZXy1W5UHP4zNtAUjmrKuqHVh0aqF31QshWqCIMMJabfIM5v2nvODNRZuMupMyfEWHsBnNl6eJTevDbeWjnlnHSDvQcVpPBqBXrwwTv4+SI38gKfiUVkmCIYPCqUf9x9vJAYkFPducG2FTbZgrLGzkgzw7ygNYVzfZa/KiqCFsFOSu9uWADeGza6LEU1tFsqqrCusvmd9C6DKYMVVs4Qtkvf49EL41CoKrI9rWP/MtjReXC9BImZ+bWY8F6B39Hhr7kFkALCyupetLo/5w1NYhUNjjtmWyHfAyEkGx3HAJmb92RK9WyZZ/J0oO0qx048eOE3lBqvxaqXUrOKNoLJ5ps/LUeJ0rtZ92Js/ejEoK5+/IvHeXrq3dYHRrw C/Qiirnh uDP6MP/XtVMVew/p48K/3O+A5OiE91hsAWH3gLcm7mBdX6AglQcq4qn2c8QdnBXBaxH43YwLpjU3v+8igxojW+RuBk45ApiMpteD5l62bMSNNy57iKnl80E6A7y3Yi7qnNmDFXIyoUhXXt2swbhrGt3q0XUL9L4MEIGt7eQa/XaB3yicgWfSB7C16yc5Kx1AMBf0V/qWITHO+cqqMnm516CIhWapSOUd5PIDwAl/+0qA2TnYXhdTEo2Zv6JW5B1f2m6uJbb6mO5t1OdoQCvnO1sjh/fy1R8kJta3eIixusxw/fZlRHpnQK9hrhg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.073721, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We only can read pages from zspool in writeback, zram_read_page() is not really right in that context not only because it's a more generic function that handles ZRAM_WB pages, but also because it requires us to unlock slot between slot flag check and actual page read. Use zram_read_from_zspool() instead and do slot flags check and page read under the same slot lock. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 465297d31bdf..7dd72b58e921 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -55,8 +55,8 @@ static size_t huge_class_size; static const struct block_device_operations zram_devops; static void zram_free_page(struct zram *zram, size_t index); -static int zram_read_page(struct zram *zram, struct page *page, u32 index, - struct bio *parent); +static int zram_read_from_zspool(struct zram *zram, struct page *page, + u32 index); static int zram_slot_trylock(struct zram *zram, u32 index) { @@ -831,13 +831,10 @@ static ssize_t writeback_store(struct device *dev, */ if (!zram_test_flag(zram, index, ZRAM_PP_SLOT)) goto next; + if (zram_read_from_zspool(zram, page, index)) + goto next; zram_slot_unlock(zram, index); - if (zram_read_page(zram, page, index, NULL)) { - release_pp_slot(zram, pps); - continue; - } - bio_init(&bio, zram->bdev, &bio_vec, 1, REQ_OP_WRITE | REQ_SYNC); bio.bi_iter.bi_sector = blk_idx * (PAGE_SIZE >> 9); From patchwork Wed Dec 18 06:34:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13913090 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02E13E77183 for ; Wed, 18 Dec 2024 06:35:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6A46C6B009B; Wed, 18 Dec 2024 01:35:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 652EA6B009C; Wed, 18 Dec 2024 01:35:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 51CC46B009D; Wed, 18 Dec 2024 01:35:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 30DCA6B009B for ; Wed, 18 Dec 2024 01:35:50 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id E1F02160BDB for ; Wed, 18 Dec 2024 06:35:49 +0000 (UTC) X-FDA: 82907118336.12.D69D700 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf15.hostedemail.com (Postfix) with ESMTP id 4075FA0008 for ; Wed, 18 Dec 2024 06:35:02 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="Y/QgStrj"; spf=pass (imf15.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.210.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734503715; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=06dLgy7e/21/5Xh2W7aoiU59dLBqGpGpJ1upbma1LzU=; b=zp1AOQwJA0dm0Ao/Zo0XbrTGFPiHLivHvMrg0YdtjdWSdgWpQTnOxH9gne9dKozn2oscpI Rf9FEl5uMZsoSWtCp/SoFTKZZi1N92Znvubihy/3Tw+tniN1zuKkVskQbgAPMF6WIANQZI 9kRkkISNzabbxxUxP+rQJddvwbn6p4A= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="Y/QgStrj"; spf=pass (imf15.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.210.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734503715; a=rsa-sha256; cv=none; b=Ueu4eWznAWKJodNrgNZcCrx0BjRoXVpRWYdOPe2GsijbxBy4xkgvpTjGkVN+cHQxBfdil6 RZCTP7Shm7w+MV7H+xtvEBorEGLESc6FFe82L/QPcW8a82z4W/jpiL7PArF/tZMvKXGu2p 9FQfnOjoKEOgFrbtVlZQvL02tVlnzYQ= Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-725e71a11f7so320164b3a.1 for ; Tue, 17 Dec 2024 22:35:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1734503747; x=1735108547; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=06dLgy7e/21/5Xh2W7aoiU59dLBqGpGpJ1upbma1LzU=; b=Y/QgStrja9OJ6iAAC9BU0067zf8VMaVeK7foSDrq5p+hQHJ4lXWG/dzx0RlqCDa/G5 gILuI1U3nwYg0jk2qcznGVahUILbrErrXx6aLlmo3tnkxLri+7XeXQXDQi77lPSb+9Le 2Ur9fktLg8zccPI7AuNdJQquUMInE2dLvcGzM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734503747; x=1735108547; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=06dLgy7e/21/5Xh2W7aoiU59dLBqGpGpJ1upbma1LzU=; b=k2LhVApGQntb3/U+3jiAN/sxWPXZIEkbTsHNdy6e5tDzVyPySDKD4o5X/YrEqt5dBm kHhLacyjhYXrLvwM5ob5z0z00Tu2aqG2ME+hkR18A92CVJ7jKg3IaDEIlzzWPwG6Ruco IvYGDxfMnscRnYeDBI2MLoOb7GtTRyTdAqyBWiVGF/oDDv6/gPvpZnWmL+nVdPM0tec4 LOnU5KEAp4MCXeGd3P7zQSDvlNLdVdF1ZsUGICbFp1dQIYi21HpY1QZkN1JrK0vPROae 4aU++vhCNz1bdUfqHTfgKWLBueVaVGq/pGBxnogJHEp0SaYxxa1Uh0OHTqdjimYlzHhd 2xHQ== X-Forwarded-Encrypted: i=1; AJvYcCX8liPNu3wpOyleXukvR7cG9X7vjSVnGbt6cjXdiF9RAgIf8h+Ix2uyNhWGhaAjrz1pbjqHkFfG4Q==@kvack.org X-Gm-Message-State: AOJu0YzV3tOhEcM7+Tgc757HYb/TFoL08Ms0mU0RNMD+w3M9sdBJXimB 7WZU9y34kuimtH6+n41R63z2yXFijDSvr9tUN21oY8hg63UcOx0VWNk4IlbbwQ== X-Gm-Gg: ASbGnctfMA5iMrOU4umw548It9Ovs7bNzlKxWRprZtWyhXVe9d8RXpqUJ2Czs5prOUd 8Cq6GXjs8XUYqheUfPQBGxjWYtB/sR8aQjxdm1aY3fVZs11Wk/e8Jqa3m58uLqMZ7YzpLuGZ3Xd qTBq8l67hfKclwpY2bI+bQ0PSpQsmpEY+CxBG4AI9glj0B7ji0ZJAogJmPEIbV697+vv/vSbPqQ IUsImFLbCRCb+sQoBzGcECIX1xQWcbH/S94G6KtiQehMQSCwbeYhialjco= X-Google-Smtp-Source: AGHT+IGf8g96scNATdGQ6vdl6cixlVJ73v6GuTOkPIuKqnPqyqGhnKzXBehwZAxlCiqlPr2dEdXf8A== X-Received: by 2002:a05:6a20:431b:b0:1e1:aef4:9cd9 with SMTP id adf61e73a8af0-1e5b5a7da91mr2332683637.23.1734503746822; Tue, 17 Dec 2024 22:35:46 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:3bcc:36cc:b9fe:9379]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72918ad5c07sm8023644b3a.60.2024.12.17.22.35.45 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 17 Dec 2024 22:35:46 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv2 7/7] zram: cond_resched() in writeback loop Date: Wed, 18 Dec 2024 15:34:24 +0900 Message-ID: <20241218063513.297475-8-senozhatsky@chromium.org> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog In-Reply-To: <20241218063513.297475-1-senozhatsky@chromium.org> References: <20241218063513.297475-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 4075FA0008 X-Rspam-User: X-Stat-Signature: et6ga3qhe9hkd8idcqqbpkqpyj818jnw X-HE-Tag: 1734503702-798744 X-HE-Meta: U2FsdGVkX18w+k1PRV+LpHwtH2+M05Dc3pTOvLru0zZOJdcDm1xi9GWItXOqK7M5ZvQwwetRV9Pc/FOSoE/YJD+U73fj82eDNNavKyrnQhpq3Hvp1p9axJAlRgzhRQK5PUfajTbyrI3lK5nHuKnLvyaVTRImWn+VnhvqIRPmkEthpmve+xigI+TjeVG40vzDeH5G/hvZwC7YaDxydTZ+BaX48sYPEiI6y7/myNw3fUkShlbH4lhxvPEpBN9tdK3fuwuPP2PDYZQ6kzDVKNRqQkkMaoGVAxN9qQ3rHkRH77sIQckI3eMP+/XgL3VzxOXIXveCGUhrMJCfmMbNdmHH7xdsjwLF+osZ7aKxfuM2H3dy74lS4y0E+W23b3+WusLu8L/X/vJ/S4REV0AZmCdxjA6ecOFpkiKCoG6gg1eYmxv3NvVIb/68Lqai/DfxeI6Axjh/eky96m6Wu3sd6RUoZvR2LqYDI/yCHgK/YeRzZMNcTe4mGtjtfFg4ZOFQWH+apchAcGwhbUR667ASiwMlmA2GYXsl5Nh5YJwdm9Ff1mb5+MqEAMYtWPvpE+hOoFiyhHtqsr/42gZC4uXfij9WFl5be+m8Xc2efAMABNZp6cIQXXYcEgjNf78u0J+CPciBIcTMQhBkE8h6FVS2GgiTcgcUkREM7eL/JlgnPK8hzEpc92HHBvNvObO9oKj7u6vg/9QDfjSc5Di69bUfB4rdPsglLgUICcQl+sKxGjV+4xquej80pNWrhY3XgPxcHDfg3/RJrpfXfb3WsaWF8q4A8v8F/62cl17mCA/K/9sy3SuSgxdc5LJiP0RMmS0++g7s0BuKtOBxLAp95KvOtxtUhCecUKZbT+6U9veNs/M5mX3tf9fE/qTIn9R3JoSC9vtlbZ8we5qgHoujzoOCnJQjLnwT1y+454d7cg3IEQgVNy7B6U2TEKo+lRjS6V66OS7fbo73l/HUaKf+8B6tUPV WmPRoIP2 IQZMJ3A5lzXSNLrmI5FoMhhej2enI+fP8pwlwxUKkoulQ2orXPwYa9AOpFB0N3LHIZAkIQ+qagDFaWUsWkO6IivRkDvCY0km+mz8p9ScKc7bNkCv+ThmXw4o9uggj+8kgd3Wm2HNc7ND04y4840Y0tMEyLGSa8UvlmgdMleO0iuQxKQlZA0tLuzmkDES5FHH9yvPVvNxJih3y+TlmNeBl7o2NrXGABLbuH5jDQUwa6YkwivMm+6fp0iTEpebD8MT5xF8yo+0PXGadDVxvr7kIey8dOUtZUYiHMY+JZFCboJv4vJwaknAqIAy/Tw== X-Bogosity: Unsure, tests=bogofilter, spamicity=0.486822, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: zram writeback is a costly operation, because every target slot (unless ZRAM_HUGE) is decompressed before it gets written to a backing device. The writeback to a backing device uses submit_bio_wait() which may look like a rescheduling point. However, if the backing device has BD_HAS_SUBMIT_BIO bit set __submit_bio() calls directly disk->fops->submit_bio(bio) on the backing device and so when submit_bio_wait() calls blk_wait_io() the I/O is already done. On such systems we effective end up in a loop for_each (target slot) { decompress(slot) __submit_bio() disk->fops->submit_bio(bio) } Which on PREEMPT_NONE systems triggers watchdogs (since there are no explicit rescheduling points). Add cond_resched() to the zram writeback loop. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 7dd72b58e921..5b8e4f4171ab 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -884,6 +884,8 @@ static ssize_t writeback_store(struct device *dev, next: zram_slot_unlock(zram, index); release_pp_slot(zram, pps); + + cond_resched(); } if (blk_idx)