From patchwork Tue Dec 3 15:31:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 13892626 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C9C5E69EA6 for ; Tue, 3 Dec 2024 15:33:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B4C3B6B009A; Tue, 3 Dec 2024 10:33:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AFAAD6B009B; Tue, 3 Dec 2024 10:33:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 94D206B009D; Tue, 3 Dec 2024 10:33:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 6C5A26B009A for ; Tue, 3 Dec 2024 10:33:02 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 0C891A097C for ; Tue, 3 Dec 2024 15:33:02 +0000 (UTC) X-FDA: 82854040458.07.219640D Received: from mail-oi1-f174.google.com (mail-oi1-f174.google.com [209.85.167.174]) by imf01.hostedemail.com (Postfix) with ESMTP id D69C740020 for ; Tue, 3 Dec 2024 15:32:49 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=2t1XGnRN; dmarc=none; spf=pass (imf01.hostedemail.com: domain of axboe@kernel.dk designates 209.85.167.174 as permitted sender) smtp.mailfrom=axboe@kernel.dk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733239974; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oKUI9o4nURcpztCkRpa2upslaug1l9SMNKWGbNjNG00=; b=8B2cYaliZcBL7AcEchjruXroq36XAiigqZjV1VNiO7br8KL/AE91Ii6h0CprG+/Bf1zIIS jPc9bjYmarn9Lk3gahS/Kd4bqrXAFYkpj+KjPhDYxb3+cJ6jZldGnuWNRhI7KKF5HQJTLZ hejSj5b+QQxpVZ9Nqvf1hO4N/sFJ4cc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733239974; a=rsa-sha256; cv=none; b=X396uSavVCJI3qKLAeSh4BXxP+bltP4lL+D4z6feglUwuqcw9yKMX9ennVxXC7ppNupy+o kzR1+r4qruKLwAiSNAt4igin3J4FA4l/amaAMDLBRPRjuYYR6z6oaZPlmS53kijQtsEz6R omY45dhxiCt3Ps76+mSskz32Ate58Cw= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=kernel-dk.20230601.gappssmtp.com header.s=20230601 header.b=2t1XGnRN; dmarc=none; spf=pass (imf01.hostedemail.com: domain of axboe@kernel.dk designates 209.85.167.174 as permitted sender) smtp.mailfrom=axboe@kernel.dk Received: by mail-oi1-f174.google.com with SMTP id 5614622812f47-3ea60f074c3so3240316b6e.1 for ; Tue, 03 Dec 2024 07:32:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20230601.gappssmtp.com; s=20230601; t=1733239978; x=1733844778; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oKUI9o4nURcpztCkRpa2upslaug1l9SMNKWGbNjNG00=; b=2t1XGnRNU4m7g4x3VT4RjUo/z03GvtVlJ2qOimY+xPQcHBTlwLvwNlwADaYcRRcDxn keBE4cOyCLG1Lki/z7aYNPuIRX5u47EB05tE0oFYY9IXx9BkQXKYuRjPM4UHWaJwHtow TCpJt9HlS09h38nlzZ3NTJsBc0I2hK3S41072dJYDMuHmOBNiH+Q/Uv7Prbb3XsJinkp 0n+6Ft+9xhlB9TpcILQGE1RLzNEiQFCRXeC7jEWOmx+Wi2h+P3j9pNAR23vqWQwk98Sy HqTqaKaXz0S9q2OKvxAFNHBakUwMNEbqXmaaKq7JPxcVjRiHzoxN2GMmvDuIUY8NgjGZ 7rVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733239978; x=1733844778; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oKUI9o4nURcpztCkRpa2upslaug1l9SMNKWGbNjNG00=; b=one4/GKFufeJ6Pyz9agm3LtmROxJ2DqFSL/aeoqz4WVVFNVSkrODRLpKJtA7zyRmCm m1uSadTlyol1KGW9vhZvDz8OiKY5TCljVekOoD+7fS7i1w4nH97HdMOn6PA7bTLSd+Nr SV5zI9/C/5wcUX9XcEc+MrC1DQ/Xbf0MVNr7SqHcEj8NYEoVGvtVdTP6MPaWzI1gEXZf Y81PhTjg4Uw9PVJmbcViAVeBSCYW5oGyfDaQx8Z0imOImRMDvwlZ+EPdIf1NXGbCYT3o Lt+9I0VB05huaqV7tX/tayOqnZBeEVLOwMjFkkqf89Cj8rVBS/9MxGjrW6efTn1T6gvL yWPA== X-Gm-Message-State: AOJu0YzPEk8BJlojJOBl5Mg+E6aKmRZhFbsV5B/3FuVU1+Jl7uDuy1ig AbFKV+1Qx7DTgWa0O9T1RliQqZ7h+xQf8yGJVDog05XmLEnzuJPg1Uo9LUSWnUbrCQXNwCJJS++ 0 X-Gm-Gg: ASbGncuwn3EixyzQidwrpfyoE03S9VN80SncdliaUOA0jAYl3MHwWZdUQ1B5ny0wgLY hFGc83kcACD47uCdr90WfqD1W0mEukNFBLvxr5liMGS2nHvCh7H44QZaHCZGnS738LCbdf7nDnF 7y08k2dJlfBv+VCrQKu8VH+05d5Iwk8b9Q9jnSuDLoVITnDhZ4YcPXGc7+tLH/bplKxmtl1xyZK 1CqxjVsuSG587w5VQgiQq6cS9k9zcYwXEOGXAh0na69kafKQQBfp8Db+CU= X-Google-Smtp-Source: AGHT+IGmoU9sxAuWmjwWvkKLQ0cqG9D7TpsEEAnWsGzkgwInxbMdKTegKVNOgiZahNH2vDY40Qhwdw== X-Received: by 2002:a05:6808:218d:b0:3ea:4aad:a877 with SMTP id 5614622812f47-3eae50d7cd4mr2734954b6e.43.1733239978709; Tue, 03 Dec 2024 07:32:58 -0800 (PST) Received: from localhost.localdomain ([130.250.255.163]) by smtp.gmail.com with ESMTPSA id 5614622812f47-3ea86036cbbsm2891878b6e.8.2024.12.03.07.32.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Dec 2024 07:32:58 -0800 (PST) From: Jens Axboe To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: hannes@cmpxchg.org, clm@meta.com, linux-kernel@vger.kernel.org, willy@infradead.org, kirill@shutemov.name, bfoster@redhat.com, Jens Axboe Subject: [PATCH 09/12] mm/filemap: drop uncached pages when writeback completes Date: Tue, 3 Dec 2024 08:31:45 -0700 Message-ID: <20241203153232.92224-11-axboe@kernel.dk> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241203153232.92224-2-axboe@kernel.dk> References: <20241203153232.92224-2-axboe@kernel.dk> MIME-Version: 1.0 X-Stat-Signature: 6ewwju34th141kw8qz47j17b6nxrcpiq X-Rspamd-Queue-Id: D69C740020 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1733239969-33832 X-HE-Meta: U2FsdGVkX1+Y/A1/Zfb6OyspBk7aEPSy2QUHceVlKzihcnd9K/jXJWKTJ9v27jmBJbolZmmuToz+NioEEvfpK9U6Nu95SAf49rDkh4rNqaSUJI9L91RrqzWYfvmBYYKAGYEnMQ3KLdp1uItItofj2ge7pUptyFH7vjih4nu20MO4OpB5tKq1O4oKxKGBGn2s9hev93t19rE7ZCL9FlI8OuSCoo9V8WAX0x99V0ql20Apqi4r00sxDU2nGdrJmmTWv/GS9QRmqAFkf3okJy8SS8nRKPxzh9TMsQk2CAcRr2CoGz+SnNfgqqy+3U5FKY5tHwG8cogwLgDsF3wV8gki8aA2OY13QzKeCNzGDhT4Kon1K6yHiqxXGCuIg44r8CHHinyCQuVnCZm4zeRyc2DgPHm27wjQ5F67bAsH6jNYbQmS0LP5OxeH9GkABgC1fsxCVXA612kNZRAzOhT6ti9qzcfNL0aY9ZwnIsKx3z3Qtu5PxTlco+zYvPUrqFebxv/LMFTGMmFRWNq4JOFofnGBkZyyWZ4R26dhxgchfTjTTy5Fy0ZvpifMTaKkgX44q5t9TFvzFlRRaG7knLvF1uRjuIUP4FCZKnXh/snQLT98EleYnCM+ixazmMJEwTSs0Qt6nbxCNrBmQFAwLhrBvOu2F3+QCSshVj70ILjNpq/eYuXGohqkRUlPQGaUvVbMriS0lkx+gjRwBQJzF79/8hr9BNs6QVRw5ZfcGyo8O14hoDWs0ic2+HClzNhoqa7y03rgEAkgJSQdMGL0XpVi/3dmqCa9dErusF7bZ6asz54p0OAwXdXWO9QoJlkzVtAJ0CNA5FoG+ZOukJ57BY6mCaKsR1sljFja4ZdUTzQivIniQtOxJWivijR5FBBf2gp5NAPhjkkMVKdiHVvAhef2yOa3PnzuLcXyvC+OG8OKCmpeFbn9SdDBXFqhvJ7sx6kVDmKUw/uz2BOocFhJUoTumdJ 5fzFINCB rImVQsXpMHzj2P3k0k0qJPoYBQwNfxeWH/dr4uxe5jUWeQCRhDk3dVMCJQDgr4xAthlz6y7xfa6Q9R+HQ1voXWhbmSnMzCTXQAEdn3yldadAidSa7AyU+hkUW56RFjsp+fcE0ZZPsw3oXVvsrPIPkTU3QuAWIazpfwa4JH6aQqZ3xUU0GV113k5LSKPXy3/AhmJKTNvV5zEP+Env2CkxKFGHLUaHozcutqx3FuqcUHrPFEgb34wTBBlB3bR7eVT1hDxMINVlx8lHZ4h/vAVJHpb/EARYU96toY4w3QyeHgX5wSAw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If the folio is marked as uncached, drop pages when writeback completes. Intended to be used with RWF_UNCACHED, to avoid needing sync writes for uncached IO. Signed-off-by: Jens Axboe --- mm/filemap.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index 139d1db79ff8..eb6a8d39f9d0 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1599,6 +1599,27 @@ int folio_wait_private_2_killable(struct folio *folio) } EXPORT_SYMBOL(folio_wait_private_2_killable); +/* + * If folio was marked as uncached, then pages should be dropped when writeback + * completes. Do that now. If we fail, it's likely because of a big folio - + * just reset uncached for that case and latter completions should invalidate. + */ +static void folio_end_uncached(struct folio *folio) +{ + /* + * Hitting !in_task() should not happen off RWF_UNCACHED writeback, but + * can happen if normal writeback just happens to find dirty folios + * that were created as part of uncached writeback, and that writeback + * would otherwise not need non-IRQ handling. Just skip the + * invalidation in that case. + */ + if (in_task() && folio_trylock(folio)) { + if (folio->mapping) + folio_unmap_invalidate(folio->mapping, folio, 0); + folio_unlock(folio); + } +} + /** * folio_end_writeback - End writeback against a folio. * @folio: The folio. @@ -1609,6 +1630,8 @@ EXPORT_SYMBOL(folio_wait_private_2_killable); */ void folio_end_writeback(struct folio *folio) { + bool folio_uncached = false; + VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio); /* @@ -1630,9 +1653,14 @@ void folio_end_writeback(struct folio *folio) * reused before the folio_wake_bit(). */ folio_get(folio); + if (!folio_test_dirty(folio)) + folio_uncached = folio_test_clear_uncached(folio); if (__folio_end_writeback(folio)) folio_wake_bit(folio, PG_writeback); acct_reclaim_writeback(folio); + + if (folio_uncached) + folio_end_uncached(folio); folio_put(folio); } EXPORT_SYMBOL(folio_end_writeback);