From patchwork Tue Apr 27 00:13:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 12225177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-21.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,INCLUDES_PULL_REQUEST,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 262BEC433B4 for ; Tue, 27 Apr 2021 00:14:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A5F5261178 for ; Tue, 27 Apr 2021 00:14:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A5F5261178 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1E8336B0036; Mon, 26 Apr 2021 20:14:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 198F36B006E; Mon, 26 Apr 2021 20:14:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 011C36B0070; Mon, 26 Apr 2021 20:14:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0019.hostedemail.com [216.40.44.19]) by kanga.kvack.org (Postfix) with ESMTP id D283B6B0036 for ; Mon, 26 Apr 2021 20:14:09 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 845FB180ACF62 for ; Tue, 27 Apr 2021 00:14:09 +0000 (UTC) X-FDA: 78076224618.31.85355EA Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf07.hostedemail.com (Postfix) with ESMTP id C4399A000394 for ; Tue, 27 Apr 2021 00:14:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1619482448; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U3rMIJssX1d2huVmRZulEKHtd5HxdDz+2MIzkHFnJbI=; b=IyRKE8cllfRd3LEFBbfwPJzLmi4wbp8uvi8qL7hs57IJMNRM/vzWg24LvC9wwvVLhGaq3b 4I3Mb2pf3EivzGV7GzHVDAf5knbEA6g4ZU0pTjehvgw+w4m94aJuDEWcOhHAVRQOTUReYb i2oxdlAXkjdXy+405Q7DxmnpTu8CuPc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-337-aGAgAgXLNhO-fNW1VoCbsA-1; Mon, 26 Apr 2021 20:14:04 -0400 X-MC-Unique: aGAgAgXLNhO-fNW1VoCbsA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E76DF18397A4; Tue, 27 Apr 2021 00:14:01 +0000 (UTC) Received: from warthog.procyon.org.uk (ovpn-112-20.rdu2.redhat.com [10.10.112.20]) by smtp.corp.redhat.com (Postfix) with ESMTP id DD9C260C4A; Tue, 27 Apr 2021 00:13:50 +0000 (UTC) Organization: Red Hat UK Ltd. Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SI4 1TE, United Kingdom. Registered in England and Wales under Company Registration No. 3798903 From: David Howells In-Reply-To: <3779937.1619478404@warthog.procyon.org.uk> References: <3779937.1619478404@warthog.procyon.org.uk> To: Linus Torvalds Cc: dhowells@redhat.com, Alexander Viro , "Matthew Wilcox (Oracle)" , Anna Schumaker , Christoph Hellwig , David Wysochanski , Dominique Martinet , Jeff Layton , Marc Dionne , Mike Marshall , Steve French , Trond Myklebust , ceph-devel@vger.kernel.org, linux-afs@lists.infradead.org, linux-cachefs@redhat.com, linux-cifs@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-nfs@vger.kernel.org, v9fs-developer@lists.sourceforge.net Subject: [GIT PULL] afs: Preparation for fscache overhaul MIME-Version: 1.0 Content-ID: <3785062.1619482429.1@warthog.procyon.org.uk> Date: Tue, 27 Apr 2021 01:13:49 +0100 Message-ID: <3785063.1619482429@warthog.procyon.org.uk> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Rspamd-Queue-Id: C4399A000394 X-Stat-Signature: kkgk3t58sr7uzxc41coykudy33su1xda X-Rspamd-Server: rspam02 Received-SPF: none (redhat.com>: No applicable sender policy available) receiver=imf07; identity=mailfrom; envelope-from=""; helo=us-smtp-delivery-124.mimecast.com; client-ip=216.205.24.124 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619482448-999996 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Linus, Here's a set of patches for the AFS filesystem for 5.13 to begin the process of overhauling the use of the fscache API by AFS and the introduction of support for features such as Transparent Huge Pages (THPs). (1) Add some support for THPs, including using core VM helper functions to find details of pages. (2) Use the ITER_XARRAY I/O iterator to mediate access to the pagecache as this handles THPs and doesn't require allocation of large bvec arrays. (3) Delegate address_space read/pre-write I/O methods for AFS to the netfs helper library. A method is provided to the library that allows it to issue a read against the server. This includes a change in use for PG_fscache (it now indicates a DIO write in progress from the marked page), so a number of waits need to be deployed for it. (4) Split the core AFS writeback function to make it easier to modify in future patches to handle writing to the cache. [This might feasibly make more sense moved out into my fscache-iter branch]. I've tested these with "xfstests -g quick" against an AFS volume (xfstests needs patching to make it work). With this, AFS without a cache passes all expected xfstests; with a cache, there's an extra failure, but that's also there before these patches. Fixing that probably requires a greater overhaul (as can be found on my fscache-iter branch, but that's for a later time). Thanks should go to Marc Dionne and Jeff Altman of AuriStor for exercising the patches in their test farm also. Changes ======= These patches are dependent on the netfs-lib branch and have been posted in association with them. The changes relevant to these patches are: ver #6: Split the afs patches out into their own branch. ver #5: Fixed some review comments from Matthew Wilcox: - Better names for wrangling functions for PG_private_2 and PG_fscache wrangling functions[3]. Came up with {set,end,wait_for}_page_private_2() and aliased these for fscache. Moved the taking of/dropping a page ref for the PG_private_2 flag into the set and end functions. ver #4: Rebased to v5.12-rc2 and added a bunch of references into individual commits. ver #3: Adjusted the functions that unlock and wait for PG_fscache according to Linus's suggestion[1]. Hold a ref on a page when PG_fscache is set as per Linus's suggestion[2]. Link: https://lore.kernel.org/r/CAHk-=wh+2gbF7XEjYc=HV9w_2uVzVf7vs60BPz0gFA=+pUm3ww@mail.gmail.com/ [1] Link: https://lore.kernel.org/r/CAHk-=wjgA-74ddehziVk=XAEMTKswPu1Yw4uaro1R3ibs27ztw@mail.gmail.com/ [2] Link: https://lore.kernel.org/r/20210321105309.GG3420@casper.infradead.org/ [3] References ========== These patches have been published for review before, firstly as part of a larger set: Link: https://lore.kernel.org/r/158861203563.340223.7585359869938129395.stgit@warthog.procyon.org.uk/ Link: https://lore.kernel.org/r/159465766378.1376105.11619976251039287525.stgit@warthog.procyon.org.uk/ Link: https://lore.kernel.org/r/159465784033.1376674.18106463693989811037.stgit@warthog.procyon.org.uk/ Link: https://lore.kernel.org/r/159465821598.1377938.2046362270225008168.stgit@warthog.procyon.org.uk/ Link: https://lore.kernel.org/r/160588455242.3465195.3214733858273019178.stgit@warthog.procyon.org.uk/ Then as a cut-down set: Link: https://lore.kernel.org/r/161118128472.1232039.11746799833066425131.stgit@warthog.procyon.org.uk/ # v1 Link: https://lore.kernel.org/r/161161025063.2537118.2009249444682241405.stgit@warthog.procyon.org.uk/ # v2 Link: https://lore.kernel.org/r/161340385320.1303470.2392622971006879777.stgit@warthog.procyon.org.uk/ # v3 Link: https://lore.kernel.org/r/161539526152.286939.8589700175877370401.stgit@warthog.procyon.org.uk/ # v4 Link: https://lore.kernel.org/r/161653784755.2770958.11820491619308713741.stgit@warthog.procyon.org.uk/ # v5 Link: https://lore.kernel.org/r/161789062190.6155.12711584466338493050.stgit@warthog.procyon.org.uk/ # v6 Link: https://lore.kernel.org/r/161918446704.3145707.14418606303992174310.stgit@warthog.procyon.org.uk # v7 David --- The following changes since commit 26aaeffcafe6cbb7c3978fa6ed7555122f8c9f8c: fscache, cachefiles: Add alternate API to use kiocb for read/write to cache (2021-04-23 10:14:32 +0100) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git tags/afs-netfs-lib-20210426 for you to fetch changes up to 3003bbd0697b659944237f3459489cb596ba196c: afs: Use the netfs_write_begin() helper (2021-04-23 10:17:28 +0100) ---------------------------------------------------------------- AFS: Use the new netfs lib ---------------------------------------------------------------- David Howells (14): afs: Disable use of the fscache I/O routines afs: Pass page into dirty region helpers to provide THP size afs: Print the operation debug_id when logging an unexpected data version afs: Move key to afs_read struct afs: Don't truncate iter during data fetch afs: Log remote unmarshalling errors afs: Set up the iov_iter before calling afs_extract_data() afs: Use ITER_XARRAY for writing afs: Wait on PG_fscache before modifying/releasing a page afs: Extract writeback extension into its own function afs: Prepare for use of THPs afs: Use the fs operation ops to handle FetchData completion afs: Use new netfs lib read helper API afs: Use the netfs_write_begin() helper fs/afs/Kconfig | 1 + fs/afs/dir.c | 225 +++++++++++----- fs/afs/file.c | 483 +++++++++------------------------ fs/afs/fs_operation.c | 4 +- fs/afs/fsclient.c | 108 +++----- fs/afs/inode.c | 7 +- fs/afs/internal.h | 59 ++-- fs/afs/rxrpc.c | 150 ++++------- fs/afs/write.c | 657 +++++++++++++++++++++++---------------------- fs/afs/yfsclient.c | 82 ++---- include/net/af_rxrpc.h | 2 +- include/trace/events/afs.h | 74 +++-- net/rxrpc/recvmsg.c | 9 +- 13 files changed, 805 insertions(+), 1056 deletions(-)