From patchwork Fri Jun 2 22:24:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 13265926 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 354C7C7EE2C for ; Fri, 2 Jun 2023 22:25:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236574AbjFBWY7 (ORCPT ); Fri, 2 Jun 2023 18:24:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33792 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236490AbjFBWY6 (ORCPT ); Fri, 2 Jun 2023 18:24:58 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 129349D; Fri, 2 Jun 2023 15:24:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:In-Reply-To:References; bh=iQ0827CwQnnlSwpm+aCdM8sh89dkXAHIYux3uuAVwMw=; b=dYiN6hgrBf7BhxOgIIJ45hksR8 gjn0PVUsFdlrY1p5jBic5I3Q1H3G1xuPJGOVcASl5lI0KET4BoAqD4OTMjBDMmBSBd8GFa3vUL4tP 68+KxKlOP1bQgT0sI9kSzjUhec1KMnBLm0uQZxrz9QnM8IlrrwyKiPOoMnFcy230H1liLUUPS1PO/ xNz760nafRuDfQjlbMVd5yxKwA2REfNnxdgDxV/vN9gtag3ZakfyqcFk+szdGObaytQs6/PJHx8p8 jv7iO26ti7Vd7bgbmL1l+igKFdWh9MwAj1rvkqWnMR6yeaqJZERXfcrue7Vu2pObUVCD1NK39KGwo kqK+r1/w==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1q5DCQ-009aPG-Dx; Fri, 02 Jun 2023 22:24:46 +0000 From: "Matthew Wilcox (Oracle)" To: linux-fsdevel@vger.kernel.org Cc: "Matthew Wilcox (Oracle)" , linux-xfs@vger.kernel.org, Wang Yugui , Dave Chinner , Christoph Hellwig , "Darrick J . Wong" Subject: [PATCH v2 0/7] Create large folios in iomap buffered write path Date: Fri, 2 Jun 2023 23:24:37 +0100 Message-Id: <20230602222445.2284892-1-willy@infradead.org> X-Mailer: git-send-email 2.37.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-xfs@vger.kernel.org Commit ebb7fb1557b1 limited the length of ioend chains to 4096 entries to improve worst-case latency. Unfortunately, this had the effect of limiting the performance of: fio -name write-bandwidth -rw=write -bs=1024Ki -size=32Gi -runtime=30 \ -iodepth 1 -ioengine sync -zero_buffers=1 -direct=0 -end_fsync=1 \ -numjobs=4 -directory=/mnt/test The problem ends up being lock contention on the i_pages spinlock as we clear the writeback bit on each folio (and propagate that up through the tree). By using larger folios, we decrease the number of folios to be processed by a factor of 256 for this benchmark, eliminating the lock contention. It's also the right thing to do. This is a project that has been on the back burner for years, it just hasn't been important enough to do before now. I think it's probably best if this goes through the iomap tree since the changes outside iomap are either to the page cache or they're trivial. v2: - Fix misplaced semicolon - Rename fgp_order to fgp_set_order - Rename FGP_ORDER to FGP_GET_ORDER - Add fgp_t - Update the documentation for ->release_folio - Fix iomap_invalidate_folio() - Update iomap_release_folio() Matthew Wilcox (Oracle) (7): iomap: Remove large folio handling in iomap_invalidate_folio() doc: Correct the description of ->release_folio iomap: Remove unnecessary test from iomap_release_folio() filemap: Add fgp_t typedef filemap: Allow __filemap_get_folio to allocate large folios iomap: Create large folios in the buffered write path iomap: Copy larger chunks from userspace Documentation/filesystems/locking.rst | 14 ++++-- fs/btrfs/file.c | 6 +-- fs/f2fs/compress.c | 2 +- fs/f2fs/f2fs.h | 2 +- fs/gfs2/bmap.c | 2 +- fs/iomap/buffered-io.c | 43 ++++++++-------- include/linux/iomap.h | 2 +- include/linux/pagemap.h | 71 ++++++++++++++++++++++----- mm/filemap.c | 61 ++++++++++++----------- mm/folio-compat.c | 2 +- mm/readahead.c | 13 ----- 11 files changed, 130 insertions(+), 88 deletions(-)