From patchwork Wed Mar 8 14:37:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13165936 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 350F0C64EC4 for ; Wed, 8 Mar 2023 14:38:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AEC416B0074; Wed, 8 Mar 2023 09:38:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A9BEB6B0075; Wed, 8 Mar 2023 09:38:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 93D246B0078; Wed, 8 Mar 2023 09:38:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 85F5C6B0074 for ; Wed, 8 Mar 2023 09:38:10 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 60328A099F for ; Wed, 8 Mar 2023 14:38:10 +0000 (UTC) X-FDA: 80545985940.14.76706BB Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf22.hostedemail.com (Postfix) with ESMTP id 87AE4C0004 for ; Wed, 8 Mar 2023 14:38:07 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=VvFHQb8p; spf=pass (imf22.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678286287; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dL8s47Tp1nK8GIg8PijNkCrDLEgPha2fkCIMRUXm1zk=; b=lcI06Xw6coR/lrbTRKJZc2negLC1UBMM2xk91FfNd1sjT3S5X4JA28fi3R67zmVedtGNpX 3s7gXd9lGaYy26onuPsTO88fOVt4+pCiKzbNj24ELOOk+WUpFb7M+RPtuctobfJ9GBpNMo Ori6pHTOHvS0RLSmVtUokp13pEaeUvc= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=VvFHQb8p; spf=pass (imf22.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678286287; a=rsa-sha256; cv=none; b=N40dkKqfJ2n4GWouoo+KCAdQxrZdo3QCXJlRrztWxZaTbTKrXS5uDo8J3Osa4BwjNVtDCd l8m1/6UMkgxZBFfO5L0LeBGAagKtFicFpPSsJm721r//QEbvyRxe8nP6EMzGGIsFWM9nD0 0idbrnSK+hfS4LNjbswelBUBC7jNOPI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678286286; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dL8s47Tp1nK8GIg8PijNkCrDLEgPha2fkCIMRUXm1zk=; b=VvFHQb8pdLfOvLL/lLP95S3DGaPgh++BEBAhDpKKJRCgMtQl45foWnFo4gGF7ms6b961rK zG4YYgTRq3croVPkEg2juxQCcyEKqKsGAcq3CEgbMRr8KJDymwaqoNoHA4/ragilrOQrgl nTSASzRs6+0j85ujsqb1YW6xVQaFN1o= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-20-WwOWsJk3MvajoYgGpEO4_w-1; Wed, 08 Mar 2023 09:38:02 -0500 X-MC-Unique: WwOWsJk3MvajoYgGpEO4_w-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B1438857A84; Wed, 8 Mar 2023 14:38:01 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id ACA4114171B6; Wed, 8 Mar 2023 14:37:59 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v16 01/13] splice: Clean up direct_splice_read() a bit Date: Wed, 8 Mar 2023 14:37:42 +0000 Message-Id: <20230308143754.1976726-2-dhowells@redhat.com> In-Reply-To: <20230308143754.1976726-1-dhowells@redhat.com> References: <20230308143754.1976726-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.7 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 87AE4C0004 X-Rspam-User: X-Stat-Signature: p7bbrodyim3zygwpbbryzo4xr9d1z5do X-HE-Tag: 1678286287-559509 X-HE-Meta: U2FsdGVkX1+GzcJBf2Sonm5LU6D92mLCdO+xRq5WKjUA2ZY6iA8TlR7Hb/qM5ZiDGvXMjP5RhZaERYkF19+Z383X3+tHUF5epWq0Jkgg1okxHadlLQpYaiAggcuP4qaU0maiR5fc0qlklGnXM6VTTzVj/pNGUn0c6CldKFZpQASO5VVky24/ROKa2U2WDvmzSIJ5J4As++KkvAvSHeD6rcyDnxrKuNXYz+aJcjQeXbuHo/4D14jXKeiXq5VdEJ85V/BtGpe83ljdE0rPBBm2RMdHOMNKtN1U4YUwxTxhtD8mRvXEhNSgji4jFQrXfBq89JKt3Nie3ufRRDK6eL5cI/GF9ctPbr1n6XzsbM2w9XKj/XJt5/kvGzVAV42MYdjkkFiZIC2xtla31f+8xdfMcOvWcSYpteR7O020xnw6/gmXDu2Y0oBhu9YO2d5iTEAEiPKcw0F7OZ7VFQFAmMZGo5YIyLauIwjfezyeJt8FZgcRylw2dA4WZ8f9d2SiP/W7Jo/R8LnWKf9Pc1ARQTOHYsS4Z6jKuvXb1Wl1MT/tODgpqmQDomjETiPXxTh2sG6qYd0UdAhAEMlCbZyHTTexo5Wp5ymKiYTf23krp32aA6q3gV6wVTwlm8eE+Kw36kTc6+rOsAmdktO7seSfrbduqAXpUY1ktSEMBwxfbVvDTURd5tl6sofT/uYzp7MjQPAQGYnjI8QbFpYWP1OMsEnNOsYNLL245MdeHQxUN0Nhc/RljQ27Dfwt9UWxUQtwmCkoBSuUpGAuIWfK2qU6pk/ZQQSINB4kaCSMNxXj7D3PfXwxwUvO9+fiaR/8Lrd9ewy+3J+CT+Gx60ATU21j1q0JK+pMCttRgY2MA6L/Izeu1odcToKedE5cn42O9kOMowKRPPT4Nw14fD+iL6VGqOQn+y7kQlucYBv+K3BypTy58DyWgh8zvR1GMGimWnLnFjIYYdvcSYctmyEXhoyNpwA BLp7tWqy uzBVTZq4Uiaiv/WN83O1yRzCGHvGd6LK3jZTje1JEFSGhhx/Tn75WJRbj6Zpp3CaPxFn6qZGucxAbdnBSnlsfQVeveXDgTKuEvTbMUkwdDAia80s+OkGba/oG+KI/nNtbVvWr4myttFnDVMv18lGvElJfhCGEWCfmDEU/V2f38fNzDmlWVj+Q0+O8+W4wuZtPiOxRONvNks1uo+PxvSpdUpxCVRJn7fg4wqi7WBtSlX7HrL1BBcUoSC4abHg4wfB9pTLo8gX/OPGGHSApzDPxnhwhpXNROfYItYlnIrYSXPlh0hTbHzr+gmsddXiFDkJYSuOO5RHt1NqKb/x0IpRyOzTWQwN51WDPd8d3JKsFojvS79ctKM/SkiAR862xF8Pch9ZhIjx9neRh6qhf12Ur11uOKb49mzz6NKqEEokCn+FL61CLrTlXSHB3jWHSjh68/CFFtcEIwTwrNVSC0pEdlPJmCVB6cEMpvgK/GeOufPJ7dw/TyCc7Azgs/Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Do a couple of cleanups to direct_splice_read(): (1) Cast to struct page **, not void *. (2) Simplify the calculation of the number of pages to keep/reclaim in direct_splice_read(). Suggested-by: Christoph Hellwig Signed-off-by: David Howells cc: Jens Axboe cc: Christoph Hellwig cc: Al Viro cc: David Hildenbrand cc: John Hubbard cc: linux-mm@kvack.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org --- fs/splice.c | 19 +++++++------------ 1 file changed, 7 insertions(+), 12 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 2e76dbb81a8f..abd21a455a2b 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -295,7 +295,7 @@ ssize_t direct_splice_read(struct file *in, loff_t *ppos, struct kiocb kiocb; struct page **pages; ssize_t ret; - size_t used, npages, chunk, remain, reclaim; + size_t used, npages, chunk, remain, keep = 0; int i; /* Work out how much data we can actually add into the pipe */ @@ -309,7 +309,7 @@ ssize_t direct_splice_read(struct file *in, loff_t *ppos, if (!bv) return -ENOMEM; - pages = (void *)(bv + npages); + pages = (struct page **)(bv + npages); npages = alloc_pages_bulk_array(GFP_USER, npages, pages); if (!npages) { kfree(bv); @@ -332,11 +332,8 @@ ssize_t direct_splice_read(struct file *in, loff_t *ppos, kiocb.ki_pos = *ppos; ret = call_read_iter(in, &kiocb, &to); - reclaim = npages * PAGE_SIZE; - remain = 0; if (ret > 0) { - reclaim -= ret; - remain = ret; + keep = DIV_ROUND_UP(ret, PAGE_SIZE); *ppos = kiocb.ki_pos; file_accessed(in); } else if (ret < 0) { @@ -349,14 +346,12 @@ ssize_t direct_splice_read(struct file *in, loff_t *ppos, } /* Free any pages that didn't get touched at all. */ - reclaim /= PAGE_SIZE; - if (reclaim) { - npages -= reclaim; - release_pages(pages + npages, reclaim); - } + if (keep < npages) + release_pages(pages + keep, npages - keep); /* Push the remaining pages into the pipe. */ - for (i = 0; i < npages; i++) { + remain = ret; + for (i = 0; i < keep; i++) { struct pipe_buffer *buf = pipe_head_buf(pipe); chunk = min_t(size_t, remain, PAGE_SIZE); From patchwork Wed Mar 8 14:37:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13165937 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 165DCC678D5 for ; Wed, 8 Mar 2023 14:38:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8A22C6B0075; Wed, 8 Mar 2023 09:38:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 82AF16B0078; Wed, 8 Mar 2023 09:38:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6A7A36B007B; Wed, 8 Mar 2023 09:38:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5C9836B0075 for ; Wed, 8 Mar 2023 09:38:12 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 31FCC1603F8 for ; Wed, 8 Mar 2023 14:38:12 +0000 (UTC) X-FDA: 80545986024.21.B3074C0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf06.hostedemail.com (Postfix) with ESMTP id 599D6180020 for ; Wed, 8 Mar 2023 14:38:10 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gxC310uW; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678286290; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=n2MkOxcvB8unEejbMWkMEf534o0h2+e07KJxot0+dsE=; b=X+mqiAdHPnsk3gIPvEs8vcrFss7vb/DECHVIykAjpAetfVgOyRIhMLE78nI3RLd2+lWqM/ 66p6IBvAup1wJ8eYdDzDE74BNtQOVqon/9jXejWy4D7mhApjRRhrrgb2Mpb3qq4aQrDE9Z hm3RgK/fW681Z/cf3kM/BHv0siXpLCw= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gxC310uW; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678286290; a=rsa-sha256; cv=none; b=EGySzVwWSHrXI+Slzl+uBHge9OwYcWAhsCIKp1rBad4NwgomxHIFUhjVvAyzgDeIsnF3/4 P/SW07qCmmgcLmuoFxFS1bndKcScGN+2WCx5EazFmeyjc6y5kvNHqkSe955kA9WzPOSsUK mBiWh7OIRmMyvVcGkekSOT+DoTeH3W4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678286289; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n2MkOxcvB8unEejbMWkMEf534o0h2+e07KJxot0+dsE=; b=gxC310uWHS6NiaRAV6FqVIzCdFCdVAuRZdDcZohlLWJsY1xcd/9s5VMivGDSjYZ++7z4BX D6ApfGvtSBsEJzVDW+6/pQbbIUfwkzXa+BthGdy2m+QjvFYx5x4USMyv6N7L9LRj7t4G4h dWM3pgd/DyXAYfINsOjk5wdIQrGHwzc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-475-lOpGQlcJP-ePYAyUmIIffg-1; Wed, 08 Mar 2023 09:38:06 -0500 X-MC-Unique: lOpGQlcJP-ePYAyUmIIffg-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BD49D185A7A4; Wed, 8 Mar 2023 14:38:04 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 718F3492C3E; Wed, 8 Mar 2023 14:38:02 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Daniel Golle , Guenter Roeck , Christoph Hellwig , John Hubbard , Hugh Dickins Subject: [PATCH v16 02/13] shmem: Implement splice-read Date: Wed, 8 Mar 2023 14:37:43 +0000 Message-Id: <20230308143754.1976726-3-dhowells@redhat.com> In-Reply-To: <20230308143754.1976726-1-dhowells@redhat.com> References: <20230308143754.1976726-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Rspamd-Queue-Id: 599D6180020 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: p14doejgraxodcn4s7yi49zxd5c7z64b X-HE-Tag: 1678286290-123324 X-HE-Meta: U2FsdGVkX18edV2vHgQsG31ekTJMS3GkNtLX4JFVXBBQysrRh0vh5Wg9A3MQ5EcXa4G2AUkNZpO/BhmbRiTDXdTrRSEhiMxuFj3cOymcvyVpaxt9DxPmLMmG8ZLh5g4CcKvKuZHwCBdfynUPowZck8xHIXcmAN02j1V7+l7feakDS9dsDamoU72r71zfjFg9NkvREwuA9GIuOcBZWPk2J4LkOlTrbhhujU1RYJBBnz54srwn8gR0dDzy3B6iMQLegPUMgWA3VITXjJcrzCEzfS+lg4bG1gVyOvZFSczG6emlSRq23+9m8BJ/iJXbzH8tyzf1Ru49e9o7zZp13Ng77haTRjr6ToYzJcb8WSSsdwtl+C8fdbnrBZFXAYj6fw3zhfn7y5bhpgiGj7mD5ewD5tSsYzHCW6qgViZOK+PtjHHrCxbnPOzDyP/uor5dJqqa6Yn/B4g2NhlT2lbbGGOr87N3y9VJLB4kSCnuHqilNAOcTW9yN6oV8Zrst9nTiCCDyz52mS2jVsL686Qr1Dzfs8hiZygH6KwdJmFJroILubWT+7Nrc+qMCJnfli1a4NcnNHir6ZgIQ1Mv02dhvEtRdaUWrWAJoQWqNYIseU2Sgt2sw6Dl68UPV45LAhWhN03TeqH5AeoG8x2cYCuHpS54Dzhd/NYGbQQ2fGUDgzg9AAoKDFbsqyrlk604/RBwcz7yzbM3vMfsnkKdK+mb66e0kn9tNBskpgZvaShVA0DsgNFv6bA1VAlhmI4PcanMYKMoAsZUg5TMqsMaPPmk/kwIVgv4mEJypLP1v6WfsFkKvYCxr8tMMstvPdUVkw0IPlWHfaRcaKxN5I4F7lvfD2zRtpanhsElUyo6yeR4Mlid3VZlke3taSJSwyrFvRrk6n5a3pudS6GgIOlSQmFmGIXI39rNgePrUw/K9PxM10IZvKGAe3oOlOMfefCuVyC9eBCbzKEkiwSEhhvw8yVWGwG bB+/G5Bf RK5Gz/ngm+SfT/T1SPTr5FbGXYLExZ00D1b1HwWLZ3TA31PJ3jybLqZeArfe88B8VrZIDXe1/TjyiUARWK6OEc3HyagJHmy8nXDaKTq0G4+lopvttVe69F85sF45OwV6zH5tEdeCVrmSbXgErdwAlxIqUzfXI6/+PNsNNCnnY7+uUy9QPJM4g291NbsUFOM+9RQY+79iG6h4OzQHC+k4B+YXG2v1HMDygot8MGt9lXa1tRkzA79YyTYx8VCRm1u8R58R+clPXLRW8rKPXilNNMzSkRFJRr8LvB43HFiWUvvthRMcMtchmKGl78TmAf4H3vjQpj+fcnpCSyRVGZSFgG4BTXfwCG1b4sEEO9VD3ilfa3werhIP+zbQ8MYnKpb8cYb0ImOFpsCCRkviTCVISZ+MLtXdwntIRYxLdn9r5tcETYCCuyVR5MdAGSd2RWINhIqYEyRmPlOtxTTtE0859LCFhKZO9kmgnaUgbLIiPWgdsrY1IRaavZlixKsHob64Tf192hZEXvhwt01ThsDzwRsNygtUyvywHlqIo7WZjKGN9Ru6aXdxG0zw4XA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The new filemap_splice_read() has an implicit expectation via filemap_get_pages() that ->read_folio() exists if ->readahead() doesn't fully populate the pagecache of the file it is reading from[1], potentially leading to a jump to NULL if this doesn't exist. shmem, however, (and by extension, tmpfs, ramfs and rootfs), doesn't have ->read_folio(), Work around this by equipping shmem with its own splice-read implementation, based on filemap_splice_read(), but able to paste in zero_page when there's a page missing. Signed-off-by: David Howells cc: Daniel Golle cc: Guenter Roeck cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: Hugh Dickins cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Link: https://lore.kernel.org/r/Y+pdHFFTk1TTEBsO@makrotopia.org/ [1] --- mm/shmem.c | 124 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 123 insertions(+), 1 deletion(-) diff --git a/mm/shmem.c b/mm/shmem.c index 448f393d8ab2..3cbec1d56112 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2719,6 +2719,128 @@ static ssize_t shmem_file_read_iter(struct kiocb *iocb, struct iov_iter *to) return retval ? retval : error; } +static bool zero_pipe_buf_try_steal(struct pipe_inode_info *pipe, + struct pipe_buffer *buf) +{ + return false; +} + +static const struct pipe_buf_operations zero_pipe_buf_ops = { + .release = generic_pipe_buf_release, + .try_steal = zero_pipe_buf_try_steal, + .get = generic_pipe_buf_get, +}; + +static size_t splice_zeropage_into_pipe(struct pipe_inode_info *pipe, + loff_t fpos, size_t size) +{ + size_t offset = fpos & ~PAGE_MASK; + + size = min_t(size_t, size, PAGE_SIZE - offset); + + if (!pipe_full(pipe->head, pipe->tail, pipe->max_usage)) { + struct pipe_buffer *buf = pipe_head_buf(pipe); + + *buf = (struct pipe_buffer) { + .ops = &zero_pipe_buf_ops, + .page = ZERO_PAGE(0), + .offset = offset, + .len = size, + }; + get_page(buf->page); + pipe->head++; + } + + return size; +} + +static ssize_t shmem_file_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct inode *inode = file_inode(in); + struct address_space *mapping = inode->i_mapping; + struct folio *folio = NULL; + size_t total_spliced = 0, used, npages, n, part; + loff_t isize; + int error = 0; + + /* Work out how much data we can actually add into the pipe */ + used = pipe_occupancy(pipe->head, pipe->tail); + npages = max_t(ssize_t, pipe->max_usage - used, 0); + len = min_t(size_t, len, npages * PAGE_SIZE); + + do { + if (*ppos >= i_size_read(inode)) + break; + + error = shmem_get_folio(inode, *ppos / PAGE_SIZE, &folio, SGP_READ); + if (error) { + if (error == -EINVAL) + error = 0; + break; + } + if (folio) { + folio_unlock(folio); + + if (folio_test_hwpoison(folio)) { + error = -EIO; + break; + } + } + + /* + * i_size must be checked after we know the pages are Uptodate. + * + * Checking i_size after the check allows us to calculate + * the correct value for "nr", which means the zero-filled + * part of the page is not copied back to userspace (unless + * another truncate extends the file - this is desired though). + */ + isize = i_size_read(inode); + if (unlikely(*ppos >= isize)) + break; + part = min_t(loff_t, isize - *ppos, len); + + if (folio) { + /* + * If users can be writing to this page using arbitrary + * virtual addresses, take care about potential aliasing + * before reading the page on the kernel side. + */ + if (mapping_writably_mapped(mapping)) + flush_dcache_folio(folio); + folio_mark_accessed(folio); + /* + * Ok, we have the page, and it's up-to-date, so we can + * now splice it into the pipe. + */ + n = splice_folio_into_pipe(pipe, folio, *ppos, part); + folio_put(folio); + folio = NULL; + } else { + n = splice_zeropage_into_pipe(pipe, *ppos, len); + } + + if (!n) + break; + len -= n; + total_spliced += n; + *ppos += n; + in->f_ra.prev_pos = *ppos; + if (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) + break; + + cond_resched(); + } while (len); + + if (folio) + folio_put(folio); + + file_accessed(in); + return total_spliced ? total_spliced : error; +} + static loff_t shmem_file_llseek(struct file *file, loff_t offset, int whence) { struct address_space *mapping = file->f_mapping; @@ -3938,7 +4060,7 @@ static const struct file_operations shmem_file_operations = { .read_iter = shmem_file_read_iter, .write_iter = generic_file_write_iter, .fsync = noop_fsync, - .splice_read = generic_file_splice_read, + .splice_read = shmem_file_splice_read, .splice_write = iter_file_splice_write, .fallocate = shmem_fallocate, #endif From patchwork Wed Mar 8 14:37:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13165953 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 090F4C64EC4 for ; Wed, 8 Mar 2023 14:38:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 178986B007B; Wed, 8 Mar 2023 09:38:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0FFDB280001; Wed, 8 Mar 2023 09:38:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EBA9A6B007E; Wed, 8 Mar 2023 09:38:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id DD78B6B007B for ; Wed, 8 Mar 2023 09:38:17 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id A37008022D for ; Wed, 8 Mar 2023 14:38:17 +0000 (UTC) X-FDA: 80545986234.26.1D34256 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf17.hostedemail.com (Postfix) with ESMTP id D713140010 for ; Wed, 8 Mar 2023 14:38:15 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=b0fuu4to; spf=pass (imf17.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678286295; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+3Ga7BRmh73UHrzRkOWvEl7XP18FVhE2HGszQBLwN/U=; b=IKFkrOMsb0MeKLhJWyp1tQveFQa2yyBgphzofL/y8UfPwU8rDbg2SraFY73Jdskn2xG/OI 0/D6/0S/esap4WnhVVD58IjG0rDBy3t8yvy189DywySErWY2zbNTtqrv3lJlObvlaetJb+ m5j6KNSPvgD9JIBqU3wL60DfxtpXz8c= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=b0fuu4to; spf=pass (imf17.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678286295; a=rsa-sha256; cv=none; b=ueCBUdhhQ6yoMJolJF35+ZFBAZWSliGCLdCAfz4f2IeyvC5psOJVyhHqklfdZlsIWIFXW4 VgEssz+101C8EymGBpq0FtPTsn9akjxKRS51wRoT6AU32mg4XkEU3IXKMCtMeOI97/iMWV eC7Oc/cvfjDmnxNSTb1+/ib36oh7MJA= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678286295; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+3Ga7BRmh73UHrzRkOWvEl7XP18FVhE2HGszQBLwN/U=; b=b0fuu4tozIAcarxr8PdhQZpFXhabyNdWy5ebmCSfgZ7bunaAGm/0o5nuZhcYNMUtCpPf+E jTgrtG+cY5uSLVjzA1jVKZKGzuAKjyZT+TqN7YmyA4zsqdWls349NUXqQMm97Zly0RUFFd 5GjDKwucBpHz2VWMoAKK2zsa0e5ruGM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-591-Ie2N4SlAMG22ke7XWWlVVQ-1; Wed, 08 Mar 2023 09:38:09 -0500 X-MC-Unique: Ie2N4SlAMG22ke7XWWlVVQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.rdu2.redhat.com [10.11.54.1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 01EBA18E0049; Wed, 8 Mar 2023 14:38:08 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5E61140C83B6; Wed, 8 Mar 2023 14:38:05 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard , Miklos Szeredi , linux-unionfs@vger.kernel.org Subject: [PATCH v16 03/13] overlayfs: Implement splice-read Date: Wed, 8 Mar 2023 14:37:44 +0000 Message-Id: <20230308143754.1976726-4-dhowells@redhat.com> In-Reply-To: <20230308143754.1976726-1-dhowells@redhat.com> References: <20230308143754.1976726-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.1 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: D713140010 X-Stat-Signature: znptih3zdcdyffxxio5psonp5k3ggfff X-HE-Tag: 1678286295-341710 X-HE-Meta: U2FsdGVkX194vBa1kU4oKfjAlPJCZMGHSDp9Xgq9NII/3FLjY/rJxXB3CYBEwmPzAB0LyPUehYhutFFpRFoMyzRWPSYOnun19AZGm4PHeVfINOu3bZ3p2bBBU3e4Ucqzx0NslRKwyV7ZkaPKRjxTrfD5zy/ulquxyvTcODzGdsKukkYG8HVUsToicBNQSpIkYx4Z/I0ybBZm21mYaOEKY4rTVowlGhxDdsF44TxtJT8Eg6C1BxSn2hsKMl5QAqLwEVK1EJi10BESv6iD4So+7DXyKkL9olrrQCiv2bKffMhd/fB8+0J7xVH0Bf5VhNudoHnWsEV5FupATcUQv8i5zCuCz/SjVYW9IPXRWxPYBTjv2eN/KV9FtPzzlRBWL8bPiT9Leqlyq1ekdIM5+zojDGp6xYbaUFOw85yGRmnk20WouHEHG0vDJPgSzN5J7xHnMyDJKW3GbUdVpIl4L7RLd5ob2uVpJPXrJoSwBP9Cqq3g0ipPmUnGC67mxWT6ou3zOyTFHAE+5AIvYKrllgZz+M3bzrELoIo6WHZUGCfO07OPl1Ll6g8aqQUQRIv6D5abyWBrnOUvVhP6/1q8bxUgIvRVC/HrfI5z9c0p6M76WwdZi0lcEm6GjgZ3+gsU0vKaxFiyTmJxrutNZjpFoCbzHqgTO8PGJfGKbbX1bXQOMx7NSIA0KTfaWnafys4C2CPdXJNpaW4Fe445wL28hwuK64noX0qjkIIf6vGoycxnHLVtzQ1R31q+GKi1l5Ff9PAEJ92WuOBzUBBPHeN8YZHz+OgfTzVvXTfSy8Y2864Up1rf3oSB6oLC41RibMM82AlDJaLDro/1AT6EwMslvyojm2qljS386ytlBg38OwAGqNSRaHp56JE9xgm5WF6xxs5RZGSgaP0YorATcXEVPgkMcgK4Xw6yycRSTlmkh0SaI01EP82QKM9VyqCsDJrRa2mloiUvs9iy7SyFOhsnQn2 mEFwR9fp rXQv7pQBTXIrbUKWjzBTYW1Dxk8Hd05PVWt4n6XkvmPPLOjtR22sDnLvGY8fi3f40bv/hp93C5HNSb8nfILdcp+W2RmM43rqgA9FQH4z0+jxEJYaFqT4LdSUmo6wg+pUSIMpFP2P7wA3/PPb9p/coxzRpIGE2YY7NEFwf2wfrZnmOdkE/XM47q95u0NC7CFzSzk0/X1GXxkPKcEmPrUdTptiIBs2+ND4oCeHmtSAC2y7rDt033tK5P6UB9WjThUAHNoKtPtZ2sHWsUbPSiJ2czNdfl301EtijmgEwlacX6i7/JueyTv7bGJ4T/RCUmrEazHsZHlu00LwC9YCgnsjLI0/hR5sKjFwCAWkogA/8ly0UrRH87vKIZM0/wfIOYx9t0PrASXYm+ZkVLmF2VjzW0TOpjubnJ2WB7hYZ5kzk33y/VcJ6HSHWOaiDPUpTdOzTDwXJ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Implement splice-read for overlayfs by passing the request down a layer rather than going through generic_file_splice_read() which is going to be changed to assume that ->read_folio() is present on buffered files. Signed-off-by: David Howells cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: Miklos Szeredi cc: linux-unionfs@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- Notes: ver #15) - Remove redundant FMODE_CAN_ODIRECT check on real file. - Do rw_verify_area() on the real file, not the overlay file. - Fix a file leak. fs/overlayfs/file.c | 33 ++++++++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-) diff --git a/fs/overlayfs/file.c b/fs/overlayfs/file.c index 7c04f033aadd..a12919e9ccba 100644 --- a/fs/overlayfs/file.c +++ b/fs/overlayfs/file.c @@ -419,6 +419,37 @@ static ssize_t ovl_write_iter(struct kiocb *iocb, struct iov_iter *iter) return ret; } +static ssize_t ovl_splice_read(struct file *in, loff_t *ppos, + struct pipe_inode_info *pipe, size_t len, + unsigned int flags) +{ + const struct cred *old_cred; + struct fd real; + ssize_t ret; + + ret = ovl_real_fdget(in, &real); + if (ret) + return ret; + + ret = -EINVAL; + if (!real.file->f_op->splice_read) + goto out_fdput; + + ret = rw_verify_area(READ, real.file, ppos, len); + if (unlikely(ret < 0)) + goto out_fdput; + + old_cred = ovl_override_creds(file_inode(in)->i_sb); + ret = real.file->f_op->splice_read(real.file, ppos, pipe, len, flags); + + revert_creds(old_cred); + ovl_file_accessed(in); +out_fdput: + fdput(real); + + return ret; +} + /* * Calling iter_file_splice_write() directly from overlay's f_op may deadlock * due to lock order inversion between pipe->mutex in iter_file_splice_write() @@ -695,7 +726,7 @@ const struct file_operations ovl_file_operations = { .fallocate = ovl_fallocate, .fadvise = ovl_fadvise, .flush = ovl_flush, - .splice_read = generic_file_splice_read, + .splice_read = ovl_splice_read, .splice_write = ovl_splice_write, .copy_file_range = ovl_copy_file_range, From patchwork Wed Mar 8 14:37:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13165952 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D2A4C678D5 for ; Wed, 8 Mar 2023 14:38:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9B3C96B0078; Wed, 8 Mar 2023 09:38:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9634A6B007B; Wed, 8 Mar 2023 09:38:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 80592280001; Wed, 8 Mar 2023 09:38:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 71C3B6B0078 for ; Wed, 8 Mar 2023 09:38:17 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 45F3E40203 for ; Wed, 8 Mar 2023 14:38:17 +0000 (UTC) X-FDA: 80545986234.27.E94DA69 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf09.hostedemail.com (Postfix) with ESMTP id 7D03B140014 for ; Wed, 8 Mar 2023 14:38:15 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=HV7rsBjn; spf=pass (imf09.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678286295; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2VxSf4XlD6Tq7Rp6h04Xh27xM5QEWu2hXwF5fPrnqQ0=; b=VzyWH3eycU6399aLXkJzkleekuBlJpCuGvcnIZ1+YxA6B3JC8muIpD4vlp03ynvLgDRPxP ZLGxg3dkhqioUFUhLavzf57De0jUw2ulOVLx0bss7ujlZaeHk1WacXndQfBGcf0m7Wywtk iQTHAB6yMdcyAdNOyQIvcAXq6F1hUqk= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=HV7rsBjn; spf=pass (imf09.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678286295; a=rsa-sha256; cv=none; b=jmQ3xCv7/E7tLOVuGV3MVUNR8KPLkIHLEieN9311/GxGrd0/hjoK2auBW+BvDDNN41LwkN BzXQ7DrdL9x590LJaSuJP27AgnYgZg8+v5zGLz3qOhnvRE0vuTHddzItbR4Z0+3Oe3e+8c rM2QG0ZXXbOw/KFtSswi5dY9W3nRPm0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678286294; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2VxSf4XlD6Tq7Rp6h04Xh27xM5QEWu2hXwF5fPrnqQ0=; b=HV7rsBjn15P2H3unth1ccCDTUSQlkFLHR+gZORhpNB2Ju/0NZZW5bHw0EVSfUQMc3JZ/k2 8Uinab6Xwdlu/dbUND47NJrYjsLuT/TyJn2ohhxVhjvs1kPCd54TNAUVdpeLvL2sBlVFf3 pkAy0q1R+pvExGxdgWQdNQzayiGpZJM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-647-K6sCAm2YNpimy6kvSNJiow-1; Wed, 08 Mar 2023 09:38:12 -0500 X-MC-Unique: K6sCAm2YNpimy6kvSNJiow-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2AB61101A55E; Wed, 8 Mar 2023 14:38:11 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9F6222166B26; Wed, 8 Mar 2023 14:38:08 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Jan Harkes , Christoph Hellwig , John Hubbard , coda@cs.cmu.edu, codalist@coda.cs.cmu.edu, linux-unionfs@vger.kernel.org Subject: [PATCH v16 04/13] coda: Implement splice-read Date: Wed, 8 Mar 2023 14:37:45 +0000 Message-Id: <20230308143754.1976726-5-dhowells@redhat.com> In-Reply-To: <20230308143754.1976726-1-dhowells@redhat.com> References: <20230308143754.1976726-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7D03B140014 X-Stat-Signature: spekw35y3m4gqrq7h3xz6bp99jr11eo6 X-HE-Tag: 1678286295-381814 X-HE-Meta: U2FsdGVkX19536Q4FSN/4UXw1n5GD9K/BrJUjnpBuqZUBpoWBC3AmYPVgBR5fjnWXZZLNzAPPRXXQoV5uQzHyVXMmlPbeZFmcMerlVEuQk4uDK0L25r1rlEQWXlFIE/pkge8IsKTLLzvVlH3cZaO1XkayhjHH3GCQH8a0w6Y+9JdPb2S+hMmVqXriUMTOHc/zvHUpfr3maETeKcVPKF75e7rSWoI7/+H8U9fr+TDN8rdoqvESKHsCo3cHJVT6ts8SdH3VhktHk2jtfflJgOGSOUWpcGLV1T/sYicp1hOqRQ3R7pmxtV9N8SJFCk2mjXRTt37MjOCo/7yAJlqk65VYU/FTl7oLF+R3DRcx91dGh7SMI0G6eWvBwhswv5FXD8HT1BdkvJjn2XQ/N88a6mOWac6TSniMIPJFKlaJdYXHJDocI1pgLq1u8QrzYrl1nPoGDV3iMBJcB6/quXlgWLPnNgr1rBk9st/nc8K7ThMZglvBb9RKLsiSqGrMm0l+S0y0NNU1OFqCOy5CfWJbI+XiDPt3Iq9Fng8vBsdRj5J41+7vUQ7BLlSRHT27gUrbXFE39X2HgwjK+lQXC/iyPmKH7s9X934OGKmER/2SehcN8DfTK/M5CyqDpHDewByAVEtFYLmysfPF84sfiscRcb7IE7AOfQXXv3gN2zpLrCtXUqPrHPRrmS2R6/y28cOrOjld8JURb3mzxxulwonhpeRINwg4xXSdJLkRReYK2CrIzbQVv3Ta7luIiMHS8nM/PzKU7FPNqNAC27QZkZuddw07J+PRP3aX9do1R4VDplT3FPxbV2J+tHCbD5LYN8UOoFc78R3K+gW3koGhYXsd7L0blbvqV+JnagJd12z5yuZ0TuMVCx7hzcBE584mXGoyObJx//yAguWcVOVAalOcX3VHoRru+chI7Xm9ZE+lL+2d4RzS/WpECob/X/NdIfg/OnvJWQql/4nVvghYOgRynm Wd5JXFoK xDAFSlkb28YH51AAthSlr2HkTw/pu29glK6nr8ve5JsOBLRNGCcqe/7PmLQs88vb/G5fMaBrgMjS8PMqLniSQZbWSSUcwQ5pIROeNYHaMpHs67uK9B+MSb48BG0F4MZ+b3JGyKS01GLWL2lwEf1Dvq1bnP87WDTGqgmbrflA33zWiuB921DCx86pyZZNXrOADnvVwoN/MYvHGiIcuF2Ann3XlQL3ZXVElsETJqgYjdWLGjhi7N+4Tky366GyWpkZdSV88AA5Yx22wu5nbrZwUH025WVn6KMGbvoMiSwqUEpk7OK64ThcKQ0glPGlX1zROR3ft/738g5F+Y5Co3zIpJaOie80Ct/m4ECt1HEHn0m0MdTwiKLhxYpkEXZgCev6PrIAYibgrzqPM40Zrbk8P+TD4gpP++PhCJ9bQbxcpIsiqWKVMadLB8TStiAHM4bo6gzQa X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Implement splice-read for coda by passing the request down a layer rather than going through generic_file_splice_read() which is going to be changed to assume that ->read_folio() is present on buffered files. Signed-off-by: David Howells Acked-by: Jan Harkes cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: coda@cs.cmu.edu cc: codalist@coda.cs.cmu.edu cc: linux-unionfs@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org --- fs/coda/file.c | 36 +++++++++++++++++++++++++++++++++++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/fs/coda/file.c b/fs/coda/file.c index 3f3c81e6b1ab..33cd7880d30e 100644 --- a/fs/coda/file.c +++ b/fs/coda/file.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include "coda_psdev.h" @@ -94,6 +95,39 @@ coda_file_write_iter(struct kiocb *iocb, struct iov_iter *to) return ret; } +static ssize_t +coda_file_splice_read(struct file *coda_file, loff_t *ppos, + struct pipe_inode_info *pipe, + size_t len, unsigned int flags) +{ + struct inode *coda_inode = file_inode(coda_file); + struct coda_file_info *cfi = coda_ftoc(coda_file); + struct file *in = cfi->cfi_container; + loff_t ki_pos = *ppos; + ssize_t ret; + + if (!in->f_op->splice_read) + return -EINVAL; + + ret = rw_verify_area(READ, in, ppos, len); + if (unlikely(ret < 0)) + return ret; + + ret = venus_access_intent(coda_inode->i_sb, coda_i2f(coda_inode), + &cfi->cfi_access_intent, + len, ki_pos, CODA_ACCESS_TYPE_READ); + if (ret) + goto finish_read; + + ret = in->f_op->splice_read(in, ppos, pipe, len, flags); + +finish_read: + venus_access_intent(coda_inode->i_sb, coda_i2f(coda_inode), + &cfi->cfi_access_intent, + len, ki_pos, CODA_ACCESS_TYPE_READ_FINISH); + return ret; +} + static void coda_vm_open(struct vm_area_struct *vma) { @@ -302,5 +336,5 @@ const struct file_operations coda_file_operations = { .open = coda_open, .release = coda_release, .fsync = coda_fsync, - .splice_read = generic_file_splice_read, + .splice_read = coda_file_splice_read, }; From patchwork Wed Mar 8 14:37:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13165955 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63DDEC678D5 for ; Wed, 8 Mar 2023 14:38:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F33CC6B007E; Wed, 8 Mar 2023 09:38:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E93F86B0080; Wed, 8 Mar 2023 09:38:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE6566B0081; Wed, 8 Mar 2023 09:38:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BAD3C6B007E for ; Wed, 8 Mar 2023 09:38:24 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 8F2991A0A8C for ; Wed, 8 Mar 2023 14:38:24 +0000 (UTC) X-FDA: 80545986528.20.3DD3B19 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf18.hostedemail.com (Postfix) with ESMTP id C2C571C0011 for ; Wed, 8 Mar 2023 14:38:21 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="SPMR90V/"; spf=pass (imf18.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678286301; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gNjp2803YzC3zT/XJu0Jjeq/3mtCoxBl7OitHcYTUso=; b=zW7vgrJR5LkM/zv4CkGAEhjrjYBhMhxdlPCBlNjMbL4hjthTOZ9sqT/Xz6b/HGsnUMzmKC DU0smJdmFvwXnZnGoCEtXs1cmRxeQMhUJq/p0v3saFkW/O5idsWaqlfaOma7f6qDvPFLQu ydb9R+GvagwWNDgNaN4MzkZxI/JNWRg= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="SPMR90V/"; spf=pass (imf18.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678286301; a=rsa-sha256; cv=none; b=Vvk1YghR2DMu5D3H9noj0xsqT3bUoFx6EDjK34HVUmyJdpAL0A845RPw24HZAVfVysArWx wVNB+af9HJyHLVTy3tPh5Lh/YHkny0tEz3IdRHpw9tMIYSITnoVxCWoSMoUvAxJaNuKm+X 9O7KIDUnWV3Y0RLv0wOdf25ZWJtkLRs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678286301; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gNjp2803YzC3zT/XJu0Jjeq/3mtCoxBl7OitHcYTUso=; b=SPMR90V/xvUSygVImf226vX8gIcj2bpHN4PcEgbapj8oRj1EuYaphikKq63MEyVptpsa6V Mp2+paRoj4qkkpUNvbUNgzAO2A/JvpFcYydmcw+0XWjYU4rYGIo06IGiJJuajs10ODLcgT 0DA36ouBa+r8NidGAy00XiN/I1BHlro= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-617-gep4CwD9MCiui4WOcwo1DA-1; Wed, 08 Mar 2023 09:38:15 -0500 X-MC-Unique: gep4CwD9MCiui4WOcwo1DA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 31D8C882821; Wed, 8 Mar 2023 14:38:14 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id BB8352166B2A; Wed, 8 Mar 2023 14:38:11 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard , Miklos Szeredi , Arnd Bergmann , Greg Kroah-Hartman Subject: [PATCH v16 05/13] tty, proc, kernfs, random: Use direct_splice_read() Date: Wed, 8 Mar 2023 14:37:46 +0000 Message-Id: <20230308143754.1976726-6-dhowells@redhat.com> In-Reply-To: <20230308143754.1976726-1-dhowells@redhat.com> References: <20230308143754.1976726-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Queue-Id: C2C571C0011 X-Stat-Signature: fe5man6hye59ne8ixknubeo77treh67w X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1678286301-216764 X-HE-Meta: U2FsdGVkX1+4wTNlZcleNqSAP32sml37eYdJ0L6ZvRM8dp2WXaGtCHtrlmk+BGtb6Kq+rpc2f8/h+r9pASw8VrxeTBGsEl2A0MwFHisIPfg200epRHGwfHM293kUbV1ZvQcEGs2xHPVPlDPIOzctjGJbP8Avme5n+VCM3MnJ2Wh7XYto01P+B2Da5JvOJ8R5auzXYEe9ywWHC3fNSr9Ea8/BAG3b9EHt/qSt0OLGAmIPldYOhD7ZPu23skr/DWTBW4YJ4X5xz2bvdX6ruldkDzWUuEFpzuOfhnW4FekeOHBJA0Xkdkfn2seY+7BHzG7fKoSCQEW2UyRT9AMnwThilCg171LKcf71sNo1sYij6y/GpMsMf8FCYMWOlD4XiXScWIQoba00IKa0mj4zIXCPtz8HeXQGYLz8/LvvR5tdYaqVRQmQxPOzUlvaame6XN2q5VZPU/gS/o4bgruBZA4U9REG0pTzA2XZh9TXv3FSwVEzaQuINb8JFkA0Awy9h0r6LUHkYjHV3fNXGetUWwpZrFc7zntYNjL2CCoQ56NPfLOrEY3PGwCXBmIj40eHwRZODx6IBOqz9DRs8NZqhz1oAT+JN+CsyonhEguEniX4+x5f1NQa7jUiPdD6y6HPozHZOFNoCJhTqOvJLT/8npurqSACO6RwI3zuuhUonQZVHki6eGAr7IRyGFuo2Ebl32IdnHP1ocK1ChNqnD4Q1c4EYJc8kbjfMIO5bot7xmJzaNVsqDegUm7SKTaivIshSe0b3YneeQXQQr22t6q+piHS27SW946QqsB996eZtVTwvFt4ZB5UOuIVA+Au5uZJQLFNjlK8HuRKOg+lWhoaz3p4oFPzQrFvTDl8nYWTSFj2T0hVbsGjGF2BAGjfy5Ean1hI91F3K3K2+aHSLii2mPhyeX+1D+yqMlaFTlA5Btqh67+x41Dh+O6DQ/E9dAu2T3T6737TCJQPFaFcKNLmG81 /60fh4DM MQUtfAecetdY3RcWq6MmRPGSeCiH2qmzxVAFrDI0W2zAB1iCqU3YtbquL/rD/7oYpEgwFYSb2HEnMRkrnUhPdaco+27OFTcbv9BvGY9+JmR2PwX5xIYkRHJKem842pAlXZTfjIzMUYmX84IoYKtvZ16MX/ZrbB9iu9eRyI3AtDhysPzpBEOfEzEB2tzeyXXbFwMyMskRqHXtVf3olgjGfU6zARt4/Z84li1dgFnlXD2N8fwP3rwbwIMMz6QCokMcNoLbSnVgLioLvj0y98VCo86VdaQwFj6RQYX63CmZ48tXSAzBOJ/nbnSxS4QGOPrDK+5uxDs2IJobcK2YjdULJ4X1ruFUMq69Np/e83PtX1X4s+pY9MerGZ/DDpv8QYSeczI2LB5OiOQB7/apprVbmtmXjVqCV9r3O6yj9kSUZEoWf7heRB6ALp87hitysqrz2bbRzaKSpmillaElCoj5luVarphwH8gmfh2dXxdDus+6ahIptye69yvtbY/bnDS/HFSeIup0W5R6Sv7CeJhHYEvjmUcnHQWS5QW69OO0+27GK5tBzRU4xWvB0eg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use direct_splice_read() for tty, procfs, kernfs and random files rather than going through generic_file_splice_read() as they just copy the file into the output buffer and don't splice pages. This avoids the need for them to have a ->read_folio() to satisfy filemap_splice_read(). Signed-off-by: David Howells cc: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: John Hubbard cc: David Hildenbrand cc: Matthew Wilcox cc: Miklos Szeredi cc: Arnd Bergmann cc: Greg Kroah-Hartman cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org cc: linux-mm@kvack.org Acked-by: Greg Kroah-Hartman --- drivers/char/random.c | 4 ++-- drivers/tty/tty_io.c | 4 ++-- fs/kernfs/file.c | 2 +- fs/proc/inode.c | 4 ++-- fs/proc/proc_sysctl.c | 2 +- fs/proc_namespace.c | 6 +++--- 6 files changed, 11 insertions(+), 11 deletions(-) diff --git a/drivers/char/random.c b/drivers/char/random.c index ce3ccd172cc8..792713616ba8 100644 --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -1546,7 +1546,7 @@ const struct file_operations random_fops = { .compat_ioctl = compat_ptr_ioctl, .fasync = random_fasync, .llseek = noop_llseek, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, }; @@ -1557,7 +1557,7 @@ const struct file_operations urandom_fops = { .compat_ioctl = compat_ptr_ioctl, .fasync = random_fasync, .llseek = noop_llseek, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, }; diff --git a/drivers/tty/tty_io.c b/drivers/tty/tty_io.c index 36fb945fdad4..9d117e579dfb 100644 --- a/drivers/tty/tty_io.c +++ b/drivers/tty/tty_io.c @@ -466,7 +466,7 @@ static const struct file_operations tty_fops = { .llseek = no_llseek, .read_iter = tty_read, .write_iter = tty_write, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, .poll = tty_poll, .unlocked_ioctl = tty_ioctl, @@ -481,7 +481,7 @@ static const struct file_operations console_fops = { .llseek = no_llseek, .read_iter = tty_read, .write_iter = redirected_tty_write, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, .poll = tty_poll, .unlocked_ioctl = tty_ioctl, diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c index e4a50e4ff0d2..9d23b8141db7 100644 --- a/fs/kernfs/file.c +++ b/fs/kernfs/file.c @@ -1011,7 +1011,7 @@ const struct file_operations kernfs_file_fops = { .release = kernfs_fop_release, .poll = kernfs_fop_poll, .fsync = noop_fsync, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, }; diff --git a/fs/proc/inode.c b/fs/proc/inode.c index f495fdb39151..711f12706469 100644 --- a/fs/proc/inode.c +++ b/fs/proc/inode.c @@ -591,7 +591,7 @@ static const struct file_operations proc_iter_file_ops = { .llseek = proc_reg_llseek, .read_iter = proc_reg_read_iter, .write = proc_reg_write, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .poll = proc_reg_poll, .unlocked_ioctl = proc_reg_unlocked_ioctl, .mmap = proc_reg_mmap, @@ -617,7 +617,7 @@ static const struct file_operations proc_reg_file_ops_compat = { static const struct file_operations proc_iter_file_ops_compat = { .llseek = proc_reg_llseek, .read_iter = proc_reg_read_iter, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .write = proc_reg_write, .poll = proc_reg_poll, .unlocked_ioctl = proc_reg_unlocked_ioctl, diff --git a/fs/proc/proc_sysctl.c b/fs/proc/proc_sysctl.c index 5851eb5bc726..e49f99657d1c 100644 --- a/fs/proc/proc_sysctl.c +++ b/fs/proc/proc_sysctl.c @@ -869,7 +869,7 @@ static const struct file_operations proc_sys_file_operations = { .poll = proc_sys_poll, .read_iter = proc_sys_read, .write_iter = proc_sys_write, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .splice_write = iter_file_splice_write, .llseek = default_llseek, }; diff --git a/fs/proc_namespace.c b/fs/proc_namespace.c index 846f9455ae22..492abbbeff5e 100644 --- a/fs/proc_namespace.c +++ b/fs/proc_namespace.c @@ -324,7 +324,7 @@ static int mountstats_open(struct inode *inode, struct file *file) const struct file_operations proc_mounts_operations = { .open = mounts_open, .read_iter = seq_read_iter, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .llseek = seq_lseek, .release = mounts_release, .poll = mounts_poll, @@ -333,7 +333,7 @@ const struct file_operations proc_mounts_operations = { const struct file_operations proc_mountinfo_operations = { .open = mountinfo_open, .read_iter = seq_read_iter, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .llseek = seq_lseek, .release = mounts_release, .poll = mounts_poll, @@ -342,7 +342,7 @@ const struct file_operations proc_mountinfo_operations = { const struct file_operations proc_mountstats_operations = { .open = mountstats_open, .read_iter = seq_read_iter, - .splice_read = generic_file_splice_read, + .splice_read = direct_splice_read, .llseek = seq_lseek, .release = mounts_release, }; From patchwork Wed Mar 8 14:37:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13165954 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 616D6C64EC4 for ; Wed, 8 Mar 2023 14:38:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 03DF36B007D; Wed, 8 Mar 2023 09:38:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F0972280001; Wed, 8 Mar 2023 09:38:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA99E6B0080; Wed, 8 Mar 2023 09:38:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id CAFEB6B007D for ; Wed, 8 Mar 2023 09:38:23 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 9D4A780242 for ; Wed, 8 Mar 2023 14:38:23 +0000 (UTC) X-FDA: 80545986486.30.F5A0193 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf06.hostedemail.com (Postfix) with ESMTP id E9108180015 for ; Wed, 8 Mar 2023 14:38:21 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=D5BzN3Sv; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678286302; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=t6OAyEYn0daMnp8L9pWL3EjurZ3FwJIkgFcgCsYP8ro=; b=yDr1w4qGWIT2XQnKQi0OmHjwyXcba0uqsXuFKm4JP4/laRluHoOVXcSUKdwr+xq38TBHv6 S7qRRRVcBPuJArOyWcWxoalyzRDtGu7wOl4ihMnsaow2t/MpvkU0Kz1mLYey2mi8MdkvEU EHEL0kyD/9ulNAWXEFG4EaeiZAftZXQ= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=D5BzN3Sv; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678286302; a=rsa-sha256; cv=none; b=GdGlKDjWIwhqB05wUdPcYDIBLm0oyfy2mgOIoA+g3uaKis9caepKx3LXyuHiRoscyYnYmu JRk6EBnNZyqDohj1cFOlA9C5+HhisqqsqnZfhTEyxQ9d/6QG6TSoqF6jsoyroPsA7C3BUF AkwY8Qu6JKEKPaAe6DG4nl23DVuaeR0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678286301; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t6OAyEYn0daMnp8L9pWL3EjurZ3FwJIkgFcgCsYP8ro=; b=D5BzN3SvnWLIMiWlIziLC31sMPI0vptz4Q3jvEiQ+kg2tdXNkpMcj9XKDRE5KCK6YSuY6K Xavb7koOY8ruLfyn95cxnPN336VDaWhEX0AnoL34ur0c7Njq6FGvjBMh2lNeF9WCe5k9Yw EBf2oX++5ARJBjNX0ouiTVBJMURuylk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-373-z2E0ldL8PlCMxbtTf0W6YQ-1; Wed, 08 Mar 2023 09:38:18 -0500 X-MC-Unique: z2E0ldL8PlCMxbtTf0W6YQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 135FD802DD4; Wed, 8 Mar 2023 14:38:17 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id CF5211121314; Wed, 8 Mar 2023 14:38:14 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Steve French , Christoph Hellwig , John Hubbard , linux-cifs@vger.kernel.org Subject: [PATCH v16 06/13] splice: Do splice read from a file without using ITER_PIPE Date: Wed, 8 Mar 2023 14:37:47 +0000 Message-Id: <20230308143754.1976726-7-dhowells@redhat.com> In-Reply-To: <20230308143754.1976726-1-dhowells@redhat.com> References: <20230308143754.1976726-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Stat-Signature: 6cskyh1hswy59pq5ar3yhz3hs7fu63r9 X-Rspam-User: X-Rspamd-Queue-Id: E9108180015 X-Rspamd-Server: rspam06 X-HE-Tag: 1678286301-325654 X-HE-Meta: U2FsdGVkX1/MrIw9zve1qPRXlISAP/yOcP2bZd37zDple2QEuqyj/oXETnZOXC6/Ku0V6gbabD1LEvxpKaA3e97PFSYkvoH59Z+An2of5DS22XJsLMCbnCWh39eG2dEiaD6+QmLzZeY8uBMiHECjLrV4cuyh4fOgoxnkQZzn0Wcii2uGcKIF3hl1S1Qk6iY5sWx6MhwgH19vXdA6F5X0pGvwFkBDefeehYx/VrATsmeJ+AQXdDvvnqUT1UxioC51Zhsc6+ZC1v0gDmD7x2u8u35+oaWBDuc9zqAHRQXnsmpAgoh3CJpkIS+oy2fKugTCQ4iSGqFoXxlflfSeaWJt8hIh3y9G1TLHrq60JC/OO1aHAUYy8L835Oe89uKhCHg5Kp72y1mGYgkcU9HDyFNE2kh6umtN2pE/e2gXFSLJLR4ZpWOEh6r5DIor0iA6UjCCLTFJZWyeab0acJuNVJ5GW3YiNkAeaA5JmR9VLNssa/bC39hLId2PaLyYku8GFbGqKs8gdmYzc+AO+NaruzJG0KLahEj7W+XubGYSgcaR+Qm/nByWSoqJBq+2xI2SKbSftr/qWyryZdpqEsORQ5JPpyt8M4VHERy2vR7H9+gjOk0pyNUgdCcfnWZKwRRrbep84arlDiUWdpwRMXf2wkpXY3ixT3BMpPnZTzU0C+DKCiVq0h93PkylC914IiVRSwIsujKZCWSpFaZBXxjyG2ikaxTu4CFNN9GDFTo8BCk2ADb9aJjPKrmuTG5HJR5S2ifzdxLkpeLCpnAgSfFGWGFHLziohvbcecFT98R+FHHStOXPNlbQxibUWE6pxTgq6Y9nq6kIR027D6gWx0pEUzoqp0q+4A5uxmxPd5oQzoiwWJx47Hru5nymtvmv+zcve9pKPUipUkJYWlSDxDJSMh/d1d4qS6VlDBTHQywn0goVL0z18b1vGGK5rbopFV/cntWGABFgo9Natpg+cV+Qd9+ /o5GjvQf bu1ClsND0OcJh/ycra8QMaAlicVCd8IyvH2KDRMLRUnZO1dwC0b4ULKmmI+QVdURW18jwfJEBX3xEzxc3Ty8sa30QKFdtSHktzeDtulmQLT5bTVOEZBRYkF8HKf4uvsVhZbxiVnOTYP4oTfkE4H0JOsQZvgg7Vve7iRMURFDfUawHDzFZq9pYDUMtQjPdR4TjR3xnxA2b50pgrHJVWGKCtm6gHHeqZFt5eruR6sJIBa3wx8zL2ZrpFeKDsZCjWNrHqD7r0csXR/0SOIzM/vLWFlnNdJA0ZoEUWGkpkq6uifxrTE2nBx3u4zi7wNGsQsfqi/juA99yc+6gjgJMyYbbfw/RLvNNdMW9dAiSYsASz8iXmx3Pc7BF4HI6PF3QyNk15ZFKGdH/p4YqaKhq793T4h0yAgaEieKYp+E1n+yJlNUR6vZC0kMAtxlfdROhK10VZeZFDFI92Saj5L0ocL5lGmu3+DyeIjmCk0ccYh5KmzQa4T4jOM3EKpJL+kVFOlCdx7KOSRRxPozE2OD4xKG9hrx9VQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make generic_file_splice_read() use filemap_splice_read() and direct_splice_read() rather than using an ITER_PIPE and call_read_iter(). Make cifs use generic_file_splice_read() rather than doing it for itself. Unexport filemap_splice_read(). With this, ITER_PIPE is no longer used. Signed-off-by: David Howells cc: Jens Axboe cc: Steve French cc: Christoph Hellwig cc: Al Viro cc: David Hildenbrand cc: John Hubbard cc: linux-mm@kvack.org cc: linux-block@vger.kernel.org cc: linux-cifs@vger.kernel.org cc: linux-fsdevel@vger.kernel.org --- fs/cifs/cifsfs.c | 8 ++++---- fs/cifs/cifsfs.h | 3 --- fs/cifs/file.c | 16 ---------------- fs/splice.c | 30 +++++++----------------------- mm/filemap.c | 1 - 5 files changed, 11 insertions(+), 47 deletions(-) diff --git a/fs/cifs/cifsfs.c b/fs/cifs/cifsfs.c index cbcf210d56e4..ba963a26cb19 100644 --- a/fs/cifs/cifsfs.c +++ b/fs/cifs/cifsfs.c @@ -1359,7 +1359,7 @@ const struct file_operations cifs_file_ops = { .fsync = cifs_fsync, .flush = cifs_flush, .mmap = cifs_file_mmap, - .splice_read = cifs_splice_read, + .splice_read = generic_file_splice_read, .splice_write = iter_file_splice_write, .llseek = cifs_llseek, .unlocked_ioctl = cifs_ioctl, @@ -1379,7 +1379,7 @@ const struct file_operations cifs_file_strict_ops = { .fsync = cifs_strict_fsync, .flush = cifs_flush, .mmap = cifs_file_strict_mmap, - .splice_read = cifs_splice_read, + .splice_read = generic_file_splice_read, .splice_write = iter_file_splice_write, .llseek = cifs_llseek, .unlocked_ioctl = cifs_ioctl, @@ -1417,7 +1417,7 @@ const struct file_operations cifs_file_nobrl_ops = { .fsync = cifs_fsync, .flush = cifs_flush, .mmap = cifs_file_mmap, - .splice_read = cifs_splice_read, + .splice_read = generic_file_splice_read, .splice_write = iter_file_splice_write, .llseek = cifs_llseek, .unlocked_ioctl = cifs_ioctl, @@ -1435,7 +1435,7 @@ const struct file_operations cifs_file_strict_nobrl_ops = { .fsync = cifs_strict_fsync, .flush = cifs_flush, .mmap = cifs_file_strict_mmap, - .splice_read = cifs_splice_read, + .splice_read = generic_file_splice_read, .splice_write = iter_file_splice_write, .llseek = cifs_llseek, .unlocked_ioctl = cifs_ioctl, diff --git a/fs/cifs/cifsfs.h b/fs/cifs/cifsfs.h index 71fe0a0a7992..8b239854e590 100644 --- a/fs/cifs/cifsfs.h +++ b/fs/cifs/cifsfs.h @@ -100,9 +100,6 @@ extern ssize_t cifs_strict_readv(struct kiocb *iocb, struct iov_iter *to); extern ssize_t cifs_user_writev(struct kiocb *iocb, struct iov_iter *from); extern ssize_t cifs_direct_writev(struct kiocb *iocb, struct iov_iter *from); extern ssize_t cifs_strict_writev(struct kiocb *iocb, struct iov_iter *from); -extern ssize_t cifs_splice_read(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, size_t len, - unsigned int flags); extern int cifs_flock(struct file *pfile, int cmd, struct file_lock *plock); extern int cifs_lock(struct file *, int, struct file_lock *); extern int cifs_fsync(struct file *, loff_t, loff_t, int); diff --git a/fs/cifs/file.c b/fs/cifs/file.c index 4d4a2d82636d..321f9b7c84c9 100644 --- a/fs/cifs/file.c +++ b/fs/cifs/file.c @@ -5066,19 +5066,3 @@ const struct address_space_operations cifs_addr_ops_smallbuf = { .launder_folio = cifs_launder_folio, .migrate_folio = filemap_migrate_folio, }; - -/* - * Splice data from a file into a pipe. - */ -ssize_t cifs_splice_read(struct file *in, loff_t *ppos, - struct pipe_inode_info *pipe, size_t len, - unsigned int flags) -{ - if (unlikely(*ppos >= file_inode(in)->i_sb->s_maxbytes)) - return 0; - if (unlikely(!len)) - return 0; - if (in->f_flags & O_DIRECT) - return direct_splice_read(in, ppos, pipe, len, flags); - return filemap_splice_read(in, ppos, pipe, len, flags); -} diff --git a/fs/splice.c b/fs/splice.c index abd21a455a2b..9c6adcd0a2e9 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -387,29 +387,13 @@ ssize_t generic_file_splice_read(struct file *in, loff_t *ppos, struct pipe_inode_info *pipe, size_t len, unsigned int flags) { - struct iov_iter to; - struct kiocb kiocb; - int ret; - - iov_iter_pipe(&to, ITER_DEST, pipe, len); - init_sync_kiocb(&kiocb, in); - kiocb.ki_pos = *ppos; - ret = call_read_iter(in, &kiocb, &to); - if (ret > 0) { - *ppos = kiocb.ki_pos; - file_accessed(in); - } else if (ret < 0) { - /* free what was emitted */ - pipe_discard_from(pipe, to.start_head); - /* - * callers of ->splice_read() expect -EAGAIN on - * "can't put anything in there", rather than -EFAULT. - */ - if (ret == -EFAULT) - ret = -EAGAIN; - } - - return ret; + if (unlikely(*ppos >= file_inode(in)->i_sb->s_maxbytes)) + return 0; + if (unlikely(!len)) + return 0; + if (in->f_flags & O_DIRECT) + return direct_splice_read(in, ppos, pipe, len, flags); + return filemap_splice_read(in, ppos, pipe, len, flags); } EXPORT_SYMBOL(generic_file_splice_read); diff --git a/mm/filemap.c b/mm/filemap.c index 2723104cc06a..3a93515ae2ed 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2967,7 +2967,6 @@ ssize_t filemap_splice_read(struct file *in, loff_t *ppos, return total_spliced ? total_spliced : error; } -EXPORT_SYMBOL(filemap_splice_read); static inline loff_t folio_seek_hole_data(struct xa_state *xas, struct address_space *mapping, struct folio *folio, From patchwork Wed Mar 8 14:37:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13165956 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FCF1C678D5 for ; Wed, 8 Mar 2023 14:38:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 128A96B0080; Wed, 8 Mar 2023 09:38:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0D9916B0081; Wed, 8 Mar 2023 09:38:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E93F86B0082; Wed, 8 Mar 2023 09:38:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D8ED76B0080 for ; Wed, 8 Mar 2023 09:38:28 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 956271C5FFD for ; Wed, 8 Mar 2023 14:38:28 +0000 (UTC) X-FDA: 80545986696.07.86ABC52 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf11.hostedemail.com (Postfix) with ESMTP id A55CF40008 for ; Wed, 8 Mar 2023 14:38:26 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KMG6w74U; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678286306; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XsRpjIP88f7UuB0TiBadjal83kAgF2un+t2WHwzlM/w=; b=qoy5Lpmv7FzdYb96hvKUbujZB5HxLST57NjczjATLOQ3Occizo3lzfzqPjXzY8gI82MQ0l Xf0hPyapQoGT8zHvGIQSFz3jaGCE/CQj+/KUvJd4VR9Ovlwbuuev//7CrfcIMSeluzh0SR 1cq+9SfnnSdrTKz0+BQA9YY1I23bYnE= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=KMG6w74U; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678286306; a=rsa-sha256; cv=none; b=IsQ0Afs0jGkps8OStZSoE8FaOCAUnRK4cVp1jl9AFH2OrePVbbMqd3hBvvogjnpooZNLgN 8TIAJfPi35c+a/wLuexwv168IsbNIUPcsQfWuoWoIiHbXYCxA6vBgRHa4CZlMmO76d4M5e QY7woLlie0X1O4MkWbOVEVdi3CYP1eQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678286306; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XsRpjIP88f7UuB0TiBadjal83kAgF2un+t2WHwzlM/w=; b=KMG6w74U446V2PqskT2+hkK1y8CGvdTEKUvKYTp7ERsF5hFzsXiuBC4z2id7R91Bgxn/3S FB7eoWoaVspcx8xQlbtMEKGYMKlaxCIkIbPuxkTbARq3hQh4g0tQih2FS6v73YBS4OFevl lykG5SBc1XD+euygwjPMSFMRlGBEEu8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-324-tX-bSsmGPty3GOtIlEwCIQ-1; Wed, 08 Mar 2023 09:38:20 -0500 X-MC-Unique: tX-bSsmGPty3GOtIlEwCIQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CF3C2811E6E; Wed, 8 Mar 2023 14:38:19 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id AB7E6C15BA0; Wed, 8 Mar 2023 14:38:17 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v16 07/13] iov_iter: Kill ITER_PIPE Date: Wed, 8 Mar 2023 14:37:48 +0000 Message-Id: <20230308143754.1976726-8-dhowells@redhat.com> In-Reply-To: <20230308143754.1976726-1-dhowells@redhat.com> References: <20230308143754.1976726-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: A55CF40008 X-Stat-Signature: zpbgbej1jstuu19kq9tgwfuszouy5ziy X-HE-Tag: 1678286306-157991 X-HE-Meta: U2FsdGVkX18LD5/Yky4xR85q4+zyGpVls5digfAfqlvv/Q7FqJxhO+7cdcYZGBShcAIIgWhcE2Xyh2RZxbFB28KzfAcnFt1J7kXBhh1++YtOFoqTMVS850TBaKCTvCDQ7qtO29ZByPlDbeeJQn7EAFAjNPU9N2NMV/EigtdAk2eGJ4YlxaULXvxxv3WI7VGjWKY9jOXt+zPqZC/A+KYpNRiDxR0C0+GyBvxn9sPJGJdD0Ye4v22hN6+HECVCV+8CDD4qc/spTEAez1j55623eYHqSNPYOa8DT31mFQ4DWwp80dycWozddpmUMsR38WhtVUxon4OgfHtxJQIPQ/9snKRx+a9qfc9UsjvxXkoCkgAuyVtn6mA16NlHUN3zfOLa0oVO6WQYEomdDUInj9bWHLGVWpVp50MoMRR7OsesMy8MnFfEDFa0uSGybOiOiqNu93FdhsEiIr7flIwjOOM9jbsWt5i//jb8n814ZH22UCvbwQdxITh+pgmWujQlyOdLdDjDCecf3DUjcmF5H/c+fkwBAWEGns831RDumk/q7WNeHPoYhxotlkCEh+pHbMZtxSyVCdeI0Pr/uF3nHkP223emQJ71ktKZRQxIlH0+UhH8vojZHUjwiyfS50/3GxCJqNaXU3PS+pY6DLfpNpN7rmMQ9lL2GiIZN/zuQeqoHxWxNnJm0cz+uG3LxQ1hcIASt8EFcRRArLR2Xe0EnHEp6pBz/FuoCeH111PBpB8yBd0rzICH9O0pddv+KDe1X97DvTN0PQbUJ5nu8Pok/BblB2kUfyguAThHgUpk0b7wD3yRMrMyvaAKi0WrS8Ncl9gHEMBM1yvQL8asbMuDHPuGOZrek7+RVD0jkYxL6DdNob73Ui336J0/4/IW35nA290CFnP9cwKRN5ZX8oMwNVsgtll14MV6W9xXGtMgsBQZakDoLJbIs7RmJBzreVnOKgtcCWB4tIUGiLQBqGs17f7 Zgz+r00b ISpVYBY4crpZ0w4O9GQqzwmVubwrHYYro5tEeeB45fA/JpeT6sIKDaxZAvOvhV7BjbllAL2yv9RtVqHiO+d1UJl1FrtilylATsxMdzN7myo4KOW8hDe/0tgWvaa6x63Wsyl+0LmwZ/NliJSKeEzjIBrvtQsJ/VFcMQqW8A1ZWu2kUDb20D3oetpHZ0WBK+wiFFsSTLQL4baL/7u1uvqR4bkTXeLgNAVYFjvHHzLehjHu4V+3tWtl3Uh7f78NsVQBGalpw1IuW2Pvb0phh1+EfmbhnZbnpaZYSC48b+viHVsXt5o3wNCDeuIxmcy+CKstp1muNBv4SwwCzF+XN5pwJ+kLm01UccbtIJRMPzT5a+wcNP6jHjG/jxGT79udYr8m/bAA7+0IlVJ4OXx9RMECImm+W/5XZzNYFE6qV6OaT86J1f/oT8j0Weeu+rMpaB5m1P2Le X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The ITER_PIPE-type iterator was only used for generic_file_splice_read(), but that has now been switched to either pull pages directly from the pagecache for buffered file splice-reads or to use ITER_BVEC instead for O_DIRECT file splice-reads. This leaves ITER_PIPE unused - so remove it. Signed-off-by: David Howells Reviewed-by: Christoph Hellwig cc: Jens Axboe cc: Al Viro cc: David Hildenbrand cc: John Hubbard cc: linux-mm@kvack.org cc: linux-block@vger.kernel.org cc: linux-fsdevel@vger.kernel.org --- include/linux/uio.h | 14 -- lib/iov_iter.c | 429 +------------------------------------------- mm/filemap.c | 3 +- 3 files changed, 4 insertions(+), 442 deletions(-) diff --git a/include/linux/uio.h b/include/linux/uio.h index 27e3fd942960..74598426edb4 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -11,7 +11,6 @@ #include struct page; -struct pipe_inode_info; typedef unsigned int __bitwise iov_iter_extraction_t; @@ -25,7 +24,6 @@ enum iter_type { ITER_IOVEC, ITER_KVEC, ITER_BVEC, - ITER_PIPE, ITER_XARRAY, ITER_DISCARD, ITER_UBUF, @@ -55,15 +53,10 @@ struct iov_iter { const struct kvec *kvec; const struct bio_vec *bvec; struct xarray *xarray; - struct pipe_inode_info *pipe; void __user *ubuf; }; union { unsigned long nr_segs; - struct { - unsigned int head; - unsigned int start_head; - }; loff_t xarray_start; }; }; @@ -101,11 +94,6 @@ static inline bool iov_iter_is_bvec(const struct iov_iter *i) return iov_iter_type(i) == ITER_BVEC; } -static inline bool iov_iter_is_pipe(const struct iov_iter *i) -{ - return iov_iter_type(i) == ITER_PIPE; -} - static inline bool iov_iter_is_discard(const struct iov_iter *i) { return iov_iter_type(i) == ITER_DISCARD; @@ -247,8 +235,6 @@ void iov_iter_kvec(struct iov_iter *i, unsigned int direction, const struct kvec unsigned long nr_segs, size_t count); void iov_iter_bvec(struct iov_iter *i, unsigned int direction, const struct bio_vec *bvec, unsigned long nr_segs, size_t count); -void iov_iter_pipe(struct iov_iter *i, unsigned int direction, struct pipe_inode_info *pipe, - size_t count); void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count); void iov_iter_xarray(struct iov_iter *i, unsigned int direction, struct xarray *xarray, loff_t start, size_t count); diff --git a/lib/iov_iter.c b/lib/iov_iter.c index 274014e4eafe..fad95e4cf372 100644 --- a/lib/iov_iter.c +++ b/lib/iov_iter.c @@ -14,8 +14,6 @@ #include #include -#define PIPE_PARANOIA /* for now */ - /* covers ubuf and kbuf alike */ #define iterate_buf(i, n, base, len, off, __p, STEP) { \ size_t __maybe_unused off = 0; \ @@ -186,150 +184,6 @@ static int copyin(void *to, const void __user *from, size_t n) return res; } -#ifdef PIPE_PARANOIA -static bool sanity(const struct iov_iter *i) -{ - struct pipe_inode_info *pipe = i->pipe; - unsigned int p_head = pipe->head; - unsigned int p_tail = pipe->tail; - unsigned int p_occupancy = pipe_occupancy(p_head, p_tail); - unsigned int i_head = i->head; - unsigned int idx; - - if (i->last_offset) { - struct pipe_buffer *p; - if (unlikely(p_occupancy == 0)) - goto Bad; // pipe must be non-empty - if (unlikely(i_head != p_head - 1)) - goto Bad; // must be at the last buffer... - - p = pipe_buf(pipe, i_head); - if (unlikely(p->offset + p->len != abs(i->last_offset))) - goto Bad; // ... at the end of segment - } else { - if (i_head != p_head) - goto Bad; // must be right after the last buffer - } - return true; -Bad: - printk(KERN_ERR "idx = %d, offset = %d\n", i_head, i->last_offset); - printk(KERN_ERR "head = %d, tail = %d, buffers = %d\n", - p_head, p_tail, pipe->ring_size); - for (idx = 0; idx < pipe->ring_size; idx++) - printk(KERN_ERR "[%p %p %d %d]\n", - pipe->bufs[idx].ops, - pipe->bufs[idx].page, - pipe->bufs[idx].offset, - pipe->bufs[idx].len); - WARN_ON(1); - return false; -} -#else -#define sanity(i) true -#endif - -static struct page *push_anon(struct pipe_inode_info *pipe, unsigned size) -{ - struct page *page = alloc_page(GFP_USER); - if (page) { - struct pipe_buffer *buf = pipe_buf(pipe, pipe->head++); - *buf = (struct pipe_buffer) { - .ops = &default_pipe_buf_ops, - .page = page, - .offset = 0, - .len = size - }; - } - return page; -} - -static void push_page(struct pipe_inode_info *pipe, struct page *page, - unsigned int offset, unsigned int size) -{ - struct pipe_buffer *buf = pipe_buf(pipe, pipe->head++); - *buf = (struct pipe_buffer) { - .ops = &page_cache_pipe_buf_ops, - .page = page, - .offset = offset, - .len = size - }; - get_page(page); -} - -static inline int last_offset(const struct pipe_buffer *buf) -{ - if (buf->ops == &default_pipe_buf_ops) - return buf->len; // buf->offset is 0 for those - else - return -(buf->offset + buf->len); -} - -static struct page *append_pipe(struct iov_iter *i, size_t size, - unsigned int *off) -{ - struct pipe_inode_info *pipe = i->pipe; - int offset = i->last_offset; - struct pipe_buffer *buf; - struct page *page; - - if (offset > 0 && offset < PAGE_SIZE) { - // some space in the last buffer; add to it - buf = pipe_buf(pipe, pipe->head - 1); - size = min_t(size_t, size, PAGE_SIZE - offset); - buf->len += size; - i->last_offset += size; - i->count -= size; - *off = offset; - return buf->page; - } - // OK, we need a new buffer - *off = 0; - size = min_t(size_t, size, PAGE_SIZE); - if (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) - return NULL; - page = push_anon(pipe, size); - if (!page) - return NULL; - i->head = pipe->head - 1; - i->last_offset = size; - i->count -= size; - return page; -} - -static size_t copy_page_to_iter_pipe(struct page *page, size_t offset, size_t bytes, - struct iov_iter *i) -{ - struct pipe_inode_info *pipe = i->pipe; - unsigned int head = pipe->head; - - if (unlikely(bytes > i->count)) - bytes = i->count; - - if (unlikely(!bytes)) - return 0; - - if (!sanity(i)) - return 0; - - if (offset && i->last_offset == -offset) { // could we merge it? - struct pipe_buffer *buf = pipe_buf(pipe, head - 1); - if (buf->page == page) { - buf->len += bytes; - i->last_offset -= bytes; - i->count -= bytes; - return bytes; - } - } - if (pipe_full(pipe->head, pipe->tail, pipe->max_usage)) - return 0; - - push_page(pipe, page, offset, bytes); - i->last_offset = -(offset + bytes); - i->head = head; - i->count -= bytes; - return bytes; -} - /* * fault_in_iov_iter_readable - fault in iov iterator for reading * @i: iterator @@ -433,46 +287,6 @@ void iov_iter_init(struct iov_iter *i, unsigned int direction, } EXPORT_SYMBOL(iov_iter_init); -// returns the offset in partial buffer (if any) -static inline unsigned int pipe_npages(const struct iov_iter *i, int *npages) -{ - struct pipe_inode_info *pipe = i->pipe; - int used = pipe->head - pipe->tail; - int off = i->last_offset; - - *npages = max((int)pipe->max_usage - used, 0); - - if (off > 0 && off < PAGE_SIZE) { // anon and not full - (*npages)++; - return off; - } - return 0; -} - -static size_t copy_pipe_to_iter(const void *addr, size_t bytes, - struct iov_iter *i) -{ - unsigned int off, chunk; - - if (unlikely(bytes > i->count)) - bytes = i->count; - if (unlikely(!bytes)) - return 0; - - if (!sanity(i)) - return 0; - - for (size_t n = bytes; n; n -= chunk) { - struct page *page = append_pipe(i, n, &off); - chunk = min_t(size_t, n, PAGE_SIZE - off); - if (!page) - return bytes - n; - memcpy_to_page(page, off, addr, chunk); - addr += chunk; - } - return bytes; -} - static __wsum csum_and_memcpy(void *to, const void *from, size_t len, __wsum sum, size_t off) { @@ -480,44 +294,10 @@ static __wsum csum_and_memcpy(void *to, const void *from, size_t len, return csum_block_add(sum, next, off); } -static size_t csum_and_copy_to_pipe_iter(const void *addr, size_t bytes, - struct iov_iter *i, __wsum *sump) -{ - __wsum sum = *sump; - size_t off = 0; - unsigned int chunk, r; - - if (unlikely(bytes > i->count)) - bytes = i->count; - if (unlikely(!bytes)) - return 0; - - if (!sanity(i)) - return 0; - - while (bytes) { - struct page *page = append_pipe(i, bytes, &r); - char *p; - - if (!page) - break; - chunk = min_t(size_t, bytes, PAGE_SIZE - r); - p = kmap_local_page(page); - sum = csum_and_memcpy(p + r, addr + off, chunk, sum, off); - kunmap_local(p); - off += chunk; - bytes -= chunk; - } - *sump = sum; - return off; -} - size_t _copy_to_iter(const void *addr, size_t bytes, struct iov_iter *i) { if (WARN_ON_ONCE(i->data_source)) return 0; - if (unlikely(iov_iter_is_pipe(i))) - return copy_pipe_to_iter(addr, bytes, i); if (user_backed_iter(i)) might_fault(); iterate_and_advance(i, bytes, base, len, off, @@ -539,42 +319,6 @@ static int copyout_mc(void __user *to, const void *from, size_t n) return n; } -static size_t copy_mc_pipe_to_iter(const void *addr, size_t bytes, - struct iov_iter *i) -{ - size_t xfer = 0; - unsigned int off, chunk; - - if (unlikely(bytes > i->count)) - bytes = i->count; - if (unlikely(!bytes)) - return 0; - - if (!sanity(i)) - return 0; - - while (bytes) { - struct page *page = append_pipe(i, bytes, &off); - unsigned long rem; - char *p; - - if (!page) - break; - chunk = min_t(size_t, bytes, PAGE_SIZE - off); - p = kmap_local_page(page); - rem = copy_mc_to_kernel(p + off, addr + xfer, chunk); - chunk -= rem; - kunmap_local(p); - xfer += chunk; - bytes -= chunk; - if (rem) { - iov_iter_revert(i, rem); - break; - } - } - return xfer; -} - /** * _copy_mc_to_iter - copy to iter with source memory error exception handling * @addr: source kernel address @@ -594,9 +338,8 @@ static size_t copy_mc_pipe_to_iter(const void *addr, size_t bytes, * alignment and poison alignment assumptions to avoid re-triggering * hardware exceptions. * - * * ITER_KVEC, ITER_PIPE, and ITER_BVEC can return short copies. - * Compare to copy_to_iter() where only ITER_IOVEC attempts might return - * a short copy. + * * ITER_KVEC and ITER_BVEC can return short copies. Compare to + * copy_to_iter() where only ITER_IOVEC attempts might return a short copy. * * Return: number of bytes copied (may be %0) */ @@ -604,8 +347,6 @@ size_t _copy_mc_to_iter(const void *addr, size_t bytes, struct iov_iter *i) { if (WARN_ON_ONCE(i->data_source)) return 0; - if (unlikely(iov_iter_is_pipe(i))) - return copy_mc_pipe_to_iter(addr, bytes, i); if (user_backed_iter(i)) might_fault(); __iterate_and_advance(i, bytes, base, len, off, @@ -711,8 +452,6 @@ size_t copy_page_to_iter(struct page *page, size_t offset, size_t bytes, return 0; if (WARN_ON_ONCE(i->data_source)) return 0; - if (unlikely(iov_iter_is_pipe(i))) - return copy_page_to_iter_pipe(page, offset, bytes, i); page += offset / PAGE_SIZE; // first subpage offset %= PAGE_SIZE; while (1) { @@ -761,36 +500,8 @@ size_t copy_page_from_iter(struct page *page, size_t offset, size_t bytes, } EXPORT_SYMBOL(copy_page_from_iter); -static size_t pipe_zero(size_t bytes, struct iov_iter *i) -{ - unsigned int chunk, off; - - if (unlikely(bytes > i->count)) - bytes = i->count; - if (unlikely(!bytes)) - return 0; - - if (!sanity(i)) - return 0; - - for (size_t n = bytes; n; n -= chunk) { - struct page *page = append_pipe(i, n, &off); - char *p; - - if (!page) - return bytes - n; - chunk = min_t(size_t, n, PAGE_SIZE - off); - p = kmap_local_page(page); - memset(p + off, 0, chunk); - kunmap_local(p); - } - return bytes; -} - size_t iov_iter_zero(size_t bytes, struct iov_iter *i) { - if (unlikely(iov_iter_is_pipe(i))) - return pipe_zero(bytes, i); iterate_and_advance(i, bytes, base, len, count, clear_user(base, len), memset(base, 0, len) @@ -821,32 +532,6 @@ size_t copy_page_from_iter_atomic(struct page *page, unsigned offset, size_t byt } EXPORT_SYMBOL(copy_page_from_iter_atomic); -static void pipe_advance(struct iov_iter *i, size_t size) -{ - struct pipe_inode_info *pipe = i->pipe; - int off = i->last_offset; - - if (!off && !size) { - pipe_discard_from(pipe, i->start_head); // discard everything - return; - } - i->count -= size; - while (1) { - struct pipe_buffer *buf = pipe_buf(pipe, i->head); - if (off) /* make it relative to the beginning of buffer */ - size += abs(off) - buf->offset; - if (size <= buf->len) { - buf->len = size; - i->last_offset = last_offset(buf); - break; - } - size -= buf->len; - i->head++; - off = 0; - } - pipe_discard_from(pipe, i->head + 1); // discard everything past this one -} - static void iov_iter_bvec_advance(struct iov_iter *i, size_t size) { const struct bio_vec *bvec, *end; @@ -898,8 +583,6 @@ void iov_iter_advance(struct iov_iter *i, size_t size) iov_iter_iovec_advance(i, size); } else if (iov_iter_is_bvec(i)) { iov_iter_bvec_advance(i, size); - } else if (iov_iter_is_pipe(i)) { - pipe_advance(i, size); } else if (iov_iter_is_discard(i)) { i->count -= size; } @@ -913,26 +596,6 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll) if (WARN_ON(unroll > MAX_RW_COUNT)) return; i->count += unroll; - if (unlikely(iov_iter_is_pipe(i))) { - struct pipe_inode_info *pipe = i->pipe; - unsigned int head = pipe->head; - - while (head > i->start_head) { - struct pipe_buffer *b = pipe_buf(pipe, --head); - if (unroll < b->len) { - b->len -= unroll; - i->last_offset = last_offset(b); - i->head = head; - return; - } - unroll -= b->len; - pipe_buf_release(pipe, b); - pipe->head--; - } - i->last_offset = 0; - i->head = head; - return; - } if (unlikely(iov_iter_is_discard(i))) return; if (unroll <= i->iov_offset) { @@ -1020,24 +683,6 @@ void iov_iter_bvec(struct iov_iter *i, unsigned int direction, } EXPORT_SYMBOL(iov_iter_bvec); -void iov_iter_pipe(struct iov_iter *i, unsigned int direction, - struct pipe_inode_info *pipe, - size_t count) -{ - BUG_ON(direction != READ); - WARN_ON(pipe_full(pipe->head, pipe->tail, pipe->ring_size)); - *i = (struct iov_iter){ - .iter_type = ITER_PIPE, - .data_source = false, - .pipe = pipe, - .head = pipe->head, - .start_head = pipe->head, - .last_offset = 0, - .count = count - }; -} -EXPORT_SYMBOL(iov_iter_pipe); - /** * iov_iter_xarray - Initialise an I/O iterator to use the pages in an xarray * @i: The iterator to initialise. @@ -1162,19 +807,6 @@ bool iov_iter_is_aligned(const struct iov_iter *i, unsigned addr_mask, if (iov_iter_is_bvec(i)) return iov_iter_aligned_bvec(i, addr_mask, len_mask); - if (iov_iter_is_pipe(i)) { - size_t size = i->count; - - if (size & len_mask) - return false; - if (size && i->last_offset > 0) { - if (i->last_offset & addr_mask) - return false; - } - - return true; - } - if (iov_iter_is_xarray(i)) { if (i->count & len_mask) return false; @@ -1244,14 +876,6 @@ unsigned long iov_iter_alignment(const struct iov_iter *i) if (iov_iter_is_bvec(i)) return iov_iter_alignment_bvec(i); - if (iov_iter_is_pipe(i)) { - size_t size = i->count; - - if (size && i->last_offset > 0) - return size | i->last_offset; - return size; - } - if (iov_iter_is_xarray(i)) return (i->xarray_start + i->iov_offset) | i->count; @@ -1303,36 +927,6 @@ static int want_pages_array(struct page ***res, size_t size, return count; } -static ssize_t pipe_get_pages(struct iov_iter *i, - struct page ***pages, size_t maxsize, unsigned maxpages, - size_t *start) -{ - unsigned int npages, count, off, chunk; - struct page **p; - size_t left; - - if (!sanity(i)) - return -EFAULT; - - *start = off = pipe_npages(i, &npages); - if (!npages) - return -EFAULT; - count = want_pages_array(pages, maxsize, off, min(npages, maxpages)); - if (!count) - return -ENOMEM; - p = *pages; - for (npages = 0, left = maxsize ; npages < count; npages++, left -= chunk) { - struct page *page = append_pipe(i, left, &off); - if (!page) - break; - chunk = min_t(size_t, left, PAGE_SIZE - off); - get_page(*p++ = page); - } - if (!npages) - return -EFAULT; - return maxsize - left; -} - static ssize_t iter_xarray_populate_pages(struct page **pages, struct xarray *xa, pgoff_t index, unsigned int nr_pages) { @@ -1482,8 +1076,6 @@ static ssize_t __iov_iter_get_pages_alloc(struct iov_iter *i, } return maxsize; } - if (iov_iter_is_pipe(i)) - return pipe_get_pages(i, pages, maxsize, maxpages, start); if (iov_iter_is_xarray(i)) return iter_xarray_get_pages(i, pages, maxsize, maxpages, start); return -EFAULT; @@ -1573,9 +1165,7 @@ size_t csum_and_copy_to_iter(const void *addr, size_t bytes, void *_csstate, } sum = csum_shift(csstate->csum, csstate->off); - if (unlikely(iov_iter_is_pipe(i))) - bytes = csum_and_copy_to_pipe_iter(addr, bytes, i, &sum); - else iterate_and_advance(i, bytes, base, len, off, ({ + iterate_and_advance(i, bytes, base, len, off, ({ next = csum_and_copy_to_user(addr + off, base, len); sum = csum_block_add(sum, next, off); next ? 0 : len; @@ -1660,15 +1250,6 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages) return iov_npages(i, maxpages); if (iov_iter_is_bvec(i)) return bvec_npages(i, maxpages); - if (iov_iter_is_pipe(i)) { - int npages; - - if (!sanity(i)) - return 0; - - pipe_npages(i, &npages); - return min(npages, maxpages); - } if (iov_iter_is_xarray(i)) { unsigned offset = (i->xarray_start + i->iov_offset) % PAGE_SIZE; int npages = DIV_ROUND_UP(offset + i->count, PAGE_SIZE); @@ -1681,10 +1262,6 @@ EXPORT_SYMBOL(iov_iter_npages); const void *dup_iter(struct iov_iter *new, struct iov_iter *old, gfp_t flags) { *new = *old; - if (unlikely(iov_iter_is_pipe(new))) { - WARN_ON(1); - return NULL; - } if (iov_iter_is_bvec(new)) return new->bvec = kmemdup(new->bvec, new->nr_segs * sizeof(struct bio_vec), diff --git a/mm/filemap.c b/mm/filemap.c index 3a93515ae2ed..470be06b6096 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -2690,8 +2690,7 @@ ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter, if (unlikely(iocb->ki_pos >= i_size_read(inode))) break; - error = filemap_get_pages(iocb, iter->count, &fbatch, - iov_iter_is_pipe(iter)); + error = filemap_get_pages(iocb, iter->count, &fbatch, false); if (error < 0) break; From patchwork Wed Mar 8 14:37:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13165957 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 977F2C64EC4 for ; Wed, 8 Mar 2023 14:38:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 34BA36B0081; Wed, 8 Mar 2023 09:38:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2FADC6B0082; Wed, 8 Mar 2023 09:38:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 19C316B0083; Wed, 8 Mar 2023 09:38:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 0A7CF6B0081 for ; Wed, 8 Mar 2023 09:38:32 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id C1525807A3 for ; Wed, 8 Mar 2023 14:38:31 +0000 (UTC) X-FDA: 80545986822.05.0DF93D7 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf28.hostedemail.com (Postfix) with ESMTP id 2B8D7C001C for ; Wed, 8 Mar 2023 14:38:29 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=LeDegEF9; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf28.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678286310; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=T0MgyEE2up5lyn6F4xZiFP7ZLu50ZEhkrR3VW4b8xrI=; b=e+wfF8mi5FdxA3U462rZdc/am3E+UrCqYkKKAxeEcngvSLaMQOaUS4CEmV2MNRff3YdvDs 2nOkjGyci5+BMrW73Ha/wKLToMQk+aaUf8XZrNkB76lwrR/A1JHxEIDW8O70A7o/X5IZCG EI4neaJmH7xp+otE6YjPicmHL9LmIF4= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=LeDegEF9; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf28.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678286310; a=rsa-sha256; cv=none; b=kc1tU8V2dKdyR4UqLHGAfnREGwKIfUCVA97XxCnV6xrpepNWHwHfyKmqXp0my/63E/hpS/ kb3h0V34xdJxgrv7ac2zL47n5s84jPhdlYwRu0XYh9Japc+9ywjAqHE/dJnh7LlTWIQDdF oviLXXc0mQSDnKZUZa1DskGHu1hfEP4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678286309; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T0MgyEE2up5lyn6F4xZiFP7ZLu50ZEhkrR3VW4b8xrI=; b=LeDegEF9xunR33G/QWvOa7Pz/eG2tKtiOTJJBMDn9Z4d4CmjCAJN+kssE8Ciyhot7L0nBL QxE0NKqSzFIvYYDXmeEE5muBxwZKrlwQ6XuXkrH6BGxUYeyLLt5dvhOdkCCFCYcbdH3MkT ekliw7V1IbHCmNa9Ucfm8M5Z9bRWAQc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-349-LwKNUDAjOYOokcFxinhM6A-1; Wed, 08 Mar 2023 09:38:26 -0500 X-MC-Unique: LwKNUDAjOYOokcFxinhM6A-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5AD9F101A55E; Wed, 8 Mar 2023 14:38:22 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 71FC01121314; Wed, 8 Mar 2023 14:38:20 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, John Hubbard Subject: [PATCH v16 08/13] iomap: Don't get an reference on ZERO_PAGE for direct I/O block zeroing Date: Wed, 8 Mar 2023 14:37:49 +0000 Message-Id: <20230308143754.1976726-9-dhowells@redhat.com> In-Reply-To: <20230308143754.1976726-1-dhowells@redhat.com> References: <20230308143754.1976726-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Rspamd-Queue-Id: 2B8D7C001C X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: bwcg97bz3w4spjcp5tdpaw9n5qebrbto X-HE-Tag: 1678286309-589625 X-HE-Meta: U2FsdGVkX1/2hgkLa+gpOIooa7p4k4elibxIHb2K04P+KXxmP8Zu1VCxCcCEUItWgC+wPbeD1o0B9xZSgpkneS0VgpXz1CiA5MNYKEi+zlde1LHf9sxzOt/7fqRLJWaHERCKU3YSmaDX1AsAktEIfs4nVWNu/H6A/7uW0ZzaG25GERA5HpIy0R6ORKXNgaHT9S0yU3KdpGN1p6Jh95nDTrjRbRG35vsLvIoX8XRlpurT8iGFPK5MHYktMtCY48vqgGDUcauzNhd0gn2olQmTEoERrG0qD4KaRecPLRHhj4Js3lIb3adLSLCbRW6fxfrvwRpXG2sRz3G/2WMh8s5puJ9axIQpClrFDLmi7ecSB04frv01Txt2AVZOIkOICZoLsLwLB+7t0g37OE3B1hLbPW1W32NdtMSlle+K3pH75ZgL8u2+JokbAN9UR4N17D9nwaBmAVSL12VE2ybQkecyRd+qPAZfv4C+wn7ZkVAfBgBuuEN4HPm6Qse8gqUKhSQsvGYs00rdYtbIdxTsuWHECLLI/aswUtKJhtWHc88/xRf1sIs+HFkxk2UZP2m+r7vFAp9A7CAQwLaOJVuutTexmwmAuRHRMqAUwqHN07xvOYienp6Bx4ccynK8DBL1RwEWjHLCjJCXhOIMGRKIbpktC4+5XyxIGlNDlY9DEUTIO0l69GBU2M7a4GAhH31B3SV/eSVk57eiMTerHFAG3sJFXMJLfFiDI605GoJQXjoaaCeBdU3Fp4YWv6NN6RW7jywNK/4xPph6iOjAxa+CqY2raJtGS6M30O0ijgK0eTCk7GSsRx1jM7yn25U6Lz7CbfZm/EOn3R9Xux4TD1v827Jhg801MmdCg0P12YxvLI/KPIANXyN5P+4MaXBXr+yGdxNKgnssGCMBgfGVgUdDJHU632epRpkRuGDCKp4bcSG/uOmID5kEZhQW6IufpkzFXFNPETECqtC3UBu0KAxSLlU hlhRFhly vmlfVskYZ6tmbMEwkhh01m7g/n4Z+GbUTYgiliFpD3PKEwP6vox0RUhZgL1tAl9pq8Azjh/1PqYjKMtL8CKjuNQn9HXKgS/vpojT7J4KjtPfueav4wugbwPfuXDPM6hqcb3ZZu2swcN71nfSimK0oBx4s6cUNpmQX6PDI7oC675hArbkIFOaO7DztN9oI4LgnYRbK8+umHPj0CHLV8K7AQFbbeqqArFHG0kpRx0fnnZdqF2OCRIP8nU2LedsPyOUIOQt8vMtdZGlBhjQXuytpvUddb6Bm2oeFNZXs0nr+8IMmnUMVJOo/c6YupArG0aFsT1Nd3VqCd8cd65zGmxpcAUjczMx7xQvJkTEyf+lo1eWYcTfOSOeZ3kEIf+JG73CwD7v+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ZERO_PAGE can't go away, no need to hold an extra reference. Signed-off-by: David Howells Reviewed-by: David Hildenbrand Reviewed-by: John Hubbard cc: Al Viro cc: David Hildenbrand cc: linux-fsdevel@vger.kernel.org --- fs/iomap/direct-io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index f771001574d0..850fb9870c2f 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -202,7 +202,7 @@ static void iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio, bio->bi_private = dio; bio->bi_end_io = iomap_dio_bio_end_io; - get_page(page); + bio_set_flag(bio, BIO_NO_PAGE_REF); __bio_add_page(bio, page, len, 0); iomap_dio_submit_bio(iter, dio, bio, pos); } From patchwork Wed Mar 8 14:37:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13165958 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68ECFC678D5 for ; Wed, 8 Mar 2023 14:38:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 03D0F6B0082; Wed, 8 Mar 2023 09:38:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F300D6B0083; Wed, 8 Mar 2023 09:38:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD0836B0085; Wed, 8 Mar 2023 09:38:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CF3916B0082 for ; Wed, 8 Mar 2023 09:38:38 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id A2614C0928 for ; Wed, 8 Mar 2023 14:38:38 +0000 (UTC) X-FDA: 80545987116.23.C5CCB43 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf13.hostedemail.com (Postfix) with ESMTP id D943420012 for ; Wed, 8 Mar 2023 14:38:36 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="VU/7DXSR"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf13.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678286316; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0+QymJ+GCZW28tJ8JK9HTXv9GxubpscA9882LeMgv9w=; b=CFPrPUxQI/mBpidLY6T3r0/BBw6WY1EN/vkIWL5xYPrYUIUI3DvmzAVJU+OHowuWQRn7BA Yc+aoJbD3iLo31iKjEuWtpWOCHhihcJa/y4mDJfwTBJgex5tRXrCbcntSzJcLBb71RsE8h vRvAhaSXtloNNbWNAUzS9JvM9D8W8r8= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="VU/7DXSR"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf13.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678286316; a=rsa-sha256; cv=none; b=7me3RCc/vp5c/jjuPkJ9OCAtX58Yhk/HMydY7x8HTELa5QHlpDuIbdR2q+zD7ozOUFEj/9 hb4sqTOYNTgXHjVhX0XAYfOW+pFEYuD5FFlpMWeUIE1zK0CjO9TQqBGogU8AMlUsOPUj+A wd869ubwfjCN2DviNcvITICDXVL/Dm4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678286316; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0+QymJ+GCZW28tJ8JK9HTXv9GxubpscA9882LeMgv9w=; b=VU/7DXSR3FdInzB59T4HXp+vfDM9KX55TPo2xoEW9TVoONigm98t+v0QELbEqPp+crhq1G fq31dLuABtBFo9cjQxBp1Gr83To2e5ykW1A0zQR1ZqTCEv8Hp7njH4ee9O9D8CUlxwsoYf fFz7OdwSu0McvZnfukO9sgn+yTdx7Xg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-522-MQY17EnwNZGepEkwDtQ80A-1; Wed, 08 Mar 2023 09:38:34 -0500 X-MC-Unique: MQY17EnwNZGepEkwDtQ80A-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E96C0803D62; Wed, 8 Mar 2023 14:38:24 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id EA7C22166B2A; Wed, 8 Mar 2023 14:38:22 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v16 09/13] block: Fix bio_flagged() so that gcc can better optimise it Date: Wed, 8 Mar 2023 14:37:50 +0000 Message-Id: <20230308143754.1976726-10-dhowells@redhat.com> In-Reply-To: <20230308143754.1976726-1-dhowells@redhat.com> References: <20230308143754.1976726-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: D943420012 X-Stat-Signature: 7fjr1f1sdma9jsdm6bx6qo4beiu4bp1h X-HE-Tag: 1678286316-239347 X-HE-Meta: U2FsdGVkX18+x9Jh73xSwATg4ccHlT27OblCg2rx8CumVzRE/gPSph187tpYvf7HxRQuVB5BbllaXWyIK0lbizNigNIevFOkukR+V+furiAviLLLaRhnLv2qsuV5fHK8vIXqNXyj5V8NEqi/WdZYa7s7NE247C64gCb+e1F1z1OBpSfjZdbjzXrqISNQ3AaXWloVvrRl8rSfViYyGc2qsGWZ8DsXTBYRHl2EIBxeZiEa5zZnVqebwwYCSgqfcXfUseLJj/dJIh1cmghuVzNWHp85CXDolsmNouTPdeSg2+q6rvSwWqBmFzSfhi4SQTgmt9mSJj9Cnf8upHULgGyc17YOBNexpM+8bo26dLT0PLoPQyh73dznYMpNGj+Nk3wce4PDqolA+sbrrCviTJ+bNXWRGG1x4Iy/xE/lo6jkMNt1no1Vt+OK1/OjjYOambvWjc0Vj0v8aOYDAQJ4rs5xTsYioMUK9s3P0Rk9SjWUC8HsSNyBcICP4oXRgovzGuCbbDr/1njntDo3qkVFNkmd9mUAJRzZtM/756h/IaV4W35gOaBLMdOB/7pdCuj1xu5/QLizxdlJIHK+MyM+fQ2cb7uC7mpji06iDH3JM+MDyOVA0wEyteI6uUetIGAKcd6+nKzvPswRTyhkH3kse6OgrGnl/j7zpPX9Ma8wt1L1e8E6aIJYonZx8zWCo4uLJFjZe/Npep1UPhkxEL+YWi/cFdDWZ5FBeI0y+nPZ+K97bR1CuFRpm371i6+ktopUevIwQLkHGJr+IZlucBQnyu4hGQnwUIgr94YlZ/YmsfIsed2jQTHOQ+iDaRpgy9WoNg6cq50r+Ojav/KOg5QFy9B5zvc4pyappQS9CYsZVEruphWLVZ6EUYUpejhnXh+ANm8wH1rV7qDaHfinbFeLdRlN5IWJgVhGGoQJJbj21+ChXk4IMXN6lhbFaTBePoa6tu+XnZKKcqN4NkFg8MJi6uZ z2y7PTh0 yx/RPfs1Gn90xICLojgrexWRRbI7J36A/wS2/uqNZ7r8/WVVn3HhzozZo5OtD8jxdqQsDkX5RIfuNSGOb+zfePF2hCwYn3tY1UWUOxHy3krgaUsa5lGrckoREyqukQfOlQ4F171RRDd1tyLii7mjWm2fr2TkrYqPVL14LD/5JmDcbBdgsKTql/yDFVo6FhSMoUqBPStRX5jDauECiDgWyqBZ52GNyq99L+OVkFTCZgkdyb/9DEVbJyOpL2KkLk6Cg5Q3HtFMYEtDX8/tfjlaRa7X6qdyvsU94rnjHehOAwxfC5Bw10ypMzZbBfQUBAzsOp7f/XCwhREra7qY78om/NfOsYlQ9HOyIlkwjURzYic7Jw133RmyIC7r5V5L4uWS/Rb39q2YQx3/Fl0nAHHragvtxV54XOiwrx3ydnwB4k6pqufLsJO2HXqBd2pb6a4tfMNBep3m7dmnGSW6o+wQvJ7mkz7uPhhwgyTv7CBLOFzejDJo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Fix bio_flagged() so that multiple instances of it, such as: if (bio_flagged(bio, BIO_PAGE_REFFED) || bio_flagged(bio, BIO_PAGE_PINNED)) can be combined by the gcc optimiser into a single test in assembly (arguably, this is a compiler optimisation issue[1]). The missed optimisation stems from bio_flagged() comparing the result of the bitwise-AND to zero. This results in an out-of-line bio_release_page() being compiled to something like: <+0>: mov 0x14(%rdi),%eax <+3>: test $0x1,%al <+5>: jne 0xffffffff816dac53 <+7>: test $0x2,%al <+9>: je 0xffffffff816dac5c <+11>: movzbl %sil,%esi <+15>: jmp 0xffffffff816daba1 <__bio_release_pages> <+20>: jmp 0xffffffff81d0b800 <__x86_return_thunk> However, the test is superfluous as the return type is bool. Removing it results in: <+0>: testb $0x3,0x14(%rdi) <+4>: je 0xffffffff816e4af4 <+6>: movzbl %sil,%esi <+10>: jmp 0xffffffff816dab7c <__bio_release_pages> <+15>: jmp 0xffffffff81d0b7c0 <__x86_return_thunk> instead. Also, the MOVZBL instruction looks unnecessary[2] - I think it's just 're-booling' the mark_dirty parameter. Signed-off-by: David Howells Reviewed-by: Christoph Hellwig Reviewed-by: John Hubbard cc: Jens Axboe cc: linux-block@vger.kernel.org Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108370 [1] Link: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108371 [2] Link: https://lore.kernel.org/r/167391056756.2311931.356007731815807265.stgit@warthog.procyon.org.uk/ # v6 --- include/linux/bio.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/bio.h b/include/linux/bio.h index d766be7152e1..d9d6df62ea57 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -229,7 +229,7 @@ static inline void bio_cnt_set(struct bio *bio, unsigned int count) static inline bool bio_flagged(struct bio *bio, unsigned int bit) { - return (bio->bi_flags & (1U << bit)) != 0; + return bio->bi_flags & (1U << bit); } static inline void bio_set_flag(struct bio *bio, unsigned int bit) From patchwork Wed Mar 8 14:37:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13165959 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F553C6FD1E for ; Wed, 8 Mar 2023 14:38:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 016136B0074; Wed, 8 Mar 2023 09:38:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F08736B0078; Wed, 8 Mar 2023 09:38:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D5BFE280001; Wed, 8 Mar 2023 09:38:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C6DE06B0074 for ; Wed, 8 Mar 2023 09:38:40 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5E6931A0F57 for ; Wed, 8 Mar 2023 14:38:40 +0000 (UTC) X-FDA: 80545987200.06.45D4BF5 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf03.hostedemail.com (Postfix) with ESMTP id A014E2001D for ; Wed, 8 Mar 2023 14:38:38 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=dcOQosEl; spf=pass (imf03.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678286318; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cRZ6zwkATo5WH/jIVyGNX8/GaSvHJQOFy9f4tGJFuL8=; b=MPWpE3utg09TxpNWXcKgN78yEzYsaXvaNdrQRwseCS3ac/m3tE0ZlrLW6q8b9Vh3V9rmSO nySmhtGvMtP19SijPocUkG+gjUspUuQeuQqnisGUtNYNSfa2KsvWb0rrD8Dg8ry6BlDGKP M8HOPDMZ+M2zTJGsiYlPbHxnjYQbpP8= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=dcOQosEl; spf=pass (imf03.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678286318; a=rsa-sha256; cv=none; b=yqSf+vjHkfHqAgaPlPGngvKi65XVezILEuk/Id0RMyv16jGXqyNJdxQfoPb1cj4Vz5+DZH 32RoeampY0Vfn0O8Fb02EOnAeBXtJdIwaN0KEL3WTzJqRQtKVfZg2jzpBOAP+tDdZEK1Uf VispuoLSp0jHHLm7ZSwMsf8oWbv0xcs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678286318; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cRZ6zwkATo5WH/jIVyGNX8/GaSvHJQOFy9f4tGJFuL8=; b=dcOQosElHrvaV6bYRKUfDxvRqeiOE0/RwWdS7ieU3TfwOEKh5FYcYk5DZxpgfTtZhQrMXO iv7ixpF5ajJULgxZLLo0/RrKXD7GM2jHhEna2XXglTmBb8Xl2Nsheq/YRo1JHs15FhLNSq DUH2WU4SAFJbKDjJlxL4jEuzURvdtW8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-377-hIiZUSLuPjqJrRjMvg_wpg-1; Wed, 08 Mar 2023 09:38:35 -0500 X-MC-Unique: hIiZUSLuPjqJrRjMvg_wpg-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AC064823DF1; Wed, 8 Mar 2023 14:38:27 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id A710D492B00; Wed, 8 Mar 2023 14:38:25 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v16 10/13] block: Replace BIO_NO_PAGE_REF with BIO_PAGE_REFFED with inverted logic Date: Wed, 8 Mar 2023 14:37:51 +0000 Message-Id: <20230308143754.1976726-11-dhowells@redhat.com> In-Reply-To: <20230308143754.1976726-1-dhowells@redhat.com> References: <20230308143754.1976726-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A014E2001D X-Stat-Signature: ysdxoozzjpxr4cpmixj14i1p9k1oqfj7 X-HE-Tag: 1678286318-660471 X-HE-Meta: U2FsdGVkX1/XlfFpymntjXhq2eIWTUzb4wQwH3unrGtuO1N7GJZs6Qu9vxGdy3edMqm/fD8GIIscNf7H6X4BKj/XKl78Hr/xuP9WoeusEwUImLCbsqNSou6srOeMaTlZysJ3yhwvQ6ZgwXkcomO9UBZn7PPP7bN5iSDllgEyl3nGKiuO7X//Qb+5kUm5yC42nu1xRnHCmphTHM/2cp47+r7sq4rPfYYk6uwIQPagOm66Gt/CPLFOCeZLvZYvJ1+5QgdWOuBbhclFxECFDQ/OB9bBEzpS07k1NKCiv7fITl1GtoU7NMqw1oekW5Vm43btkc1vuBJK3XC5+qmwvbT9X55Nd0mQL7tOoP1F7jSZRtMohAeIJAQ+Iu6BfVRrLyiU8bBRAenTf09ikp7S+zIgImW9ktBsat7TUuwK6iN5HXZgMYBoXyaqnoGhpZi/L8x8ODSVzCxONQY2tDeVewGU1nm/SCc4DoorD5wr0t9eL3JVzv7YHQ6PYeHsiFuWeGx12AaByNMQUy674aqCXBJ/ns7P1ZjFkN9P0Ij5lhUQZlXUGIu6OhWr+SuSH7nUCDxbjrWRBJbWZyqdJuyPRU5a4wghy40utD+/Gpc+ZmmBzT/TZwESIYLgfMmT2B8CQwdnTy0hJHyrjwX15dcrpqAmaCcn7wMJRD523B0f7iG0iVAMPD0NPhpn2tHFAwYGTlWB2noLGecZTfp3Qo2TjTFijYOvm9xjnLf+BsOd3L/WfkBGnvJ/WNydxB67joAXDVTU7kztS8hjwkuaGgJ8S4qmLrd5KIPhDYZPZAC2YZBtZR2wHlp/TsvpX1ZenD1SyArQEXTVohWVy8dJewIXdBncaepY9SM84Xqz0MK4lW65j4VTP/LcleJLREdUhgyCZ1GG79KvmC5DVVREgZRcl3rA49QvofrcS5t3uMqXWUmUiUM2M1KTEatHzCA//B6GOJLnl70Tl3LNFylJ5Qj4VyV lgwmiYLM oWWiGPc4vPLUZMOHCOqxBgOJXZG325vJycNfCgka0vJf5NOFbX9WkkNcjhCUZfeGaIfAboCjZerYmN2VfAB+KJmgTCq35TptECqBw3ESKyGhEFW8Ru5SkQ23NPdCKI9SmDDaM8Irvx6gCxodgTN2Z++QSTa/JEMAHxr95NfUfj6gGfSPTBvIOyF3dpZ3/D4/60+Eyqq3Z3VsUymB1K+geyxc3sLoc4TG/B3pqN6JzYTOp2NT+qJJt9iSYq5XxvzU+bfMVa2CGk7h6Oe/umc0GEGvdb/eKBlx8s+s35gQKWVv0/QxNADyK0mDuGjDz7ecyTgfS9YQEp4GysfrejAYKxO43w1EcqPg7ka1FpkFkj3syzCDszR5oagyCC2O52RNz1jKv0gRzPRss+YOe/axowvOkG/qVTPJ82MT36mPkokpeVOhmdEfNnkhOyaSHw2FJt4aZmwexQFyUiaj4NdToHm7FlCaLDFTIB3exmBkozcZ3OEobT6RnCsu6qQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Replace BIO_NO_PAGE_REF with a BIO_PAGE_REFFED flag that has the inverted meaning is only set when a page reference has been acquired that needs to be released by bio_release_pages(). Signed-off-by: Christoph Hellwig Signed-off-by: David Howells Reviewed-by: John Hubbard cc: Al Viro cc: Jens Axboe cc: Jan Kara cc: Matthew Wilcox cc: Logan Gunthorpe cc: linux-block@vger.kernel.org --- Notes: ver #8) - Split out from another patch [hch]. - Don't default to BIO_PAGE_REFFED [hch]. ver #5) - Split from patch that uses iov_iter_extract_pages(). block/bio.c | 2 +- block/blk-map.c | 1 + fs/direct-io.c | 2 ++ fs/iomap/direct-io.c | 1 - include/linux/bio.h | 2 +- include/linux/blk_types.h | 2 +- 6 files changed, 6 insertions(+), 4 deletions(-) diff --git a/block/bio.c b/block/bio.c index fd11614bba4d..4ff96a0e4091 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1190,7 +1190,6 @@ void bio_iov_bvec_set(struct bio *bio, struct iov_iter *iter) bio->bi_io_vec = (struct bio_vec *)iter->bvec; bio->bi_iter.bi_bvec_done = iter->iov_offset; bio->bi_iter.bi_size = size; - bio_set_flag(bio, BIO_NO_PAGE_REF); bio_set_flag(bio, BIO_CLONED); } @@ -1335,6 +1334,7 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) return 0; } + bio_set_flag(bio, BIO_PAGE_REFFED); do { ret = __bio_iov_iter_get_pages(bio, iter); } while (!ret && iov_iter_count(iter) && !bio_full(bio, 0)); diff --git a/block/blk-map.c b/block/blk-map.c index 9137d16cecdc..c77fdb1fbda7 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -281,6 +281,7 @@ static int bio_map_user_iov(struct request *rq, struct iov_iter *iter, if (blk_queue_pci_p2pdma(rq->q)) extraction_flags |= ITER_ALLOW_P2PDMA; + bio_set_flag(bio, BIO_PAGE_REFFED); while (iov_iter_count(iter)) { struct page **pages, *stack_pages[UIO_FASTIOV]; ssize_t bytes; diff --git a/fs/direct-io.c b/fs/direct-io.c index ab0d7ea89813..47b90c68b369 100644 --- a/fs/direct-io.c +++ b/fs/direct-io.c @@ -403,6 +403,8 @@ dio_bio_alloc(struct dio *dio, struct dio_submit *sdio, bio->bi_end_io = dio_bio_end_aio; else bio->bi_end_io = dio_bio_end_io; + /* for now require references for all pages */ + bio_set_flag(bio, BIO_PAGE_REFFED); sdio->bio = bio; sdio->logical_offset_in_bio = sdio->cur_page_fs_offset; } diff --git a/fs/iomap/direct-io.c b/fs/iomap/direct-io.c index 850fb9870c2f..ceeb0a183cea 100644 --- a/fs/iomap/direct-io.c +++ b/fs/iomap/direct-io.c @@ -202,7 +202,6 @@ static void iomap_dio_zero(const struct iomap_iter *iter, struct iomap_dio *dio, bio->bi_private = dio; bio->bi_end_io = iomap_dio_bio_end_io; - bio_set_flag(bio, BIO_NO_PAGE_REF); __bio_add_page(bio, page, len, 0); iomap_dio_submit_bio(iter, dio, bio, pos); } diff --git a/include/linux/bio.h b/include/linux/bio.h index d9d6df62ea57..b537d03377f0 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -488,7 +488,7 @@ void zero_fill_bio(struct bio *bio); static inline void bio_release_pages(struct bio *bio, bool mark_dirty) { - if (!bio_flagged(bio, BIO_NO_PAGE_REF)) + if (bio_flagged(bio, BIO_PAGE_REFFED)) __bio_release_pages(bio, mark_dirty); } diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 99be590f952f..7daa261f4f98 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -318,7 +318,7 @@ struct bio { * bio flags */ enum { - BIO_NO_PAGE_REF, /* don't put release vec pages */ + BIO_PAGE_REFFED, /* put pages in bio_release_pages() */ BIO_CLONED, /* doesn't own data */ BIO_BOUNCED, /* bio is a bounce bio */ BIO_QUIET, /* Make BIO Quiet */ From patchwork Wed Mar 8 14:37:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13165960 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B6D0C678D5 for ; Wed, 8 Mar 2023 14:38:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 56E036B0078; Wed, 8 Mar 2023 09:38:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 51E50280001; Wed, 8 Mar 2023 09:38:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 397746B0085; Wed, 8 Mar 2023 09:38:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2B40E6B0078 for ; Wed, 8 Mar 2023 09:38:41 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id F0F771209EA for ; Wed, 8 Mar 2023 14:38:40 +0000 (UTC) X-FDA: 80545987200.01.844AFA0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf14.hostedemail.com (Postfix) with ESMTP id ECF80100011 for ; Wed, 8 Mar 2023 14:38:38 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hhf1F4Fw; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678286319; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HEPrb4kIqMqW7eebyDDmobfSBttcdx5+yTOr0krGf5g=; b=AY4iQM6ohN8ZiPPChVafz2jyy2BJJZGyKjG/+Yi6AJigGTSozZpSjeFymyrc0E5siijH6k fXcnKNUS/T1QSUs1k0CxNXOQhbmkDK9/S7HLnLe77uGwDMbvYW76bgS/pyhhgKVKfegiSV Jy4eNenTbS0EOy1vYR9ueO6bVWkcG+k= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hhf1F4Fw; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678286319; a=rsa-sha256; cv=none; b=oUxKNmpxY6jomlMF02EQLA7+6Uu/6dH864dEJUmcAyieDWrTKQPKfB756zTTaG+K8V3XdK GLDd/5XLXc3PRV5emlypeRpJ0jBKR+V1Z89+/avWW1Ka/7qP/iCZbzFR1fRlcEvczhaEcN dV0eN2yDyZVpu09GR5iKnIrxZrSziso= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678286318; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HEPrb4kIqMqW7eebyDDmobfSBttcdx5+yTOr0krGf5g=; b=hhf1F4FwBK83BofBkL+/xgu3nJbWZZeV8scTuJtBGDveXPRcI5ZPdUQwWJcNGYjElYTYny lCCNj0qvSbnTPWLIMzhhVx1Tm/oFtazu92lA/FJ53Qf30DQUimlJSyyaJOyJT1cphxMQ0R pbPHEcYUSbrorMQoYfbLG43ad0W+CBE= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-647-K3nJiuC7OpWvo24vQi0Wfg-1; Wed, 08 Mar 2023 09:38:35 -0500 X-MC-Unique: K3nJiuC7OpWvo24vQi0Wfg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 49555382C991; Wed, 8 Mar 2023 14:38:30 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 482B31121330; Wed, 8 Mar 2023 14:38:28 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v16 11/13] block: Add BIO_PAGE_PINNED and associated infrastructure Date: Wed, 8 Mar 2023 14:37:52 +0000 Message-Id: <20230308143754.1976726-12-dhowells@redhat.com> In-Reply-To: <20230308143754.1976726-1-dhowells@redhat.com> References: <20230308143754.1976726-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: ECF80100011 X-Rspam-User: X-Stat-Signature: b33ab65r51erukomc696z5iux1oetxhh X-HE-Tag: 1678286318-598003 X-HE-Meta: U2FsdGVkX19wtqXfwFJPvx1QbVlOIr3pmyp/cLhxHc+c3kbAeGiP6epbLZ5SXLqNpmYGBirh+nL9LYIZfxj9HQBsJdXdUTZAYTYkqBUyrNul2o3PJa2dxF+A1EjwhcHTOrcHIOq+giAJrr1vCrAwSNUlmkUkgg9bceF3IwMzeCAWjOhiBNKwURIBWVh5B2Subml3XhdvQiWEi360seWu1m2/fkzAauRzpsDToIjWCfHDzMU0XQNRxSPqVlrsjaKc25+sZMdQYq4a8885rS23xusZC0PN5rBz4fTtl/K9n39Y5//kuR8fGcIswR576t6Q2jwFdUDibZmorxMHC6S/D/NX87dOC8/YR0W9uqlFXLBYn63F09dzgpZwODtgzDvqS+Dmonc9uRjf0PKu5GMFyNPvF/xSg4U3UT4Lbx1cvi9k8LXTpcQ2IiWNbww3apSnAvJZg6Sd7lbeSJjOhgsarXCAYo5Jz5vXPCf0yQRHfYwyaeLj3NTxPHwjHdTxt6dZbxlW0nwMnB0B5/s3pKdel4W3JWdvNW+nYbYPls0YnuNWgnDJOBasp8nGyvscNwxHtG4d6n4In3+Qabnak3Kfrj43EUTOs/vGuZjyxHqG+mKPRtwzSwCRcgJu80bh0A91243xFEejaVKEPXMBqVP5ieZx2w5fFBE2PeX1Zov2Ta+DacLmdjwepy5XbJ4xMs//PYJ0uVTuLjOcLLUGfq+uliI+l0csZhGuuOH8JHAMP/wex2CFyVvc6m45A691ODTudKHvXWHpQGoWVpj+rBbsGajTUePdkE26QbH1oWX8yzQv788fVhzUlq+4MapLCICTLhAVkJNSc4U9Zv3JscIGzmJ3SVM1Th6NF9CkmAD+Ca1KYslFx6eEddLKdQkVmWpBt4v9XmS6uGVxGqCs2pBJ9Iu7mi8lDhSovKetd78++a4boKINOHw9QeXGE4TvQyd0jEOka1K4Ba/LCidLFpY dkHSF+OK fQpiwgmtGC/RF5MZ8zz1+ZnzM//dgI4xVgG7lnCAtr0ofYfDou79vmnbAeOGLYCD54QTsrph778UHyXr+cvtek0A1UpxiJ5MrDm0ijAlNpY4sWLfv1GjO4Vj2D/ZvuqtpWZwYxhL2vpzzkuzj+LRH53ti9kBj4u5ZCG/KRLGG2BOrL2oUvrxieDOr05Dd8KjGWDjG28vJ3jkaIZm2jul609ngcdzm8uFoip/N7WjtWeSSs8ylSTUrh7upaXi5WFtfuVpaKk9dLVZ0vRvRZoD6TDNABfPchk0qa8brMLju22c4CgdQWezGPiL+/7yAxGc8j6U85SUDC0WQhD9jZpFAtASoTLCBe8hhztrRGdUSjolSoFpkuMuan+qqmNvuS8iqwvMUK3TtiklATJfvKKCxBptd6BYaP4y/WFs1PJkE70ZMrAi9D502Wf+cB8LXAz0ulMtu3b+V1UNl7DEbO/cvTMlH+b4QdkOVcH0mfST5bUr/quEEEGzwVmNqwg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add BIO_PAGE_PINNED to indicate that the pages in a bio are pinned (FOLL_PIN) and that the pin will need removing. Signed-off-by: David Howells Reviewed-by: Christoph Hellwig Reviewed-by: John Hubbard cc: Al Viro cc: Jens Axboe cc: Jan Kara cc: Matthew Wilcox cc: Logan Gunthorpe cc: linux-block@vger.kernel.org --- Notes: ver #10) - Drop bio_set_cleanup_mode(), open coding it instead. ver #9) - Only consider pinning in bio_set_cleanup_mode(). Ref'ing pages in struct bio is going away. - page_put_unpin() is removed; call unpin_user_page() and put_page() directly. - Use bio_release_page() in __bio_release_pages(). - BIO_PAGE_PINNED and BIO_PAGE_REFFED can't both be set, so use if-else when testing both of them. ver #8) - Move the infrastructure to clean up pinned pages to this patch [hch]. - Put BIO_PAGE_PINNED before BIO_PAGE_REFFED as the latter should probably be removed at some point. FOLL_PIN can then be renumbered first. block/bio.c | 6 +++--- block/blk.h | 12 ++++++++++++ include/linux/bio.h | 3 ++- include/linux/blk_types.h | 1 + 4 files changed, 18 insertions(+), 4 deletions(-) diff --git a/block/bio.c b/block/bio.c index 4ff96a0e4091..51ae957cc4b6 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1168,7 +1168,7 @@ void __bio_release_pages(struct bio *bio, bool mark_dirty) bio_for_each_segment_all(bvec, bio, iter_all) { if (mark_dirty && !PageCompound(bvec->bv_page)) set_page_dirty_lock(bvec->bv_page); - put_page(bvec->bv_page); + bio_release_page(bio, bvec->bv_page); } } EXPORT_SYMBOL_GPL(__bio_release_pages); @@ -1488,8 +1488,8 @@ void bio_set_pages_dirty(struct bio *bio) * the BIO and re-dirty the pages in process context. * * It is expected that bio_check_pages_dirty() will wholly own the BIO from - * here on. It will run one put_page() against each page and will run one - * bio_put() against the BIO. + * here on. It will unpin each page and will run one bio_put() against the + * BIO. */ static void bio_dirty_fn(struct work_struct *work); diff --git a/block/blk.h b/block/blk.h index cc4e8873dfde..d65d96994a94 100644 --- a/block/blk.h +++ b/block/blk.h @@ -432,6 +432,18 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio, struct page *page, unsigned int len, unsigned int offset, unsigned int max_sectors, bool *same_page); +/* + * Clean up a page appropriately, where the page may be pinned, may have a + * ref taken on it or neither. + */ +static inline void bio_release_page(struct bio *bio, struct page *page) +{ + if (bio_flagged(bio, BIO_PAGE_PINNED)) + unpin_user_page(page); + else if (bio_flagged(bio, BIO_PAGE_REFFED)) + put_page(page); +} + struct request_queue *blk_alloc_queue(int node_id); int disk_scan_partitions(struct gendisk *disk, fmode_t mode); diff --git a/include/linux/bio.h b/include/linux/bio.h index b537d03377f0..d8c30c791a9a 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -488,7 +488,8 @@ void zero_fill_bio(struct bio *bio); static inline void bio_release_pages(struct bio *bio, bool mark_dirty) { - if (bio_flagged(bio, BIO_PAGE_REFFED)) + if (bio_flagged(bio, BIO_PAGE_REFFED) || + bio_flagged(bio, BIO_PAGE_PINNED)) __bio_release_pages(bio, mark_dirty); } diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 7daa261f4f98..a0e339ff3d09 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -318,6 +318,7 @@ struct bio { * bio flags */ enum { + BIO_PAGE_PINNED, /* Unpin pages in bio_release_pages() */ BIO_PAGE_REFFED, /* put pages in bio_release_pages() */ BIO_CLONED, /* doesn't own data */ BIO_BOUNCED, /* bio is a bounce bio */ From patchwork Wed Mar 8 14:37:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13165961 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14DDFC6FD1E for ; Wed, 8 Mar 2023 14:38:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3C0826B0083; Wed, 8 Mar 2023 09:38:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 34A346B0085; Wed, 8 Mar 2023 09:38:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1EA6F6B0087; Wed, 8 Mar 2023 09:38:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 113226B0083 for ; Wed, 8 Mar 2023 09:38:42 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id E94E81A06B6 for ; Wed, 8 Mar 2023 14:38:41 +0000 (UTC) X-FDA: 80545987242.19.7F9A503 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf06.hostedemail.com (Postfix) with ESMTP id 3D11918001C for ; Wed, 8 Mar 2023 14:38:40 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MIW8cIro; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678286320; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7b6tezcmsCRtrHxO2uDV+B0gtFM5s5ObrJQQe5H2QZI=; b=NtflZAW3LJ5aih3M/hEi94LTBMjAQNTKkWEpSQCUy4Dn+sCsHjVngXPJpKaZ/x9cEP3kuH TaA4FfMKmVNPaWtaGV4Z4NKwQX/o4epXH06HlrvqAs2Njltj0XNTphrrmGIxEVkETGoESi kFSj/JGSPETbxETMn9ea3RQ41FvcSGU= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=MIW8cIro; spf=pass (imf06.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678286320; a=rsa-sha256; cv=none; b=2ixACymIxhvpx5bdsH+ncVqMN5wz0Mi/n2LQiZgVIji59YlfRIaefsa/j0QoiK0H22QxqG H4DcBne37eoLSSip6kwMLvTqvgv0ODW3/oeGq7DAYuFQ/Q2DXBWaW2QRlHwQJ/Qgq5nn84 FIPIOYivCr7dFbL6UIILbPYhpTr0vWI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678286319; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7b6tezcmsCRtrHxO2uDV+B0gtFM5s5ObrJQQe5H2QZI=; b=MIW8cIroCYZs6LPkKy51w6N0nLba+VioKsQ9W83mLC+m2s1ykDf/63TDg8xDjo6KdZBhCS jxodRbKagV/lTiCu+3Hk3cDBTHYUgnbY3lpHaO6PeT2v18JDP7hWiw2Yy37IfzcSZAJpWn dq7REaTOg/Bo2+m+SU2ZkTfYham4J68= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-63-tDBcsQw1OWOKllv3Uzl2mQ-1; Wed, 08 Mar 2023 09:38:35 -0500 X-MC-Unique: tDBcsQw1OWOKllv3Uzl2mQ-1 Received: from smtp.corp.redhat.com (int-mx09.intmail.prod.int.rdu2.redhat.com [10.11.54.9]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 09511384D043; Wed, 8 Mar 2023 14:38:33 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 067E8492B07; Wed, 8 Mar 2023 14:38:30 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v16 12/13] block: Convert bio_iov_iter_get_pages to use iov_iter_extract_pages Date: Wed, 8 Mar 2023 14:37:53 +0000 Message-Id: <20230308143754.1976726-13-dhowells@redhat.com> In-Reply-To: <20230308143754.1976726-1-dhowells@redhat.com> References: <20230308143754.1976726-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.9 X-Stat-Signature: o55ygkqakthdgwq5a53mkmnt3f9wbrzh X-Rspam-User: X-Rspamd-Queue-Id: 3D11918001C X-Rspamd-Server: rspam06 X-HE-Tag: 1678286320-364251 X-HE-Meta: U2FsdGVkX19ge8t4fcWBKZVvjwWDYZV1JHJwcdyx9ShVhvUC47sJ0zJ9+DrDJp8VDJSUf7w69OU60Ra/wKI/R5dFG0Cy6kNny5/FRbmQU10JrglTrYIlU+q8ta3ReYToF1xZnQSNP8WyhQWYXSqtMDvNXvPBKSUKeZrYiahiix9nY3dsfPYT3mMk7QPbrm7UlQPbNLla+VpLw38Y/jGnGx+VD91MXQhRlGCDOstTw0IcYDD4ZFzP+RKm+vhsFuj/nGwj6rQeciKjUIjCLmaepqfJqNAMiviR1mbVbLNjEUE0vGxuYtCgqqftK/6kU0lWTAZwQUh0h0nWxi16Appb4ZGPO/gVlgGsP4FsuShcvbSmhEl81My2SbsOCbBLZBMrRvrmDf96dl0+jLA2CiHT3bIKByBHUawcsgoinAOKeuBkNoBYtCwerBBjaaq61K7H70z6FL9h8LKQJm/PKdu7ohoTKAKUavn2fGFP26VxnN0m2rIq9N9DyPXZRpVZmaRzKgfHnHrZuW97tkwtL45CS5HO8Yy60tULmDBgsLzMwNY9Xpp9wLFHRDPF/tejyFsal4uzWYdVfcZMQQ7tOdZsp4cqNMr3qJsj2Ol6PJ3KxGOMyCj2a8zPqSXMM91e8es4U789O4q0HuVTgtAsZWY1A5Z5b0L3dT7+y1x8Hbea1koYN7YWS08semxB8IeqrAxT58hO1tlrCjFGyQdqb+gmQyp8vUCInsxNLgfL6W1q/7w1DY8Nx+uGe1szkzD/MLL9ub9agXRzDheMoIU/s5D5dwW0V/4he2ONCq9mpKtC4TyX4xK4DOaXvWqjqZORvoKBeeUURrqpNEhk5tq9ohqNPKvP4E8wTKuxKK3ilIm9x5i7QnALPEJjwac232QYodoQRkNoJpB0PmIleWxOjttp9WQMXbnXfUrsPybV8fMtC/y2RqtO7DzQ+x+s1O1jd/D6c3qFcN8DG4mGif8nykA gM9OzXdT o7uldL9HElMRJvE/SVTrywLexbUIbN4TnIG4bRd31wgYtrbck4jxp5UZ1pBf77I0WOu4OqZ9hDhF2VOsqZUQ55L//V2C8/Wr35Qhw8nqcCsc6N90wmLwAXND02CT/LU8PZCYlVp3bLHOi+tIlc9C0CiQlrCqshFW2FTwtbDsYDt9zYSUmzfWW/lfkbpF9bHcSmV3r6Jjl6P4S33BzYhE/TMkA+v/bZEqJ1xYIWI7Hm8hSWfZ4vYserL+Z/K3WNbLUOaSVZ2yUGuW3pWCWoxvKD4oX8mnM/BCsnLn+DZZTSlm4am1Z7HUbtSu5K04dzyIVFzxHZsj/+CthRWUz0UY/F85hAP/xhWPfZWfKknvV4MMSYpbOdbaVQoX171VeoII5p+jHVc7LMoE7s8FPN8Et6DezdKEqafZY+a10DZky7s7BK8zCKCjlC8jTrevs4HqIACOmCGYAgA0yCy17+8k1TqAibZ9rZAF2xMVWhzIpuyC6vXZvK6f2be0/8Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This will pin pages or leave them unaltered rather than getting a ref on them as appropriate to the iterator. The pages need to be pinned for DIO rather than having refs taken on them to prevent VM copy-on-write from malfunctioning during a concurrent fork() (the result of the I/O could otherwise end up being affected by/visible to the child process). Signed-off-by: David Howells Reviewed-by: Christoph Hellwig Reviewed-by: John Hubbard cc: Al Viro cc: Jens Axboe cc: Jan Kara cc: Matthew Wilcox cc: Logan Gunthorpe cc: linux-block@vger.kernel.org --- Notes: ver #10) - Drop bio_set_cleanup_mode(), open coding it instead. ver #8) - Split the patch up a bit [hch]. - We should only be using pinned/non-pinned pages and not ref'd pages, so adjust the comments appropriately. ver #7) - Don't treat BIO_PAGE_REFFED/PINNED as being the same as FOLL_GET/PIN. ver #5) - Transcribe the FOLL_* flags returned by iov_iter_extract_pages() to BIO_* flags and got rid of bi_cleanup_mode. - Replaced BIO_NO_PAGE_REF to BIO_PAGE_REFFED in the preceding patch. block/bio.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/block/bio.c b/block/bio.c index 51ae957cc4b6..fc98c1c723ca 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1204,7 +1204,7 @@ static int bio_iov_add_page(struct bio *bio, struct page *page, } if (same_page) - put_page(page); + bio_release_page(bio, page); return 0; } @@ -1218,7 +1218,7 @@ static int bio_iov_add_zone_append_page(struct bio *bio, struct page *page, queue_max_zone_append_sectors(q), &same_page) != len) return -EINVAL; if (same_page) - put_page(page); + bio_release_page(bio, page); return 0; } @@ -1229,10 +1229,10 @@ static int bio_iov_add_zone_append_page(struct bio *bio, struct page *page, * @bio: bio to add pages to * @iter: iov iterator describing the region to be mapped * - * Pins pages from *iter and appends them to @bio's bvec array. The - * pages will have to be released using put_page() when done. - * For multi-segment *iter, this function only adds pages from the - * next non-empty segment of the iov iterator. + * Extracts pages from *iter and appends them to @bio's bvec array. The pages + * will have to be cleaned up in the way indicated by the BIO_PAGE_PINNED flag. + * For a multi-segment *iter, this function only adds pages from the next + * non-empty segment of the iov iterator. */ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) { @@ -1264,9 +1264,9 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) * result to ensure the bio's total size is correct. The remainder of * the iov data will be picked up in the next bio iteration. */ - size = iov_iter_get_pages(iter, pages, - UINT_MAX - bio->bi_iter.bi_size, - nr_pages, &offset, extraction_flags); + size = iov_iter_extract_pages(iter, &pages, + UINT_MAX - bio->bi_iter.bi_size, + nr_pages, extraction_flags, &offset); if (unlikely(size <= 0)) return size ? size : -EFAULT; @@ -1299,7 +1299,7 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) iov_iter_revert(iter, left); out: while (i < nr_pages) - put_page(pages[i++]); + bio_release_page(bio, pages[i++]); return ret; } @@ -1334,7 +1334,8 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) return 0; } - bio_set_flag(bio, BIO_PAGE_REFFED); + if (iov_iter_extract_will_pin(iter)) + bio_set_flag(bio, BIO_PAGE_PINNED); do { ret = __bio_iov_iter_get_pages(bio, iter); } while (!ret && iov_iter_count(iter) && !bio_full(bio, 0)); From patchwork Wed Mar 8 14:37:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13165962 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16761C678D5 for ; Wed, 8 Mar 2023 14:38:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AC4846B0085; Wed, 8 Mar 2023 09:38:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A744E6B0087; Wed, 8 Mar 2023 09:38:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 93CFE6B0088; Wed, 8 Mar 2023 09:38:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 83BEB6B0085 for ; Wed, 8 Mar 2023 09:38:46 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 564C5AAC14 for ; Wed, 8 Mar 2023 14:38:46 +0000 (UTC) X-FDA: 80545987452.01.C9BEE7C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf03.hostedemail.com (Postfix) with ESMTP id 91BE420016 for ; Wed, 8 Mar 2023 14:38:44 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=g7wvu3lL; spf=pass (imf03.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678286324; a=rsa-sha256; cv=none; b=xVSvXLYoQsX2F83YU7GMQFKR4+bKzsIsXaChiqmNr0F6rz6/KJ1w4FUsLGpZSDlOWQEABC 0S2TPt6+v2hXsQy/uxZ2HB0kuzxdRvcP2cyZuCXjX8Xl6cg6uLu+DFIPPckvPA87A9QVWy 82AUIBzg89vM77yn7vqSgSDW3XNkwQs= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=g7wvu3lL; spf=pass (imf03.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678286324; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gIKMSgtKI0+T2jjDihR3uNZ9dKHoZzHE/RnqYef6EBU=; b=1ttZpvPebCpVKzZO/gVzfXWczEWJgE2A6kNNeqiWhbFKBkrB8GP0avx69G+hB9zjwX9Jv9 iYmDebPKtIC2jTTSi/2GaT2RyaaEzAFs6r5YM5ml2JtpFDMD3e4kXB7gsBBABZQbFJwHXf rnD3bahLzBYU2rWJSVQCC4L93siWACE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678286324; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gIKMSgtKI0+T2jjDihR3uNZ9dKHoZzHE/RnqYef6EBU=; b=g7wvu3lLUQ8KJN1Z7rHEGmI8bIL8oZj8Qr5ZDcVbk7+OsthW/ENOBnraRBRhMEFK9DQMbn HYrF3mV10WMkY0dW7NFNCLRYDdE4ATfNWvGb2k0dUunJWVtUm1b3IGrXh7GOlZDgWmsAHh YYKC8mqOwSYvLWdrfy88B9WpCYf1+qw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-561-Gwr4SSb3NC615tqa25vaUQ-1; Wed, 08 Mar 2023 09:38:36 -0500 X-MC-Unique: Gwr4SSb3NC615tqa25vaUQ-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A4D16858F09; Wed, 8 Mar 2023 14:38:35 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.33.36.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9EC4C440D9; Wed, 8 Mar 2023 14:38:33 +0000 (UTC) From: David Howells To: Jens Axboe , Al Viro , Christoph Hellwig Cc: David Howells , Matthew Wilcox , Jan Kara , Jeff Layton , David Hildenbrand , Jason Gunthorpe , Logan Gunthorpe , Hillf Danton , Linus Torvalds , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , John Hubbard Subject: [PATCH v16 13/13] block: convert bio_map_user_iov to use iov_iter_extract_pages Date: Wed, 8 Mar 2023 14:37:54 +0000 Message-Id: <20230308143754.1976726-14-dhowells@redhat.com> In-Reply-To: <20230308143754.1976726-1-dhowells@redhat.com> References: <20230308143754.1976726-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Rspam-User: X-Rspamd-Queue-Id: 91BE420016 X-Rspamd-Server: rspam01 X-Stat-Signature: z7uqkjswfuawdfyiyrxrbixsethrnzs1 X-HE-Tag: 1678286324-729757 X-HE-Meta: U2FsdGVkX18AeFQ0IQnsERhA2Erb50mqNhgSTVd++2bVJG2mNLMhjTzGDjjxR0zlpXfWbLCl1PlZv/M6J4MiEh1qOF7p2YXDAir4I8aM6ojlZ+4y6USXpMtc3miPnDb6TASEqEF1XQrD5RhoFtfTVVEc4the9yUhDSzUpOl4Go8iZP2xZPgVEWhjIkN2vSWuTWgtoqVlHKguhT/XhSnh2n8KM8HYrY4XeKUMJa0+ZKm9qbynC+EAFNoQCGcGXhmir1AxoF6UhpJ4MF8pFojHINqBdlrMVeLxvkp0udTj5bGZSWVsvbE+nAWQ21j0lzGBj/G03pVxe1rBxDlONUld4/HDWxy/+dRas9QllAmNhv8xVDjYqyy+DNxK7Y7L5dvv84nRq0/KNpus67Ut4N8i0IqWhXVduzLQs/X7/5ZpxsL6pk+s2rS1NLi0BZOfWCC5eb+2FCGND3jaJAQIKnjboK2p6TaFRcD5cOBts8ogm5fi6pVTiL0gP/uN4enonyhCEp9o/RPEQ50p7NA5T6OhlwKVJiH+1okiwYzqRsP5yTYHEH0phNwAuyh9/nGYtARLs9P7f9Z1ILP/+1dm+oonQi0NPPyp4L0DDeCmQe/0tuflL/Bat0HRn657WCFCFI2e9MI9/dg7mR+nGqlh9GHs0bnwMw+bWbyaf02R1HwIUzUScLsquAErxfqlLIC+CtpvSt483SbA5/kPstoQp8nfLMWPWGY7I5jICcKltimMQg4TVqDJqlMkcqyBn6XU+Ig3cmesh+KX/VqgLuT6p/YBmgQLZl4dYxjcN6YcsCVhz8jOZSZzUN7v1hdlv03p1Op3YZ87zOnzlWMErBGq0zrEvIVU0Fk5xBap+B5VqiGhkTH5i8yjvDabosGzkcbOzz0WhuLtdbmi3j3gB+ZEq/Am/LNkQ1ytWbX4xSE4FKbef/Kwiow7msY90QTCitrEkn/09ngRC5DX/JkZ0lFRrhB HTtvTBbU /mjAdHwurF5YdJeD6sXN2WZhdWh5vM03Dgo8jB6RBcN6UppeLJQ89cYTLRAMyOkGxc9GexyP9aGbM3c3fuGSjvHg/SOtt0NQdaNLaML3KgMRvBpp12erl1AFRAEx+bakECFbLzamjIIuYjPne/Bg1bgsStqeEpkwqNcNEJ+Vkh0AJbKMo1Yv7w7xErgEzDJH/Y19eu/4xzE/TW1vKzy2eSZhZau1Rc+lTHhtj0DZrFlrmf0IkZ7i7uwEx0Ri8f+lgKJ7zJl92YtK6Wnzr26CRLUHidV4owkv+nUbTkOqe43vgPokDycnr0jtd8jeDUXUy/D44B0WNxdiVDZb0xlxQKVhOe61x/WQ62Bs7u06kOtcZZCR/6FhwXZjZo6e/9Vud1kC/Q47MdWaV13VJVWKzZTunHsSb9+dUpJnGqwDmIitEqjP4BwVdvRl+U0VKWbU8TjjIftdp6RYSCg96ELFssxkbroj1Sgz5Qjf0zTKdbDKl4/Tol5Ux125wmw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This will pin pages or leave them unaltered rather than getting a ref on them as appropriate to the iterator. The pages need to be pinned for DIO rather than having refs taken on them to prevent VM copy-on-write from malfunctioning during a concurrent fork() (the result of the I/O could otherwise end up being visible to/affected by the child process). Signed-off-by: David Howells Reviewed-by: Christoph Hellwig Reviewed-by: John Hubbard cc: Al Viro cc: Jens Axboe cc: Jan Kara cc: Matthew Wilcox cc: Logan Gunthorpe cc: linux-block@vger.kernel.org --- Notes: ver #10) - Drop bio_set_cleanup_mode(), open coding it instead. ver #8) - Split the patch up a bit [hch]. - We should only be using pinned/non-pinned pages and not ref'd pages, so adjust the comments appropriately. ver #7) - Don't treat BIO_PAGE_REFFED/PINNED as being the same as FOLL_GET/PIN. ver #5) - Transcribe the FOLL_* flags returned by iov_iter_extract_pages() to BIO_* flags and got rid of bi_cleanup_mode. - Replaced BIO_NO_PAGE_REF to BIO_PAGE_REFFED in the preceding patch. block/blk-map.c | 23 +++++++++++------------ 1 file changed, 11 insertions(+), 12 deletions(-) diff --git a/block/blk-map.c b/block/blk-map.c index c77fdb1fbda7..7b12f4bb4d4c 100644 --- a/block/blk-map.c +++ b/block/blk-map.c @@ -280,22 +280,21 @@ static int bio_map_user_iov(struct request *rq, struct iov_iter *iter, if (blk_queue_pci_p2pdma(rq->q)) extraction_flags |= ITER_ALLOW_P2PDMA; + if (iov_iter_extract_will_pin(iter)) + bio_set_flag(bio, BIO_PAGE_PINNED); - bio_set_flag(bio, BIO_PAGE_REFFED); while (iov_iter_count(iter)) { - struct page **pages, *stack_pages[UIO_FASTIOV]; + struct page *stack_pages[UIO_FASTIOV]; + struct page **pages = stack_pages; ssize_t bytes; size_t offs; int npages; - if (nr_vecs <= ARRAY_SIZE(stack_pages)) { - pages = stack_pages; - bytes = iov_iter_get_pages(iter, pages, LONG_MAX, - nr_vecs, &offs, extraction_flags); - } else { - bytes = iov_iter_get_pages_alloc(iter, &pages, - LONG_MAX, &offs, extraction_flags); - } + if (nr_vecs > ARRAY_SIZE(stack_pages)) + pages = NULL; + + bytes = iov_iter_extract_pages(iter, &pages, LONG_MAX, + nr_vecs, extraction_flags, &offs); if (unlikely(bytes <= 0)) { ret = bytes ? bytes : -EFAULT; goto out_unmap; @@ -317,7 +316,7 @@ static int bio_map_user_iov(struct request *rq, struct iov_iter *iter, if (!bio_add_hw_page(rq->q, bio, page, n, offs, max_sectors, &same_page)) { if (same_page) - put_page(page); + bio_release_page(bio, page); break; } @@ -329,7 +328,7 @@ static int bio_map_user_iov(struct request *rq, struct iov_iter *iter, * release the pages we didn't map into the bio, if any */ while (j < npages) - put_page(pages[j++]); + bio_release_page(bio, pages[j++]); if (pages != stack_pages) kvfree(pages); /* couldn't stuff something into bio? */