From patchwork Wed Jun 19 02:34:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kundan Kumar X-Patchwork-Id: 13703375 Received: from mailout1.samsung.com (mailout1.samsung.com [203.254.224.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69F412A8D3 for ; Wed, 19 Jun 2024 04:00:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.24 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718769634; cv=none; b=XJJHdCvf99y/PjLNc2+iTvRGODb4qFMDd06vLJczi+V64+PQSbf91F7XCIfx1OX0JZy/UfEn7G3D9ACQuB3noXouY0dz7If4xLI7Fkx42HbbbU4i7rHiHB1m3ZPhaBAERm7ixQymHDH1O1zi2qVzbc5e8Ctxw9zthvVeUwyVtUw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718769634; c=relaxed/simple; bh=YTlkJRIOJIbkKTZubAroWx4vpbjYjDL/ePvjM+YhBNg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:MIME-Version: Content-Type:References; b=EzUswQsP8zMAkfXiBonCn59Nza7YVv/3zD6InznfXLq0IxnPAjUFBpWkwDAG2wTlxh2i/NyVHwLb79yCXezWv4Hyx4bfeXfGKAHu7voDelDiy+jJIZRbgvYSDJjwXxem79XUGqoKuEb3dRmVbQREOY2rCrXAYdLru4rebJ70le8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=gWZ6DUKz; arc=none smtp.client-ip=203.254.224.24 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="gWZ6DUKz" Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20240619040030epoutp01d574a714544cbfe6ef43d73d7da549b3~aS99PzkSV1170711707epoutp01o for ; Wed, 19 Jun 2024 04:00:30 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20240619040030epoutp01d574a714544cbfe6ef43d73d7da549b3~aS99PzkSV1170711707epoutp01o DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1718769630; bh=BTtyefl8JwRad/XXg1ppvQ7OTg1YYjTD32mp9iAFC4w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gWZ6DUKz2+RoPQiZl3hRRH3Sg23etWQJtnfpi1pIXtt4TiY4xxW6wQBAWIoE8AUCC QoDroRWPMpVFTOq6kyS5Csxfq5ssVfT+9pSSksdrfnxHEl7arGhfduei6atBiBPSwt 6ngf7F1QFA6kOHAJ1N24JnpRJzFfCQNIbKgb2cN0= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20240619040029epcas5p1231097f6f8b00838594a6aeaed8fc8e8~aS980RkgX0336603366epcas5p1G; Wed, 19 Jun 2024 04:00:29 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.178]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4W3qf36tmLz4x9Pv; Wed, 19 Jun 2024 04:00:27 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 6B.01.06857.AD752766; Wed, 19 Jun 2024 13:00:26 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20240619024146epcas5p15357534fb7410c212743162b351e27e8~aR5NzhP1n2032620326epcas5p1g; Wed, 19 Jun 2024 02:41:46 +0000 (GMT) Received: from epsmgmc1p1new.samsung.com (unknown [182.195.42.40]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20240619024146epsmtrp2be20ae7dcc31a15fc2fda6829f297bc9~aR5NyjRF20348503485epsmtrp2s; Wed, 19 Jun 2024 02:41:46 +0000 (GMT) X-AuditID: b6c32a4b-88bff70000021ac9-15-667257dad335 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgmc1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 56.84.07412.A6542766; Wed, 19 Jun 2024 11:41:46 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240619024144epsmtip23ddc67300aa9f9026fcae26bab63d4db~aR5MDAe4D0512005120epsmtip2W; Wed, 19 Jun 2024 02:41:44 +0000 (GMT) From: Kundan Kumar To: axboe@kernel.dk, hch@lst.de, willy@infradead.org, kbusch@kernel.org Cc: linux-block@vger.kernel.org, joshi.k@samsung.com, mcgrof@kernel.org, anuj20.g@samsung.com, nj.shetty@samsung.com, c.gameti@samsung.com, gost.dev@samsung.com, Kundan Kumar Subject: [PATCH v5 1/3] block: Added folio-lized version of bio_add_hw_page() Date: Wed, 19 Jun 2024 08:04:18 +0530 Message-Id: <20240619023420.34527-2-kundan.kumar@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240619023420.34527-1-kundan.kumar@samsung.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrFJsWRmVeSWpSXmKPExsWy7bCmlu6t8KI0g+unOCyaJvxltlh9t5/N 4vv2PhaLmwd2MlmsXH2UyeLo/7dsFpMOXWO02PrlK6vF3lvaFjcmPGW02PZ7PrPF7x9z2Bx4 PDav0PK4fLbUY9OqTjaP3Tcb2Dz6tqxi9Pi8SS6ALSrbJiM1MSW1SCE1Lzk/JTMv3VbJOzje Od7UzMBQ19DSwlxJIS8xN9VWycUnQNctMwfoQiWFssScUqBQQGJxsZK+nU1RfmlJqkJGfnGJ rVJqQUpOgUmBXnFibnFpXrpeXmqJlaGBgZEpUGFCdsb3PYtYC57JVbybPYWxgXGRWBcjJ4eE gInEy7m/mbsYuTiEBHYzSnS9us8E4XxilHjw6DI7hPONUWL5xbvMMC0N9+9DJfYySux9eJwF wvnMKNEx5x1bFyMHB5uArsSPplCQBhEBd4mpLx8xgtQwC5xllDgx9RELSEJYwF9iY8NGNhCb RUBV4t7jzYwgNq+ArcT9Bf2MENvkJWZe+s4OMpNTwE7i5xVriBJBiZMzn4CNYQYqad46G+wH CYGJHBIHd8xmAqmXEHCRWLzGD2KMsMSr41vYIWwpic/v9rJB2NkShxo3MEHYJRI7jzRA1dhL tJ7qZwYZwyygKbF+lz5EWFZi6ql1TBBr+SR6fz+BauWV2DEPxlaTmPNuKguELSOx8NIMqLiH ROvSH2yQoJrIKLFjxWfWCYwKs5C8MwvJO7MQVi9gZF7FKJlaUJybnlpsWmCcl1oOj+Tk/NxN jOBEq+W9g/HRgw96hxiZOBgPMUpwMCuJ8DpNy0sT4k1JrKxKLcqPLyrNSS0+xGgKDO6JzFKi yfnAVJ9XEm9oYmlgYmZmZmJpbGaoJM77unVuipBAemJJanZqakFqEUwfEwenVAPTsaYtyr5y bhXfFnlw2zRf2DTrldDOR87RzPVzlMVu/tNiryqxPr182uNVR3n51yUyzxBtr3CZVH7dzrdJ gXuX0/V52amTlEQN1fOk27YcvFLS7b/3950NV7Y4/T4VqrIo+vZN8+vVRUc+Ld+z9VNPjMz2 zkOvc9/OT5ectGuNR0ZcnsKaO7FznDQi5h7I1Hhh8Xx5g4j6Nv0b6xL5riRPzTo+585v1dYW VoZTLuH563ZP+3PzeALj+uS5E8s/MRgcuM/zQLMmUf/c06W7PqhH5dQcS9Fb0vXtOr/DZOVf 4myGG24fmNL2+fT1uyUTT7DtvbT0ppCDV4XEoqt+82pXNYuKzZhWVrP9+vM1+xh1HJVYijMS DbWYi4oTAZpPW389BAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrILMWRmVeSWpSXmKPExsWy7bCSvG6Wa1GaQed/XoumCX+ZLVbf7Wez +L69j8Xi5oGdTBYrVx9lsjj6/y2bxaRD1xgttn75ymqx95a2xY0JTxkttv2ez2zx+8ccNgce j80rtDwuny312LSqk81j980GNo++LasYPT5vkgtgi+KySUnNySxLLdK3S+DK+L5nEWvBM7mK d7OnMDYwLhLrYuTkkBAwkWi4f5+9i5GLQ0hgN6PEnZV/2CASMhK77+5khbCFJVb+ew5V9JFR Ys7cmUAOBwebgK7Ej6ZQkBoRAV+JBRueM4LUMAtcZ5S4MX0rM0hCGCjx+sFFdhCbRUBV4t7j zYwgNq+ArcT9Bf2MEAvkJWZe+g42k1PATuLnFWuQsBBQSd+KDywQ5YISJ2c+AbOZgcqbt85m nsAoMAtJahaS1AJGplWMkqkFxbnpucmGBYZ5qeV6xYm5xaV56XrJ+bmbGMGxoKWxg/He/H96 hxiZOBgPMUpwMCuJ8DpNy0sT4k1JrKxKLcqPLyrNSS0+xCjNwaIkzms4Y3aKkEB6Yklqdmpq QWoRTJaJg1OqgSlK5aNhzfe1S0qdDffEPS0L/cJ7TGvSvCm7/j9ZEVCls5HDaXdbo5rY9Uy5 lY1TxbTU9/zr5Ji87eaxtCvzo78ZhqRc+3poVehsrnCvf6ICL7/ObWdiO3L937GkZ8Ydc2pv S7loqC50XnzVM0n2H8+zd6dXnH4yX0rfvOPOpTkyq5NPL31502+SwOwbq/xeTpmT2NU+7/9M NlUHhtVB3tf47XZvfL6Kba2K+B2WQ3rhVr8MW29Gqrp+zAk602h6+oQJg6KWyFfzI35frCvn T/+VkJGgG84T9WBPcQP/3ad8MyRk/7Jz7L/HofP03SF7s7Kpni+6Fu3VkDB7tujQUsuI/hmb 3Kst5+XuSy5LqZ+qxFKckWioxVxUnAgAWiZgn/QCAAA= X-CMS-MailID: 20240619024146epcas5p15357534fb7410c212743162b351e27e8 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240619024146epcas5p15357534fb7410c212743162b351e27e8 References: <20240619023420.34527-1-kundan.kumar@samsung.com> Added new bio_add_hw_folio() function. This is a prep patch. Signed-off-by: Kundan Kumar --- block/bio.c | 38 +++++++++++++++++++++++++++++--------- block/blk.h | 4 ++++ 2 files changed, 33 insertions(+), 9 deletions(-) -- 2.25.1 diff --git a/block/bio.c b/block/bio.c index e9e809a63c59..c8914febb16e 100644 --- a/block/bio.c +++ b/block/bio.c @@ -964,7 +964,7 @@ bool bvec_try_merge_hw_page(struct request_queue *q, struct bio_vec *bv, } /** - * bio_add_hw_page - attempt to add a page to a bio with hw constraints + * bio_add_hw_page - a wrapper around function bio_add_hw_folio * @q: the target queue * @bio: destination bio * @page: page to add @@ -972,13 +972,35 @@ bool bvec_try_merge_hw_page(struct request_queue *q, struct bio_vec *bv, * @offset: vec entry offset * @max_sectors: maximum number of sectors that can be added * @same_page: return if the segment has been merged inside the same page - * - * Add a page to a bio while respecting the hardware max_sectors, max_segment - * and gap limitations. */ int bio_add_hw_page(struct request_queue *q, struct bio *bio, struct page *page, unsigned int len, unsigned int offset, unsigned int max_sectors, bool *same_page) +{ + struct folio *folio = page_folio(page); + size_t folio_offset = (folio_page_idx(folio, page) << PAGE_SHIFT) + + offset; + + return bio_add_hw_folio(q, bio, folio, len, folio_offset, max_sectors, + same_page); +} + +/** + * bio_add_hw_folio - attempt to add a folio to a bio with hw constraints + * @q: the target queue + * @bio: destination bio + * @folio: folio to add + * @len: vec entry length + * @offset: vec entry offset in the folio + * @max_sectors: maximum number of sectors that can be added + * @same_page: return if the segment has been merged inside the same page + * + * Add a folio to a bio while respecting the hardware max_sectors, max_segment + * and gap limitations. + */ +int bio_add_hw_folio(struct request_queue *q, struct bio *bio, + struct folio *folio, unsigned int len, unsigned int offset, + unsigned int max_sectors, bool *same_page) { unsigned int max_size = max_sectors << SECTOR_SHIFT; @@ -992,8 +1014,8 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio, if (bio->bi_vcnt > 0) { struct bio_vec *bv = &bio->bi_io_vec[bio->bi_vcnt - 1]; - if (bvec_try_merge_hw_page(q, bv, page, len, offset, - same_page)) { + if (bvec_try_merge_hw_page(q, bv, folio_page(folio, 0), len, + offset, same_page)) { bio->bi_iter.bi_size += len; return len; } @@ -1010,9 +1032,7 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio, return 0; } - bvec_set_page(&bio->bi_io_vec[bio->bi_vcnt], page, len, offset); - bio->bi_vcnt++; - bio->bi_iter.bi_size += len; + bio_add_folio_nofail(bio, folio, len, offset); return len; } diff --git a/block/blk.h b/block/blk.h index 79e8d5d4fe0c..d0bec44a2ffb 100644 --- a/block/blk.h +++ b/block/blk.h @@ -524,6 +524,10 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio, struct page *page, unsigned int len, unsigned int offset, unsigned int max_sectors, bool *same_page); +int bio_add_hw_folio(struct request_queue *q, struct bio *bio, + struct folio *folio, unsigned int len, unsigned int offset, + unsigned int max_sectors, bool *same_page); + /* * Clean up a page appropriately, where the page may be pinned, may have a * ref taken on it or neither. From patchwork Wed Jun 19 02:34:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kundan Kumar X-Patchwork-Id: 13703376 Received: from mailout1.samsung.com (mailout1.samsung.com [203.254.224.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E49651E878 for ; Wed, 19 Jun 2024 04:00:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.24 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718769635; cv=none; b=VRGHyEtzAqOgmyw8KoyCOzkmg1AK39BGt0EBQu+sWpQ3WFpZZcs221ww7Y/07wMFxAQPiE50EfstNNu+eOzFS2vEuyfVi8yjCle/xf9EgJtThf0vj16/Nd3/gQJ8GidDA7ZCWNuqAcovKMbMhRzVmVTlOMznBcPzUKjAUgvA1hs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718769635; c=relaxed/simple; bh=yh714mMjhBzx083r6+ngC/PU0ZhtaKddSeV6+SjZw8o=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:MIME-Version: Content-Type:References; b=TCmWAO74Q6/8pksIYrHZJ2mJxgPF9Y/AYTRLon+a2vl0pOpiVz49HVqFSasI3wRSM1oXbdLhyfRCQRI4bSbcGfqZtDnHB/7Uyg25emHYlqB1ej4t+FL9lZ9GoX2lnR69onFvBl7pA79+UwL9Hu/lsNSgo0gqvnhA3bux8rQv9z0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=A+O+7ami; arc=none smtp.client-ip=203.254.224.24 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="A+O+7ami" Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20240619040031epoutp01516cd9d8ccdba7e9584b715c2c9fe8dc~aS9_CFA-Y1249412494epoutp01b for ; Wed, 19 Jun 2024 04:00:31 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20240619040031epoutp01516cd9d8ccdba7e9584b715c2c9fe8dc~aS9_CFA-Y1249412494epoutp01b DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1718769631; bh=gTaEFuMQdfljl8wzbgFV7DjFGFvYaSv9FTBFs1jOmgU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A+O+7amiP1zUc3ncARLdPui6FB6SGxhn2/V/4rjSAGUcDOa925wWHJr4uIszefrsL viHQ9/C7sveHuzZsRsKzoE6sM9hrGmE7/zOhybadZ1Fk3iFea6wvR5LFQo+m9NDWTI bYCyO0WP9CGGBq3BPj+1pUqyarIhuZuheLyJ77wI= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20240619040030epcas5p1eddd02c57896a8495bfeecd845ae4703~aS99ilRTA2658426584epcas5p1B; Wed, 19 Jun 2024 04:00:30 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.174]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4W3qf45m02z4x9QB; Wed, 19 Jun 2024 04:00:28 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id B9.8B.10047.CD752766; Wed, 19 Jun 2024 13:00:28 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20240619024150epcas5p267bd3cbd24061e723a7b632746de92d6~aR5RtXtty2382823828epcas5p2m; Wed, 19 Jun 2024 02:41:50 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20240619024150epsmtrp22795ceb696a7d2c3118ffa4e19de1a8b~aR5RsecYX0340503405epsmtrp2k; Wed, 19 Jun 2024 02:41:50 +0000 (GMT) X-AuditID: b6c32a49-1d5fa7000000273f-13-667257dc2c5b Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 78.9A.29940.E6542766; Wed, 19 Jun 2024 11:41:50 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240619024148epsmtip27ea1bf4604f93039e7a1f56d305b23c7~aR5P_-8yA0156201562epsmtip26; Wed, 19 Jun 2024 02:41:48 +0000 (GMT) From: Kundan Kumar To: axboe@kernel.dk, hch@lst.de, willy@infradead.org, kbusch@kernel.org Cc: linux-block@vger.kernel.org, joshi.k@samsung.com, mcgrof@kernel.org, anuj20.g@samsung.com, nj.shetty@samsung.com, c.gameti@samsung.com, gost.dev@samsung.com, Kundan Kumar Subject: [PATCH v5 2/3] block: add folio awareness instead of looping through pages Date: Wed, 19 Jun 2024 08:04:19 +0530 Message-Id: <20240619023420.34527-3-kundan.kumar@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240619023420.34527-1-kundan.kumar@samsung.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrFJsWRmVeSWpSXmKPExsWy7bCmlu6d8KI0g7YVOhZNE/4yW6y+289m 8X17H4vFzQM7mSxWrj7KZHH0/1s2i0mHrjFabP3yldVi7y1tixsTnjJabPs9n9ni9485bA48 HptXaHlcPlvqsWlVJ5vH7psNbB59W1YxenzeJBfAFpVtk5GamJJapJCal5yfkpmXbqvkHRzv HG9qZmCoa2hpYa6kkJeYm2qr5OIToOuWmQN0oZJCWWJOKVAoILG4WEnfzqYov7QkVSEjv7jE Vim1ICWnwKRArzgxt7g0L10vL7XEytDAwMgUqDAhO+P5m8MsBRf1Ko7eXs/WwNgt18XIySEh YCJx4tNUdhBbSGA3o8TDK7VdjFxA9idGif6Dj1ghnG+MEo8/PmWC6djZfJ4RIrGXUWLnvYvM EM5nRon3M2cCZTg42AR0JX40hYI0iAi4S0x9+QisgVngLKPEiamPWEASwgKhEvcOLgWbyiKg KtF54x8ziM0rYCvx7MUPdoht8hIzL31nB5nJKWAn8fOKNUSJoMTJmU/AxjADlTRvnQ12g4TA RA6Jpk39jBC9LhJzrl9hgbCFJV4d3wI1U0ri87u9bBB2tsShxg1Qn5VI7DzSAFVjL9F6qp8Z ZC+zgKbE+l36EGFZiamn1jFB7OWT6P39BKqVV2LHPBhbTWLOu6lQa2UkFl6aARX3kHgwrR0a ohMZJf6+esQ0gVFhFpJ/ZiH5ZxbC6gWMzKsYJVMLinPTU4tNCwzzUsvhkZycn7uJEZxotTx3 MN598EHvECMTB+MhRgkOZiURXqdpeWlCvCmJlVWpRfnxRaU5qcWHGE2B4T2RWUo0OR+Y6vNK 4g1NLA1MzMzMTCyNzQyVxHlft85NERJITyxJzU5NLUgtgulj4uCUamAqPr0gzYQ55YHi9fBz b3MrBGxnKy1aJihn91Ak/9K6i77njIVmxf6/n/vs5MeJl075JLAnHFOJ9gu0KLxjnVS7v0jM uuKXt/uRrRt/XVRsCXjwQZGL7YL91YSSdq8L7qvuqu7Kvly6x7Ul6Abnx8pXKj6zE45PqC/f aS3+xtPo5KzqsPBfwtkvazPvzwtZ4tlp1VwjZXSUN0TiWSv/jYj7F+7c4FzDfiSOc7Fn84xJ s/7pK3/Prz3rULfrrd2CgyVPrk24oszMerH30wQl2xVGCa+3CPpuNfjUpGxw9EH/DrXpfMl2 LxczmXdsOCFtJntEXvd3CPcxzml1sq/Oxj0yDT14P+Zof2414/r/tcJKLMUZiYZazEXFiQC1 HgzYPQQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrALMWRmVeSWpSXmKPExsWy7bCSvG6ea1GaQct9GYumCX+ZLVbf7Wez +L69j8Xi5oGdTBYrVx9lsjj6/y2bxaRD1xgttn75ymqx95a2xY0JTxkttv2ez2zx+8ccNgce j80rtDwuny312LSqk81j980GNo++LasYPT5vkgtgi+KySUnNySxLLdK3S+DKeP7mMEvBRb2K o7fXszUwdst1MXJySAiYSOxsPs/YxcjFISSwm1Hiy7mDbBAJGYndd3eyQtjCEiv/PWeHKPrI KHF62RqgDg4ONgFdiR9NoSA1IgK+Egs2PAcbxCxwnVHixvStzCA1wgLBEqebk0FqWARUJTpv /GMGsXkFbCWevfjBDjFfXmLmpe/sIOWcAnYSP69Yg4SFgEr6VnxggSgXlDg58wmYzQxU3rx1 NvMERoFZSFKzkKQWMDKtYpRMLSjOTc8tNiwwzEst1ytOzC0uzUvXS87P3cQIjgQtzR2M21d9 0DvEyMTBeIhRgoNZSYTXaVpemhBvSmJlVWpRfnxRaU5q8SFGaQ4WJXFe8Re9KUIC6Yklqdmp qQWpRTBZJg5OqQamC4ZcL5x//WZ/Hed3PNuk+qD21nBjzzvPzvPclpyctLd89SGdvU5Zb/+z /S3PzbnS5fbk163JBwKue+asC39wzqFpd7rztw/bW+3aTnVVF3TzCJR8MNRZ/jTEmm3DV4GF 20++2L111Z8pDo9kXzHbfWszD+oszvLT04j8OF1dPTZJYcETY8WbiVPMZ59eUlcYEHbippmA TOj1Gb0P7dauEk+49rC/ljMz5YFd4ToLq7nGzsICFVc/ctbH8zKtcwq6tWJNokBD9aPrLztj dp9omW7xv6Yn+P41vvs8H5MPftt9uK6FxefBcbPNT7Ifye1v6XVLOlkg4DA7+aXBnyW1Ytwd np8/8ZzRkdvXbvlfiaU4I9FQi7moOBEAkpeSy/MCAAA= X-CMS-MailID: 20240619024150epcas5p267bd3cbd24061e723a7b632746de92d6 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240619024150epcas5p267bd3cbd24061e723a7b632746de92d6 References: <20240619023420.34527-1-kundan.kumar@samsung.com> Add a bigger size from folio to bio and skip merge processing for pages. Fetch the offset of page within a folio. Depending on the size of folio and folio_offset, fetch a larger length. This length may consist of multiple contiguous pages if folio is multiorder. Using the length calculate number of pages which will be added to bio and increment the loop counter to skip those pages. Using a helper function check if pages are contiguous and belong to same folio, this is done as a COW may happen and change contiguous mapping of pages of folio. This technique helps to avoid overhead of merging pages which belong to same large order folio. Also folio-lize the functions bio_iov_add_page() and bio_iov_add_zone_append_page() Signed-off-by: Kundan Kumar --- block/bio.c | 72 ++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 58 insertions(+), 14 deletions(-) -- 2.25.1 diff --git a/block/bio.c b/block/bio.c index c8914febb16e..3e75b5b0eb6e 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1224,7 +1224,7 @@ void bio_iov_bvec_set(struct bio *bio, struct iov_iter *iter) bio_set_flag(bio, BIO_CLONED); } -static int bio_iov_add_page(struct bio *bio, struct page *page, +static int bio_iov_add_folio(struct bio *bio, struct folio *folio, unsigned int len, unsigned int offset) { bool same_page = false; @@ -1234,30 +1234,60 @@ static int bio_iov_add_page(struct bio *bio, struct page *page, if (bio->bi_vcnt > 0 && bvec_try_merge_page(&bio->bi_io_vec[bio->bi_vcnt - 1], - page, len, offset, &same_page)) { + folio_page(folio, 0), len, offset, + &same_page)) { bio->bi_iter.bi_size += len; if (same_page) - bio_release_page(bio, page); + bio_release_page(bio, folio_page(folio, 0)); return 0; } - __bio_add_page(bio, page, len, offset); + bio_add_folio_nofail(bio, folio, len, offset); return 0; } -static int bio_iov_add_zone_append_page(struct bio *bio, struct page *page, +static int bio_iov_add_zone_append_folio(struct bio *bio, struct folio *folio, unsigned int len, unsigned int offset) { struct request_queue *q = bdev_get_queue(bio->bi_bdev); bool same_page = false; - if (bio_add_hw_page(q, bio, page, len, offset, + if (bio_add_hw_folio(q, bio, folio, len, offset, queue_max_zone_append_sectors(q), &same_page) != len) return -EINVAL; if (same_page) - bio_release_page(bio, page); + bio_release_page(bio, folio_page(folio, 0)); return 0; } +static unsigned int get_contig_folio_len(int *num_pages, struct page **pages, + int i, struct folio *folio, + ssize_t left, size_t offset) +{ + ssize_t bytes = left; + size_t contig_sz = min_t(size_t, PAGE_SIZE - offset, bytes); + unsigned int j; + + /* + * We might COW a single page in the middle of + * a large folio, so we have to check that all + * pages belong to the same folio. + */ + bytes -= contig_sz; + for (j = i + 1; j < i + *num_pages; j++) { + size_t next = min_t(size_t, PAGE_SIZE, bytes); + + if (page_folio(pages[j]) != folio || + pages[j] != pages[j - 1] + 1) { + break; + } + contig_sz += next; + bytes -= next; + } + *num_pages = j - i; + + return contig_sz; +} + #define PAGE_PTRS_PER_BVEC (sizeof(struct bio_vec) / sizeof(struct page *)) /** @@ -1277,9 +1307,9 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) unsigned short entries_left = bio->bi_max_vecs - bio->bi_vcnt; struct bio_vec *bv = bio->bi_io_vec + bio->bi_vcnt; struct page **pages = (struct page **)bv; - ssize_t size, left; - unsigned len, i = 0; - size_t offset; + ssize_t size, left, len; + unsigned int i = 0, num_pages; + size_t offset, folio_offset; int ret = 0; /* @@ -1321,15 +1351,29 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) for (left = size, i = 0; left > 0; left -= len, i++) { struct page *page = pages[i]; + struct folio *folio = page_folio(page); + + folio_offset = (folio_page_idx(folio, page) << PAGE_SHIFT) + + offset; + + len = min_t(size_t, (folio_size(folio) - folio_offset), left); + + num_pages = DIV_ROUND_UP(offset + len, PAGE_SIZE); + + if (num_pages > 1) + len = get_contig_folio_len(&num_pages, pages, i, + folio, left, offset); - len = min_t(size_t, PAGE_SIZE - offset, left); if (bio_op(bio) == REQ_OP_ZONE_APPEND) { - ret = bio_iov_add_zone_append_page(bio, page, len, - offset); + ret = bio_iov_add_zone_append_folio(bio, folio, len, + folio_offset); if (ret) break; } else - bio_iov_add_page(bio, page, len, offset); + bio_iov_add_folio(bio, folio, len, folio_offset); + + /* Skip the pages which got added */ + i = i + (num_pages - 1); offset = 0; } From patchwork Wed Jun 19 02:34:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kundan Kumar X-Patchwork-Id: 13703377 Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 997FA1E878 for ; Wed, 19 Jun 2024 04:00:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=203.254.224.33 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718769641; cv=none; b=E2/gHQnQ5nDkGRSdTG8E1G0IqO775SqDlpZgV43qsPFuFmJ7RLk2rxxtpgBbzwdpbTNL5O+BIHZjXKuEBHqxb/zW629hAUdotVwRZhe0hcq6aB6VTM9cVpSrnwjZUbfKVSInuhjH44rAmnUZhuObLXzEfulLvo4QadNF4k/jeEo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718769641; c=relaxed/simple; bh=cuqpbnNui64IOq3qaArA7pS/2/L4OZYRqBJLrwJKpnU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:MIME-Version: Content-Type:References; b=t1AhYrtq4R/qcsAMp1P7EZ4n6HroJ+rCR9ojdRHQiw2++G3NfSuUWU7ZoxpApjErE/WBVGHU746+H8JlveE+0cG1k9ufFnBYq9ljscma/e1xcxvY45yZQ525fpq5tnouhShNVOUOe1Wd23SEHvbw1AkyHcWnJG+WB1SzmGXkoN4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com; spf=pass smtp.mailfrom=samsung.com; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b=W5dtMHdL; arc=none smtp.client-ip=203.254.224.33 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="W5dtMHdL" Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20240619040036epoutp031e41be9713a5a9dd267baa5958732cd0~aS_DSzkwe3132731327epoutp03Z for ; Wed, 19 Jun 2024 04:00:36 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20240619040036epoutp031e41be9713a5a9dd267baa5958732cd0~aS_DSzkwe3132731327epoutp03Z DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1718769636; bh=bSc1O9CNMJ738L48usJhBMa72iwE2Sgm1XUeD+T+QPI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=W5dtMHdLq8wIzOF+JGfXYU0+TGMY7yh+l3Lsx4w8EgnDjAsIQqCRZlstTeao4NeY+ 5sVS1zTufx/GiGnG/oFxPgyqBL2LiMhZLHgbCECdlqamOC0AlCIAqCbBqYqRn5GjlB y7lQai10gudBNFZrVxOWLQ/r316qsxsOeSIszK3s= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20240619040034epcas5p20e2a8daae4959a1f2b9899ae7fe43e92~aS_A0n1e_3041630416epcas5p2E; Wed, 19 Jun 2024 04:00:34 +0000 (GMT) Received: from epsmgec5p1new.samsung.com (unknown [182.195.38.181]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4W3qf73h8Xz4x9Q7; Wed, 19 Jun 2024 04:00:31 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmgec5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 82.B0.08853.FD752766; Wed, 19 Jun 2024 13:00:31 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20240619024153epcas5p4fda751acf693081824c7f1f279988f65~aR5UVaIJr0311303113epcas5p4T; Wed, 19 Jun 2024 02:41:53 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20240619024153epsmtrp1dad79b3c9f58027e345ad55c0ff56ab7~aR5UUbgVf0144301443epsmtrp1K; Wed, 19 Jun 2024 02:41:53 +0000 (GMT) X-AuditID: b6c32a44-fc3fa70000002295-4e-667257df3de1 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 1A.2C.19057.17542766; Wed, 19 Jun 2024 11:41:53 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240619024151epsmtip24593a978d8352718cf8530901fa4ff66~aR5SopwZM0387003870epsmtip2v; Wed, 19 Jun 2024 02:41:51 +0000 (GMT) From: Kundan Kumar To: axboe@kernel.dk, hch@lst.de, willy@infradead.org, kbusch@kernel.org Cc: linux-block@vger.kernel.org, joshi.k@samsung.com, mcgrof@kernel.org, anuj20.g@samsung.com, nj.shetty@samsung.com, c.gameti@samsung.com, gost.dev@samsung.com, Kundan Kumar Subject: [PATCH v5 3/3] block: unpin user pages belonging to a folio Date: Wed, 19 Jun 2024 08:04:20 +0530 Message-Id: <20240619023420.34527-4-kundan.kumar@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20240619023420.34527-1-kundan.kumar@samsung.com> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrNJsWRmVeSWpSXmKPExsWy7bCmlu798KI0g7YVEhZNE/4yW6y+289m 8X17H4vFzQM7mSxWrj7KZHH0/1s2i0mHrjFabP3yldVi7y1tixsTnjJabPs9n9ni9485bA48 HptXaHlcPlvqsWlVJ5vH7psNbB59W1YxenzeJBfAFpVtk5GamJJapJCal5yfkpmXbqvkHRzv HG9qZmCoa2hpYa6kkJeYm2qr5OIToOuWmQN0oZJCWWJOKVAoILG4WEnfzqYov7QkVSEjv7jE Vim1ICWnwKRArzgxt7g0L10vL7XEytDAwMgUqDAhO6Px9ASWgoNyFd1PfzM2MHaJdTFyckgI mEjcurWKtYuRi0NIYDejRMuCbjYI5xOjxPRNM6Ay3xglTt96wQLT8mfdLKiqvYwSF25OhnI+ M0qsWTsHqIqDg01AV+JHUyhIg4iAu8TUl48YQWqYBc4ySpyY+ghskrCAi8SRfb+YQGwWAVWJ r/8fMYPYvAK2Eo/WzmKE2CYvMfPSd3aQmZwCdhI/r1hDlAhKnJz5BGwMM1BJ89bZzCDzJQSm ckg82DaXFaLXReJRXwM7hC0s8er4FihbSuJlfxuUnS1xqHEDE4RdIrHzCEy9vUTrqX5mkL3M ApoS63fpQ4RlJaaeWscEsZdPovf3E6hWXokd82BsNYk576ZCA0tGYuGlGVBxD4lv55dBQ3Qi o8SSxatYJzAqzELyzywk/8xCWL2AkXkVo2RqQXFuemqyaYFhXmo5PJaT83M3MYJTrZbLDsYb 8//pHWJk4mA8xCjBwawkwus0LS9NiDclsbIqtSg/vqg0J7X4EKMpMLwnMkuJJucDk31eSbyh iaWBiZmZmYmlsZmhkjjv69a5KUIC6YklqdmpqQWpRTB9TBycUg1M21nUvoZ+nfz0buy0zhkv dzrfrHl9fmatX29vR2md6PIfE9fN73BacaA3L/5TUOXnX0pCPYGOwp0Hlt76U7vCWiFp/cUZ zta1vyuTbSVeX3zAePb747tPnvzQdGOsO5z5XtgwWY51eoSdYuzvbelpSXeVq75pf3VdOWHL 7/VnewPPlkimza6bKCOiEJ1u+OeJVOfGAyXyndMe7X78+ua+U0anpW6zuBVfmzJBPs45x+bw 9BsFpa58Sy6fbMz55ZVuLdkV2/9Vaf4vxb8hxfKbkjZf2xhk8ulCNHtr+DemDsftkrula3Y9 UmLae4ZbLIbdgk/75YdtH29ckfrnw60qsewX281r8uZak6Zt7FAMUmIpzkg01GIuKk4EADVc pVg+BAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrILMWRmVeSWpSXmKPExsWy7bCSvG6ha1GawcPpohZNE/4yW6y+289m 8X17H4vFzQM7mSxWrj7KZHH0/1s2i0mHrjFabP3yldVi7y1tixsTnjJabPs9n9ni9485bA48 HptXaHlcPlvqsWlVJ5vH7psNbB59W1YxenzeJBfAFsVlk5Kak1mWWqRvl8CV0Xh6AkvBQbmK 7qe/GRsYu8S6GDk5JARMJP6sm8XWxcjFISSwm1Gip/UxO0RCRmL33Z2sELawxMp/z9khij4y Spy9187SxcjBwSagK/GjKRSkRkTAV2LBhueMIDXMAtcZJW5M38oMkhAWcJE4su8XE4jNIqAq 8fX/I7A4r4CtxKO1sxghFshLzLz0nR1kJqeAncTPK9YgYSGgkr4VH1ggygUlTs58AmYzA5U3 b53NPIFRYBaS1CwkqQWMTKsYJVMLinPTc4sNC4zyUsv1ihNzi0vz0vWS83M3MYJjQUtrB+Oe VR/0DjEycTAeYpTgYFYS4XWalpcmxJuSWFmVWpQfX1Sak1p8iFGag0VJnPfb694UIYH0xJLU 7NTUgtQimCwTB6dUA5P/8pRAuyc13EWx/6YEv5qkn7E7XbtE/8V2DpEDMvrn53/xWZG55M3B 2iKnP1dtzq91/Tbnx37GyxoOIutshbet3iV4Zv43E8Hjv0yCrmftyss2yFK7+qXn1Bsv+2ev +tpmi1yaMlO+7/STy6xu4tUS14//Xf5qz8eSqcn3WX82u+WrvpzvoSjnV3piwpyr5meXz5Q4 aPUjYfW+i3/bEtZE8779W/Bztr5YYfx2/3W3n2wx72j2E3MIXFSZyn5RdtlMrfNf0tUM10kW XlM87qB7fu+WinaL1LNMf3prfvfIu146xLrXeWePsqqr9KZkfql9RZfXtbXciVzy1dJd+8uq pNBdf5IK1546/36ebeRFJZbijERDLeai4kQATWwmBPQCAAA= X-CMS-MailID: 20240619024153epcas5p4fda751acf693081824c7f1f279988f65 X-Msg-Generator: CA X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20240619024153epcas5p4fda751acf693081824c7f1f279988f65 References: <20240619023420.34527-1-kundan.kumar@samsung.com> Unpin pages which belong to same folio. This enables us to release folios on I/O completion rather than looping through pages. Introduce a function bio_release_folio() helps put refs by npages count. Suggested-by: Keith Busch Signed-off-by: Kundan Kumar --- block/bio.c | 13 ++++--------- block/blk.h | 7 +++++++ include/linux/mm.h | 1 + mm/gup.c | 13 +++++++++++++ 4 files changed, 25 insertions(+), 9 deletions(-) -- 2.25.1 diff --git a/block/bio.c b/block/bio.c index 3e75b5b0eb6e..68f6de0b0a08 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1186,20 +1186,12 @@ void __bio_release_pages(struct bio *bio, bool mark_dirty) struct folio_iter fi; bio_for_each_folio_all(fi, bio) { - struct page *page; - size_t nr_pages; - if (mark_dirty) { folio_lock(fi.folio); folio_mark_dirty(fi.folio); folio_unlock(fi.folio); } - page = folio_page(fi.folio, fi.offset / PAGE_SIZE); - nr_pages = (fi.offset + fi.length - 1) / PAGE_SIZE - - fi.offset / PAGE_SIZE + 1; - do { - bio_release_page(bio, page++); - } while (--nr_pages != 0); + bio_release_folio(bio, fi.folio, 1); } } EXPORT_SYMBOL_GPL(__bio_release_pages); @@ -1372,6 +1364,9 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter) } else bio_iov_add_folio(bio, folio, len, folio_offset); + if (num_pages > 1) + bio_release_folio(bio, folio, num_pages - 1); + /* Skip the pages which got added */ i = i + (num_pages - 1); diff --git a/block/blk.h b/block/blk.h index d0bec44a2ffb..a657282c0e4a 100644 --- a/block/blk.h +++ b/block/blk.h @@ -538,6 +538,13 @@ static inline void bio_release_page(struct bio *bio, struct page *page) unpin_user_page(page); } +static inline void bio_release_folio(struct bio *bio, struct folio *folio, + unsigned long npages) +{ + if (bio_flagged(bio, BIO_PAGE_PINNED)) + unpin_user_folio(folio, npages); +} + struct request_queue *blk_alloc_queue(struct queue_limits *lim, int node_id); int disk_scan_partitions(struct gendisk *disk, blk_mode_t mode); diff --git a/include/linux/mm.h b/include/linux/mm.h index 9849dfda44d4..b902c6c39e2b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1618,6 +1618,7 @@ void unpin_user_pages_dirty_lock(struct page **pages, unsigned long npages, void unpin_user_page_range_dirty_lock(struct page *page, unsigned long npages, bool make_dirty); void unpin_user_pages(struct page **pages, unsigned long npages); +void unpin_user_folio(struct folio *folio, unsigned long npages); static inline bool is_cow_mapping(vm_flags_t flags) { diff --git a/mm/gup.c b/mm/gup.c index ca0f5cedce9b..bc96efa43d1b 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -488,6 +488,19 @@ void unpin_user_pages(struct page **pages, unsigned long npages) } EXPORT_SYMBOL(unpin_user_pages); +/** + * unpin_user_folio() - release pages of a folio + * @folio: pointer to folio to be released + * @npages: number of pages of same folio + * + * Release npages of the folio + */ +void unpin_user_folio(struct folio *folio, unsigned long npages) +{ + gup_put_folio(folio, npages, FOLL_PIN); +} +EXPORT_SYMBOL(unpin_user_folio); + /* * Set the MMF_HAS_PINNED if not set yet; after set it'll be there for the mm's * lifecycle. Avoid setting the bit unless necessary, or it might cause write