From patchwork Wed Dec 6 10:02:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 13481350 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 975B52C868 for ; Wed, 6 Dec 2023 10:10:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="EdTbnQbj" Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20231206101041epoutp028feaa8cc17a635c567afa20b1dd047f5~eNlOZ49On1975119751epoutp02O for ; Wed, 6 Dec 2023 10:10:41 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20231206101041epoutp028feaa8cc17a635c567afa20b1dd047f5~eNlOZ49On1975119751epoutp02O DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857441; bh=SF1N2aEMUVTCNqgb0+5ohRAM5D566oKMRQvrVXH9YeE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EdTbnQbjcQdOYffSjEZPHeZm4xc/hJ/3K50mgK11YMceFuR5Wh61jJLyKUbMH7s7y PXJKZ3jsNi/gv5HtdIPXGy4fSOWcaOQIIVcE1+Ya/l8gp4JlpwATb8ij11xxBLxion p41WgW0LKBtl+OGGuU3uuKjyWmNRR+ExO62NfxLM= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20231206101041epcas5p3237c541b3548a36a5cbe4510ba0336e7~eNlNywgn70506805068epcas5p3V; Wed, 6 Dec 2023 10:10:41 +0000 (GMT) Received: from epsmges5p1new.samsung.com (unknown [182.195.38.183]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4SlY7g1vy7z4x9Pp; Wed, 6 Dec 2023 10:10:39 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 6D.03.09634.F9840756; Wed, 6 Dec 2023 19:10:39 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20231206101038epcas5p49984074d09421f0f60e231e33db8b9e7~eNlK5-s1R0156201562epcas5p4x; Wed, 6 Dec 2023 10:10:38 +0000 (GMT) Received: from epsmgmc1p1new.samsung.com (unknown [182.195.42.40]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20231206101038epsmtrp117841ac668b2752ba50e9a8c2f2d58a1~eNlK32slC1053310533epsmtrp1j; Wed, 6 Dec 2023 10:10:38 +0000 (GMT) X-AuditID: b6c32a49-eebff700000025a2-7f-6570489fb8eb Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgmc1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 43.B5.07368.D9840756; Wed, 6 Dec 2023 19:10:37 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101030epsmtip2d34b4a231c1c746d0b685aa105f7840b~eNlDvozXf0087000870epsmtip2A; Wed, 6 Dec 2023 10:10:29 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Kanchan Joshi , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 01/12] block: Introduce queue limits and sysfs for copy-offload support Date: Wed, 6 Dec 2023 15:32:33 +0530 Message-Id: <20231206100253.13100-2-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02Te1BUdRTH+917ubvrtHZZUH+Bk7hTGTA8Nhf64bih+eg2xMAMjjFFgxvc Adpn+/A5DAQDrCTPbSJWkWdhmDqCAcsrZWddhHAFRJBXEbuFESEypDysdlkq//ucc77ndx6/ OWycp2d5sVPkGkYlF0v55Aai0eTrF1BOK5lgnY6FrnTfxNH84gqBMgqf4ujieAGJZkyPALJd zwGooqqMQPevGzHUVlWMoW8umjFk/nuWRMWd9wCyDxow1D7ijyqzawjU1n6LQAMt50hU/rWd hT4baiZRreUvDA0X2gEq0g1iqNn2KUCNK+U4ujwzR6CuEW9kfWpx2+NNGw3jLNo6cZWgB3q1 dH3daZJuqEmjpxtKAd16P52kq/P1bnRe5h8kPW8fIei5jkGSzr9WB+iGnlP0Qv1LdL1tFot+ 4X3J7mRGnMiofBh5giIxRZ4k4kfExO+LDwkNFgQIwtAbfB+5WMaI+PvfjQ44mCJ17IXvc1Qs 1Tpc0WK1mh/05m6VQqthfJIVao2IzygTpUqhMlAtlqm18qRAOaPZJQgOfj3EITwiSR4/3cpS NoqOL2U8IdPB2M5cwGZDSgiH5jS5gMPmUa0AFvwuyQUbHPwIQJu5h+Uy/gSw5Ic5wqlyJvxo 1eOuQDuAlvJV0mUsADhtH8Ccz5KUL7yj1zoTPKlvcWi8KnBqcMqEQ3Plz6Qz4EF9CLPPNa4x Qb0C707VsZzMpRCc763CXdW2wdL+x2t+DhUG9b+6/FzKHd4qta11hDs0md+dXesIUt9zYJ7+ C+BK3g+N/TXrbXvA3yzXWC72gg8Kstc5AfaX3sZcrIFTbTfWORxmdRfgzmFwxzBXWoJctTbC vBUb5lodF+qyeS71djhRbHdz8RY4+WXNOtOwYKBzfVlnAJwbbcYKwTbDMyMYnhnB8H+1CoDX gRcZpVqWxKhDlAI5c+y/f01QyOrB2nX4vdMMxn96GNgJMDboBJCN8z25UquC4XETxSdOMipF vEorZdSdIMSx4yLca1OCwnFeck28QBgWLAwNDRWG7QwV8LdwZ7LKEnlUkljDSBhGyaj+zcPY HK907LW+orHeuJXmw8JXs4YelOxws1Z4d4vCc3IiYz+YrPAbW+44kHUn+0b0ZFnTiX2SJEGk tqWIdWjUc6vx/EfPr+q6q83LmyzMTDqx616q91HsOTxymO5ip70cNu7tMRPFR+r3aocljC5q 1m0+6omVuFTdtNc69nEL3zf/bFxQZqp6yWSKzdzYcfKxwEOVaql7u0I2GuM1UpZ5MKJ/+82O rgu1MTzEOR70ltHcdNf9lwsTl5L9oS0vdR67/JXpkP/yjp62M2kP961OuZ/KWAyk+rrMm6Ol R6Z5ss/1hgUY28E639NnDi8ZKN17TNS7ddGwZ+qwsZ6zFMfaHF4ZcfuTtnw+oU4WC/xwlVr8 D34F2iemBAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrJIsWRmVeSWpSXmKPExsWy7bCSvO5cj4JUg1VXLC3WnzrGbPHx628W i6YJf5ktVt/tZ7N4ffgTo8WTA+2MFgsWzWWxuHlgJ5PFnkWTmCxWrj7KZHH0/1s2i0mHrjFa PL06i8li7y1ti4VtS1gs9uw9yWJxedccNov5y56yW3Rf38Fmsfz4PyaLGxOeMlpM7LjKZLHj SSOjxbbf85kt1r1+z2Jx4pa0xfm/x1kdpD12zrrL7nH+3kYWj8tnSz02repk89i8pN7jxeaZ jB67bzaweSzum8zq0dv8js3j49NbLB7v911l8+jbsorRY/Ppao/Pm+Q8Nj15yxTAH8Vlk5Ka k1mWWqRvl8CVcbdzN3vBNtuKn00/2BoY7xh3MXJySAiYSNw/P5m5i5GLQ0hgN6PE/pMNrBAJ cYnmaz/YIWxhiZX/nrNDFH1klHjet4qti5GDg01AU+LC5FKQuIjADmaJn2ubmUAcZoHLzBLT Fh5hBOkWFoiWOP39ONhUFgFViSuPV4FN5RWwkPh4dhEzxAZ5iZmXvoPFOQUsJSY/h4gLAdXs a5zOAlEvKHFy5hMwmxmovnnrbOYJjAKzkKRmIUktYGRaxSiZWlCcm56bbFhgmJdarlecmFtc mpeul5yfu4kRHPtaGjsY783/p3eIkYmD8RCjBAezkghvzvn8VCHelMTKqtSi/Pii0pzU4kOM 0hwsSuK8hjNmpwgJpCeWpGanphakFsFkmTg4pRqY3s5iMhSVEWkPWts1p5rp5VOPZcGTXHM9 gx5vvrjbzH/FouUK+eY3s/yYuz+cvD75xG79b/qRDu8/Jvf37PMXcbp0a1PhxO8vr/jL5MZt 8haTzHB5eLHHyorj7NE7U5kyPiWZzdLrZWU6sVKY2XJbzWn32u4+c4HLrU3LWCqPi7OGLf+b svA2174uq4wHEvqzl2spn/035cdNU8W6usV3pSuFQ2MVgr0b3FY5Vc86wyDVueuOeqrS9xSP +WuYLrYc++TzJ8U2eJ2N8/5Tz1Zyau6c+UfIunKD+WLDpYd+luzk3am+LdWxZ6uJ7VWTqVXf axcYVzjqqVb4bORafvzPPK5rk5jO3p+0OG9978OdSizFGYmGWsxFxYkA66GVEmwDAAA= X-CMS-MailID: 20231206101038epcas5p49984074d09421f0f60e231e33db8b9e7 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101038epcas5p49984074d09421f0f60e231e33db8b9e7 References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty Add device limits as sysfs entries, - copy_max_bytes (RW) - copy_max_hw_bytes (RO) Above limits help to split the copy payload in block layer. copy_max_bytes: maximum total length of copy in single payload. copy_max_hw_bytes: Reflects the device supported maximum limit. Reviewed-by: Hannes Reinecke Reviewed-by: Luis Chamberlain Signed-off-by: Nitesh Shetty Signed-off-by: Kanchan Joshi Signed-off-by: Anuj Gupta --- Documentation/ABI/stable/sysfs-block | 23 ++++++++++++++++++ block/blk-settings.c | 24 +++++++++++++++++++ block/blk-sysfs.c | 36 ++++++++++++++++++++++++++++ include/linux/blkdev.h | 13 ++++++++++ 4 files changed, 96 insertions(+) diff --git a/Documentation/ABI/stable/sysfs-block b/Documentation/ABI/stable/sysfs-block index 1fe9a553c37b..96ba701e57da 100644 --- a/Documentation/ABI/stable/sysfs-block +++ b/Documentation/ABI/stable/sysfs-block @@ -155,6 +155,29 @@ Description: last zone of the device which may be smaller. +What: /sys/block//queue/copy_max_bytes +Date: August 2023 +Contact: linux-block@vger.kernel.org +Description: + [RW] This is the maximum number of bytes that the block layer + will allow for a copy request. This is always smaller or + equal to the maximum size allowed by the hardware, indicated by + 'copy_max_hw_bytes'. An attempt to set a value higher than + 'copy_max_hw_bytes' will truncate this to 'copy_max_hw_bytes'. + Writing '0' to this file will disable offloading copies for this + device, instead copy is done via emulation. + + +What: /sys/block//queue/copy_max_hw_bytes +Date: August 2023 +Contact: linux-block@vger.kernel.org +Description: + [RO] This is the maximum number of bytes that the hardware + will allow for single data copy request. + A value of 0 means that the device does not support + copy offload. + + What: /sys/block//queue/crypto/ Date: February 2022 Contact: linux-block@vger.kernel.org diff --git a/block/blk-settings.c b/block/blk-settings.c index 0046b447268f..4441711ac364 100644 --- a/block/blk-settings.c +++ b/block/blk-settings.c @@ -59,6 +59,8 @@ void blk_set_default_limits(struct queue_limits *lim) lim->zoned = BLK_ZONED_NONE; lim->zone_write_granularity = 0; lim->dma_alignment = 511; + lim->max_copy_hw_sectors = 0; + lim->max_copy_sectors = 0; } /** @@ -82,6 +84,8 @@ void blk_set_stacking_limits(struct queue_limits *lim) lim->max_dev_sectors = UINT_MAX; lim->max_write_zeroes_sectors = UINT_MAX; lim->max_zone_append_sectors = UINT_MAX; + lim->max_copy_hw_sectors = UINT_MAX; + lim->max_copy_sectors = UINT_MAX; } EXPORT_SYMBOL(blk_set_stacking_limits); @@ -183,6 +187,22 @@ void blk_queue_max_discard_sectors(struct request_queue *q, } EXPORT_SYMBOL(blk_queue_max_discard_sectors); +/* + * blk_queue_max_copy_hw_sectors - set max sectors for a single copy payload + * @q: the request queue for the device + * @max_copy_sectors: maximum number of sectors to copy + */ +void blk_queue_max_copy_hw_sectors(struct request_queue *q, + unsigned int max_copy_sectors) +{ + if (max_copy_sectors > (BLK_COPY_MAX_BYTES >> SECTOR_SHIFT)) + max_copy_sectors = BLK_COPY_MAX_BYTES >> SECTOR_SHIFT; + + q->limits.max_copy_hw_sectors = max_copy_sectors; + q->limits.max_copy_sectors = max_copy_sectors; +} +EXPORT_SYMBOL_GPL(blk_queue_max_copy_hw_sectors); + /** * blk_queue_max_secure_erase_sectors - set max sectors for a secure erase * @q: the request queue for the device @@ -578,6 +598,10 @@ int blk_stack_limits(struct queue_limits *t, struct queue_limits *b, t->max_segment_size = min_not_zero(t->max_segment_size, b->max_segment_size); + t->max_copy_sectors = min(t->max_copy_sectors, b->max_copy_sectors); + t->max_copy_hw_sectors = min(t->max_copy_hw_sectors, + b->max_copy_hw_sectors); + t->misaligned |= b->misaligned; alignment = queue_limit_alignment_offset(b, start); diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c index 0b2d04766324..7193a1f015f9 100644 --- a/block/blk-sysfs.c +++ b/block/blk-sysfs.c @@ -199,6 +199,37 @@ static ssize_t queue_discard_zeroes_data_show(struct request_queue *q, char *pag return queue_var_show(0, page); } +static ssize_t queue_copy_hw_max_show(struct request_queue *q, char *page) +{ + return sprintf(page, "%llu\n", (unsigned long long) + q->limits.max_copy_hw_sectors << SECTOR_SHIFT); +} + +static ssize_t queue_copy_max_show(struct request_queue *q, char *page) +{ + return sprintf(page, "%llu\n", (unsigned long long) + q->limits.max_copy_sectors << SECTOR_SHIFT); +} + +static ssize_t queue_copy_max_store(struct request_queue *q, const char *page, + size_t count) +{ + unsigned long max_copy; + ssize_t ret = queue_var_store(&max_copy, page, count); + + if (ret < 0) + return ret; + + if (max_copy & (queue_logical_block_size(q) - 1)) + return -EINVAL; + + max_copy >>= SECTOR_SHIFT; + q->limits.max_copy_sectors = min_t(unsigned int, max_copy, + q->limits.max_copy_hw_sectors); + + return count; +} + static ssize_t queue_write_same_max_show(struct request_queue *q, char *page) { return queue_var_show(0, page); @@ -517,6 +548,9 @@ QUEUE_RO_ENTRY(queue_nr_zones, "nr_zones"); QUEUE_RO_ENTRY(queue_max_open_zones, "max_open_zones"); QUEUE_RO_ENTRY(queue_max_active_zones, "max_active_zones"); +QUEUE_RO_ENTRY(queue_copy_hw_max, "copy_max_hw_bytes"); +QUEUE_RW_ENTRY(queue_copy_max, "copy_max_bytes"); + QUEUE_RW_ENTRY(queue_nomerges, "nomerges"); QUEUE_RW_ENTRY(queue_rq_affinity, "rq_affinity"); QUEUE_RW_ENTRY(queue_poll, "io_poll"); @@ -634,6 +668,8 @@ static struct attribute *queue_attrs[] = { &queue_discard_max_entry.attr, &queue_discard_max_hw_entry.attr, &queue_discard_zeroes_data_entry.attr, + &queue_copy_hw_max_entry.attr, + &queue_copy_max_entry.attr, &queue_write_same_max_entry.attr, &queue_write_zeroes_max_entry.attr, &queue_zone_append_max_entry.attr, diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 3f8a21cd9233..4e17945041c8 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -311,6 +311,9 @@ struct queue_limits { unsigned int discard_alignment; unsigned int zone_write_granularity; + unsigned int max_copy_hw_sectors; + unsigned int max_copy_sectors; + unsigned short max_segments; unsigned short max_integrity_segments; unsigned short max_discard_segments; @@ -893,6 +896,8 @@ extern void blk_queue_chunk_sectors(struct request_queue *, unsigned int); extern void blk_queue_max_segments(struct request_queue *, unsigned short); extern void blk_queue_max_discard_segments(struct request_queue *, unsigned short); +extern void blk_queue_max_copy_hw_sectors(struct request_queue *q, + unsigned int max_copy_sectors); void blk_queue_max_secure_erase_sectors(struct request_queue *q, unsigned int max_sectors); extern void blk_queue_max_segment_size(struct request_queue *, unsigned int); @@ -1211,6 +1216,14 @@ static inline unsigned int bdev_discard_granularity(struct block_device *bdev) return bdev_get_queue(bdev)->limits.discard_granularity; } +/* maximum copy offload length, this is set to 128MB based on current testing */ +#define BLK_COPY_MAX_BYTES (1 << 27) + +static inline unsigned int bdev_max_copy_sectors(struct block_device *bdev) +{ + return bdev_get_queue(bdev)->limits.max_copy_sectors; +} + static inline unsigned int bdev_max_secure_erase_sectors(struct block_device *bdev) { From patchwork Wed Dec 6 10:02:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 13481351 Received: from mailout1.samsung.com (mailout1.samsung.com [203.254.224.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCEC328E0E for ; Wed, 6 Dec 2023 10:10:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="OoT/jEVE" Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20231206101053epoutp014145c5a1e8245e4ab6b0255107a4e178~eNlZGAHm80126701267epoutp01Y for ; Wed, 6 Dec 2023 10:10:53 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20231206101053epoutp014145c5a1e8245e4ab6b0255107a4e178~eNlZGAHm80126701267epoutp01Y DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857453; bh=Ht/Do8cTuJq3u2yJt3wmG4hHCRAbWvK0/GxdjTa0bnE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OoT/jEVES07IzsqBprqRYW4rnbTfG+Rec80k5makyHcuCWU/kNE+Lswx5CZ5W6S8p iEg3LitIwj6qXGbDl+RHyzzy8sXDvRsQjUS9KAj/62faUujzAeziGrbP/GKHsUyJH6 /RIMzV1VgjMjJHEcyV1ubELKTxsdgj7+yoepFCUE= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20231206101052epcas5p1f74433f38c2c0a95c3448556d538f621~eNlYlyY1U0449604496epcas5p1D; Wed, 6 Dec 2023 10:10:52 +0000 (GMT) Received: from epsmgec5p1-new.samsung.com (unknown [182.195.38.179]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4SlY7v0xjCz4x9Px; Wed, 6 Dec 2023 10:10:51 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmgec5p1-new.samsung.com (Symantec Messaging Gateway) with SMTP id 2D.58.19369.AA840756; Wed, 6 Dec 2023 19:10:50 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20231206101050epcas5p2c8233030bbf74cef0166c7dfc0f41be7~eNlWbwK8W3161231612epcas5p2p; Wed, 6 Dec 2023 10:10:50 +0000 (GMT) Received: from epsmgms1p1new.samsung.com (unknown [182.195.42.41]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20231206101050epsmtrp11528effcadae6126b68eea06fd58e0ac~eNlWagLGu1050310503epsmtrp1J; Wed, 6 Dec 2023 10:10:50 +0000 (GMT) X-AuditID: b6c32a50-c99ff70000004ba9-26-657048aacece Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p1new.samsung.com (Symantec Messaging Gateway) with SMTP id B8.7B.08755.AA840756; Wed, 6 Dec 2023 19:10:50 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101046epsmtip29bbd9da58b30c5b273ec0cbf08b87c0b~eNlSljNQa1163811638epsmtip2B; Wed, 6 Dec 2023 10:10:46 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 02/12] Add infrastructure for copy offload in block and request layer. Date: Wed, 6 Dec 2023 15:32:34 +0530 Message-Id: <20231206100253.13100-3-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02Te1BUVRzH59x7ubsQ0G0xPG4ksEGGBu7Gw4PxDEbvRM3g1PQgGNxZ7iwb y+6yDytjRmJzA+TlMhm7xCwaSkCJvB+CQ7shaSUoIoIjjbooBOSAihIi7bJb+d/n9zvf3/md 7+/Mj41zdCwuWyJTM0qZUMoj3YgOS9CW4AZawfCXjuxETefP4mjhwQqB8stXcdR4vYxEs5ZF gKz9XwFUc6yaQOP93RiqbxzAkN58BaCpUSOG+ia2oaO6WgL19p0j0EjPtyQynZhioUNjXSSq G3yCoavlUwAdLhjFUJf1C4A6Vkw4Ojl7l0C/TLyAhlYHXeI20d3G6yx6aLKZoEd+19AtDYUk 3Vp7gJ5uNQD69HgeSX9XWuFCl2j/IumFqQmCvntmlKRL2xoAfa9lM91inceSPVOyojIZYQaj 9GNkInmGRCaO5iW9k56QHh7BFwQLItEOnp9MmM1E8xLfSg7eJZHaJsHz2yeUamypZKFKxdse E6WUa9SMX6ZcpY7mMYoMqSJMEaISZqs0MnGIjFHvFPD5r4XbhHuzMn9urWIpmqI+XVi7gOWB ZX4RcGVDKgyWVlZjduZQvQBqa1OKgJuNFwH82zRCOIIlADtOzNgC9nrF7MABR74PwMmvzzhF 9wDU/9ZG2kUkFQSHKzT2WzdQP+Cwu1lg1+DUYwzq75hd7AdeVCr89VAry84EFQh/HKxfZw8K QeOVdtLxPF9ouPRwPe9KRcKKO8dwh+Y5eM5gJeyM2zTa9irc3gBSja5wrqsYdxQnQkttHcvB XvDPwTYnc+FMmc7JInjJcAFzsBre6v3JybHw4Pky3G4Gt5lp6tnu6OUJS1asmGMQHrBAx3Go /eGkfsrFwRvhjcpaJ9OwpvAb53yKAXxgLQblwNf4lAXjUxaM/3erAXgD4DIKVbaYEYUrBMEy 5pP/PlYkz24B6wuxNbkLNJ5aDTEDjA3MALJx3gYP6ZCc4XhkCD/bzyjl6UqNlFGZQbhtyIdx 7vMiuW2jZOp0QVgkPywiIiIsMjRCwNvoMXuwOoNDiYVqJothFIzy3zqM7crNw+L3sjgvJ/3R 7aZ/RuSXcnJpaTihxV807XVR2jZjmE+t21OQX2JejX+zWrrWzV3stbzvHeBTyNwam/g8f9l9 AeceFy27C1DU7eKj2Tr/7989ez/OLY3X81D70fzwq6H6t3dcCw36YKDoSXxOJdYem7w6/ciU 2wbTJK8wJo0k16fZNw16zpkSxQEmoWugf+IjdYK65n7h5qX63hwdy+z+xuVpZtP+L8cvx5zy h7vXHse+tIttCfw4r31hUTsU0/nhs3FjRZKh/NSkG4bjaKIz8aI2P1ff7G1pfy+2R3xk99w+ Ive2+5bXq19MuNrp7XOz6vS2hj09AdjAzfQc32v9fQFRfTxClSkUbMWVKuE/TyWjmJkEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrFIsWRmVeSWpSXmKPExsWy7bCSvO4qj4JUg8uXRCzWnzrGbPHx628W i6YJf5ktVt/tZ7N4ffgTo8WTA+2MFgsWzWWxuHlgJ5PFytVHmSwmHbrGaPH06iwmi723tC0W ti1hsdiz9ySLxeVdc9gs5i97ym7RfX0Hm8Xy4/+YLG5MeMpoMbHjKpPFjieNjBbbfs9ntlj3 +j2LxYlb0hbn/x5ndZD02DnrLrvH+XsbWTwuny312LSqk81j85J6jxebZzJ67L7ZwOaxuG8y q0dv8zs2j49Pb7F4vN93lc2jb8sqRo/Pm+Q8Nj15yxTAF8Vlk5Kak1mWWqRvl8CVcWTzbPaC 9TYVH/+fY2pg/GnQxcjBISFgIvH6aH0XIxeHkMBuRokrD7azdTFyAsXFJZqv/WCHsIUlVv57 zg5R9JFRYuWSZnaQZjYBTYkLk0tB4iICO5glfq5tZgJxmAW6mCUm/f8L1i0sECXxoXUlI4jN IqAqsfb4SrA4r4CFxKxrW6G2yUvMvPQdLM4pYCkx+fkiZhBbCKhmX+N0Foh6QYmTM5+A2cxA 9c1bZzNPYBSYhSQ1C0lqASPTKkbJ1ILi3PTcYsMCw7zUcr3ixNzi0rx0veT83E2M4OjW0tzB uH3VB71DjEwcjIcYJTiYlUR4c87npwrxpiRWVqUW5ccXleakFh9ilOZgURLnFX/RmyIkkJ5Y kpqdmlqQWgSTZeLglGpgWnBb5h7rruOn1N+e+FfScvDaq1SrOqG8A1OlPwbtS10/l7FEgmOK xROnGZ8EJdPP3eOedtuLV+CpcumDOZ0ze+YmZig+kmHLT3ibJrsy4myZ5/Ndrod1Hp+Pbpz0 5/EToyv9G/oEbsX0i1YXhZ3T//ny463jX3Lnbnq3ZVVf3vHlZ39qeYh++jaJ7WLz7Lse+osN zBUDVsfI8pfYPRdrTzZgcExqSonRZGp8nu/btL6ib//RsBAZ8ZDJEXP2p01aemEWTyGTopUd 11aGKY2bLhi8YGZV2vpTjTP0Zlrm04/TZ3PNPmDRv39rRMCrlNl5YUuX7PaPm//HmJUr+dCv TuuTXBarzmw+G8Odv2/rRCWW4oxEQy3mouJEAHH4sytdAwAA X-CMS-MailID: 20231206101050epcas5p2c8233030bbf74cef0166c7dfc0f41be7 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101050epcas5p2c8233030bbf74cef0166c7dfc0f41be7 References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty We add two new opcode REQ_OP_COPY_SRC, REQ_OP_COPY_DST. Since copy is a composite operation involving src and dst sectors/lba, each needs to be represented by a separate bio to make it compatible with device mapper. We expect caller to take a plug and send bio with source information, followed by bio with destination information. Once the src bio arrives we form a request and wait for destination bio. Upon arrival of destination we merge these two bio's and send corresponding request down to device driver. Merging non copy offload bio is avoided by checking for copy specific opcodes in merge function. Signed-off-by: Nitesh Shetty Signed-off-by: Anuj Gupta --- block/blk-core.c | 7 +++++++ block/blk-merge.c | 41 +++++++++++++++++++++++++++++++++++++++ block/blk.h | 16 +++++++++++++++ block/elevator.h | 1 + include/linux/bio.h | 6 +----- include/linux/blk_types.h | 10 ++++++++++ 6 files changed, 76 insertions(+), 5 deletions(-) diff --git a/block/blk-core.c b/block/blk-core.c index 2eca76ccf4ee..51c6cc3022f4 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -121,6 +121,8 @@ static const char *const blk_op_name[] = { REQ_OP_NAME(ZONE_FINISH), REQ_OP_NAME(ZONE_APPEND), REQ_OP_NAME(WRITE_ZEROES), + REQ_OP_NAME(COPY_SRC), + REQ_OP_NAME(COPY_DST), REQ_OP_NAME(DRV_IN), REQ_OP_NAME(DRV_OUT), }; @@ -800,6 +802,11 @@ void submit_bio_noacct(struct bio *bio) if (!q->limits.max_write_zeroes_sectors) goto not_supported; break; + case REQ_OP_COPY_SRC: + case REQ_OP_COPY_DST: + if (!q->limits.max_copy_sectors) + goto not_supported; + break; default: break; } diff --git a/block/blk-merge.c b/block/blk-merge.c index 65e75efa9bd3..bcb55ba48107 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -158,6 +158,20 @@ static struct bio *bio_split_write_zeroes(struct bio *bio, return bio_split(bio, lim->max_write_zeroes_sectors, GFP_NOIO, bs); } +static struct bio *bio_split_copy(struct bio *bio, + const struct queue_limits *lim, + unsigned int *nsegs) +{ + *nsegs = 1; + if (bio_sectors(bio) <= lim->max_copy_sectors) + return NULL; + /* + * We don't support splitting for a copy bio. End it with EIO if + * splitting is required and return an error pointer. + */ + return ERR_PTR(-EIO); +} + /* * Return the maximum number of sectors from the start of a bio that may be * submitted as a single request to a block device. If enough sectors remain, @@ -366,6 +380,12 @@ struct bio *__bio_split_to_limits(struct bio *bio, case REQ_OP_WRITE_ZEROES: split = bio_split_write_zeroes(bio, lim, nr_segs, bs); break; + case REQ_OP_COPY_SRC: + case REQ_OP_COPY_DST: + split = bio_split_copy(bio, lim, nr_segs); + if (IS_ERR(split)) + return NULL; + break; default: split = bio_split_rw(bio, lim, nr_segs, bs, get_max_io_size(bio, lim) << SECTOR_SHIFT); @@ -922,6 +942,9 @@ bool blk_rq_merge_ok(struct request *rq, struct bio *bio) if (!rq_mergeable(rq) || !bio_mergeable(bio)) return false; + if (blk_copy_offload_mergable(rq, bio)) + return true; + if (req_op(rq) != bio_op(bio)) return false; @@ -951,6 +974,8 @@ enum elv_merge blk_try_merge(struct request *rq, struct bio *bio) { if (blk_discard_mergable(rq)) return ELEVATOR_DISCARD_MERGE; + else if (blk_copy_offload_mergable(rq, bio)) + return ELEVATOR_COPY_OFFLOAD_MERGE; else if (blk_rq_pos(rq) + blk_rq_sectors(rq) == bio->bi_iter.bi_sector) return ELEVATOR_BACK_MERGE; else if (blk_rq_pos(rq) - bio_sectors(bio) == bio->bi_iter.bi_sector) @@ -1053,6 +1078,20 @@ static enum bio_merge_status bio_attempt_discard_merge(struct request_queue *q, return BIO_MERGE_FAILED; } +static enum bio_merge_status bio_attempt_copy_offload_merge(struct request *req, + struct bio *bio) +{ + if (req->__data_len != bio->bi_iter.bi_size) + return BIO_MERGE_FAILED; + + req->biotail->bi_next = bio; + req->biotail = bio; + req->nr_phys_segments++; + req->__data_len += bio->bi_iter.bi_size; + + return BIO_MERGE_OK; +} + static enum bio_merge_status blk_attempt_bio_merge(struct request_queue *q, struct request *rq, struct bio *bio, @@ -1073,6 +1112,8 @@ static enum bio_merge_status blk_attempt_bio_merge(struct request_queue *q, break; case ELEVATOR_DISCARD_MERGE: return bio_attempt_discard_merge(q, rq, bio); + case ELEVATOR_COPY_OFFLOAD_MERGE: + return bio_attempt_copy_offload_merge(rq, bio); default: return BIO_MERGE_NONE; } diff --git a/block/blk.h b/block/blk.h index 08a358bc0919..b0c17ad635a5 100644 --- a/block/blk.h +++ b/block/blk.h @@ -159,6 +159,20 @@ static inline bool blk_discard_mergable(struct request *req) return false; } +/* + * Copy offload sends a pair of bio with REQ_OP_COPY_SRC and REQ_OP_COPY_DST + * operation by taking a plug. + * Initially SRC bio is sent which forms a request and + * waits for DST bio to arrive. Once DST bio arrives + * we merge it and send request down to driver. + */ +static inline bool blk_copy_offload_mergable(struct request *req, + struct bio *bio) +{ + return (req_op(req) == REQ_OP_COPY_SRC && + bio_op(bio) == REQ_OP_COPY_DST); +} + static inline unsigned int blk_rq_get_max_segments(struct request *rq) { if (req_op(rq) == REQ_OP_DISCARD) @@ -300,6 +314,8 @@ static inline bool bio_may_exceed_limits(struct bio *bio, case REQ_OP_DISCARD: case REQ_OP_SECURE_ERASE: case REQ_OP_WRITE_ZEROES: + case REQ_OP_COPY_SRC: + case REQ_OP_COPY_DST: return true; /* non-trivial splitting decisions */ default: break; diff --git a/block/elevator.h b/block/elevator.h index 7ca3d7b6ed82..eec442bbf384 100644 --- a/block/elevator.h +++ b/block/elevator.h @@ -18,6 +18,7 @@ enum elv_merge { ELEVATOR_FRONT_MERGE = 1, ELEVATOR_BACK_MERGE = 2, ELEVATOR_DISCARD_MERGE = 3, + ELEVATOR_COPY_OFFLOAD_MERGE = 4, }; struct blk_mq_alloc_data; diff --git a/include/linux/bio.h b/include/linux/bio.h index ec4db73e5f4e..5816e41588af 100644 --- a/include/linux/bio.h +++ b/include/linux/bio.h @@ -53,11 +53,7 @@ static inline unsigned int bio_max_segs(unsigned int nr_segs) */ static inline bool bio_has_data(struct bio *bio) { - if (bio && - bio->bi_iter.bi_size && - bio_op(bio) != REQ_OP_DISCARD && - bio_op(bio) != REQ_OP_SECURE_ERASE && - bio_op(bio) != REQ_OP_WRITE_ZEROES) + if (bio && (bio_op(bio) == REQ_OP_READ || bio_op(bio) == REQ_OP_WRITE)) return true; return false; diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h index 7c2316c91cbd..bd821eaa7a02 100644 --- a/include/linux/blk_types.h +++ b/include/linux/blk_types.h @@ -393,6 +393,10 @@ enum req_op { /* reset all the zone present on the device */ REQ_OP_ZONE_RESET_ALL = (__force blk_opf_t)17, + /* copy offload dst and src operation */ + REQ_OP_COPY_SRC = (__force blk_opf_t)19, + REQ_OP_COPY_DST = (__force blk_opf_t)21, + /* Driver private requests */ REQ_OP_DRV_IN = (__force blk_opf_t)34, REQ_OP_DRV_OUT = (__force blk_opf_t)35, @@ -481,6 +485,12 @@ static inline bool op_is_write(blk_opf_t op) return !!(op & (__force blk_opf_t)1); } +static inline bool op_is_copy(blk_opf_t op) +{ + return ((op & REQ_OP_MASK) == REQ_OP_COPY_SRC || + (op & REQ_OP_MASK) == REQ_OP_COPY_DST); +} + /* * Check if the bio or request is one that needs special treatment in the * flush state machine. From patchwork Wed Dec 6 10:02:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 13481352 Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 951F92D033 for ; Wed, 6 Dec 2023 10:11:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="uQN52hz2" Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20231206101107epoutp031a782f38cfb6e0800ca245f1a5e8e7f5~eNlmfrBE52270722707epoutp03D for ; Wed, 6 Dec 2023 10:11:07 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20231206101107epoutp031a782f38cfb6e0800ca245f1a5e8e7f5~eNlmfrBE52270722707epoutp03D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857467; bh=Ff48lndit/eTqrBB/CdVOfgqav/hSRfanUmmPiHdWuQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uQN52hz2+IoyRnPDdGThIBI3+focXLNJGnfYB9GqOH13LIFB+9u+A4txvmwKJDn0J CBiApWDf6BYKI2pz2uBsWCo47nvoCWJLoVRVPb/XE98SYxJaTsFXK4GKuWYKbwQXQX aG6Xuu8oztrr5aduBuSRVQQeMZ+dQ/Rs1dkP5nKs= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20231206101107epcas5p1946f7fef7afffbbc255f218e0bc1fddb~eNll1doh80441904419epcas5p1s; Wed, 6 Dec 2023 10:11:07 +0000 (GMT) Received: from epsmgec5p1-new.samsung.com (unknown [182.195.38.174]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4SlY892ZB5z4x9Q6; Wed, 6 Dec 2023 10:11:05 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmgec5p1-new.samsung.com (Symantec Messaging Gateway) with SMTP id 15.68.19369.9B840756; Wed, 6 Dec 2023 19:11:05 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20231206101104epcas5p41c9a32e6cbc2ef6dd8870cf60cd1c7ce~eNljw2uEJ0156001560epcas5p4j; Wed, 6 Dec 2023 10:11:04 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20231206101104epsmtrp1bb6ea52a119faef623ae70279b30626e~eNljvswjk1053310533epsmtrp17; Wed, 6 Dec 2023 10:11:04 +0000 (GMT) X-AuditID: b6c32a50-c99ff70000004ba9-3e-657048b9f86e Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id CE.42.08817.8B840756; Wed, 6 Dec 2023 19:11:04 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101059epsmtip24a1dc46fe3d60ea974f9479bbb7215f4~eNlesowsz1180411804epsmtip2G; Wed, 6 Dec 2023 10:10:59 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 03/12] block: add copy offload support Date: Wed, 6 Dec 2023 15:32:35 +0530 Message-Id: <20231206100253.13100-4-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01TbUxbZRTOe+/lUkhgdxTDmy4TaEIW6oDWFXiLY4w59SJjaVyixh/gBa5A gLb2Y0OXIIOtzuLGV1BgbjAGDKg4vqVAZ4FBVwRRJo7vIB9hSObG0EHATluKun/POec553mf 8+ZwcI88Zx4nRaZmlTImjU+6Eu19/gcCDLSCFXZM70G3BgdwtPbnNoGy86040s/kkWi17wlA i6ZPAaqovEqgCZMBQ92VhRiq0/djqLD3F4CWxsowZJx8CV3XVhGo22gh0L3Or0hUXrPkjHLv d5DopvkZhsbzlwAquDiGoY7FcwC1b5fj6JvVRwS6O7kPjVjNTkd5tKFsxpkemW0i6HvDGrq5 /jOSbqn6hH7QUgrorokskr5xuciJvpTzO0mvLU0S9KPbYyR9ubUe0C3fn6XXm1+kmxcfYtI9 76UeTmaZRFbpw8oS5IkpsqRwfvSpuFfjgkOEogCRBIXyfWRMOhvOP35CGvB6SpptJXyf00ya xpaSMioVP+jIYaVco2Z9kuUqdTifVSSmKcSKQBWTrtLIkgJlrDpMJBS+HGwjvp+a3LBQRyra jmZcKzY4Z4FqsQ5wOJASw5K1MzrgyvGgugFc1w0RjuAJgMYrs7gjeAqgrnbdWQdcdjq0M1W7 BSOAX7bfcHIE6wDeza/E7XNJyh/+WKSxN3hSX+PQ0CSyc3DqGg5b5s2YvcClwuCyRYvZ+QTl B/M6oT3tRiF43bRJOsS8Yenoxo6wCyWBRcv28XbOXmgpXSTsGLdxctqu7DwIUl0ucHv0PO5o Pg7XahZ2MRf+Zm7ddcCDK3naXZwAR0t/wBxYDRe6e3ZxBLwwmLfjBbd5udUZ5NByh5e2FzHH 6tzgRa2Hg+0LZwuXnBzYC/5aUrWLaWjp6cMc6/kcwJLNh875wLvsOQtlz1ko+1+tAuD1gMcq VOlJbEKwQhQgY8/897EJ8vRmsHMZAmkH0DdaA3sBxgG9AHJwvqdb2oic9XBLZD76mFXK45Sa NFbVC4JtOy7AeS8kyG2nJVPHicQSoTgkJEQsORQi4nu5rV64muhBJTFqNpVlFazy3z6M48LL wrQrr/20f3N4a3V9OKp4fLzxQ64v6W7gklbvgx23w7l61V9m2UTGSpt4pm5kLtMUGiuw9lhC 5ytTfH1qiOiijM5cY7k6d+UtbvbsyYXaqT9MpdWPt6cbpPJ+r8APUsxvCPc5nc489mDzHUF/ 5P2o5dqpQ8eapM+cmKGzGZ5fjB78VqcT6GclkhyT/8nRra2t5DCW+6ZHhzvIrY+JDXssaPiu OH2KzaptGDjV9feMJftOzNtSv+rzP5fepJhYF78T1qigDd9lr8i6rohKYIxfjykwKfqHzjUK 989lzO+d6lUdwCMK3j2SKXhlblAv4HU9jdy4MxBvDnBtna6Ini+IL+YTqmRGJMCVKuYfPk8q bqIEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrEIsWRmVeSWpSXmKPExsWy7bCSvO4Oj4JUg5VbVC3WnzrGbPHx628W i6YJf5ktVt/tZ7N4ffgTo8WTA+2MFgsWzWWxuHlgJ5PFnkWTmCxWrj7KZDHp0DVGi6dXZzFZ 7L2lbbGwbQmLxZ69J1ksLu+aw2Yxf9lTdovu6zvYLJYf/8dkcWPCU0aLiR1XmSx2PGlktNj2 ez6zxbrX71ksTtyStjj/9zirg5THzll32T3O39vI4nH5bKnHplWdbB6bl9R7vNg8k9Fj980G No/FfZNZPXqb37F5fHx6i8Xj/b6rbB59W1Yxemw+Xe3xeZOcx6Ynb5kC+KO4bFJSczLLUov0 7RK4MtY+XslWsNWhYt7UnewNjEtNuhg5OSQETCTa7i5h7mLk4hAS2M0o8WfxDlaIhLhE87Uf 7BC2sMTKf8/ZIYo+Mkr0NCxg7GLk4GAT0JS4MLkUJC4isINZ4ufaZiaQBmaBNcwSW+bygtjC AlYSz0+2MYHUswioSvTvkgAJ8wpYSCw88IMNYr68xMxL38F2cQpYSkx+vogZxBYCqtnXOJ0F ol5Q4uTMJywQ4+UlmrfOZp7AKDALSWoWktQCRqZVjJKpBcW56bnFhgVGeanlesWJucWleel6 yfm5mxjB0a6ltYNxz6oPeocYmTgYDzFKcDArifDmnM9PFeJNSaysSi3Kjy8qzUktPsQozcGi JM777XVvipBAemJJanZqakFqEUyWiYNTqoFJR0rxd80P4+smgbX3wpdeX/9/YtjsxTZ8q9y+ OMx4OKUg/My/PV9yhT/FuflVWR/zuMZW1JXDx/y6cKXqHt0Tjf8s7j36WddxrZ9h26kpvCrm J+XfdvlpzHP9p7vg4vzvR1rrpZdEpUUpC0v8bPfr9BPP+OI9K9rykJRZjNKp1sxNUka6Sy+r c04qPdLiLJpd0LZwUuaM1dfnHbqU3DTLaet6L7Z18lGvr252U+VoOJex/0vPQkGBy6y3/jx6 wr/g6wrD+ROD7wi7K55lemxjNM/yweJllt/acyW/fZX4e2vB9RO6AkLeUpdNrvWfC87hmT8z IPTL7fB3Bxz9ZnNrJq6++GbexIQ1gb8kRF7fU2Ipzkg01GIuKk4EADr1Q49lAwAA X-CMS-MailID: 20231206101104epcas5p41c9a32e6cbc2ef6dd8870cf60cd1c7ce X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101104epcas5p41c9a32e6cbc2ef6dd8870cf60cd1c7ce References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty Introduce blkdev_copy_offload to perform copy offload. Issue REQ_OP_COPY_SRC with source info along with taking a plug. This flows till request layer and waits for dst bio to arrive. Issue REQ_OP_COPY_DST with destination info and this bio reaches request layer and merges with src request. For any reason, if a request comes to the driver with only one of src/dst bio, we fail the copy offload. Larger copy will be divided, based on max_copy_sectors limit. Reviewed-by: Hannes Reinecke Signed-off-by: Anuj Gupta Signed-off-by: Nitesh Shetty --- block/blk-lib.c | 204 +++++++++++++++++++++++++++++++++++++++++ include/linux/blkdev.h | 4 + 2 files changed, 208 insertions(+) diff --git a/block/blk-lib.c b/block/blk-lib.c index e59c3069e835..05dbe7fa5354 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -10,6 +10,22 @@ #include "blk.h" +/* Keeps track of all outstanding copy IO */ +struct blkdev_copy_io { + atomic_t refcount; + ssize_t copied; + int status; + struct task_struct *waiter; + void (*endio)(void *private, int status, ssize_t copied); + void *private; +}; + +/* Keeps track of single outstanding copy offload IO */ +struct blkdev_copy_offload_io { + struct blkdev_copy_io *cio; + loff_t offset; +}; + static sector_t bio_discard_limit(struct block_device *bdev, sector_t sector) { unsigned int discard_granularity = bdev_discard_granularity(bdev); @@ -115,6 +131,194 @@ int blkdev_issue_discard(struct block_device *bdev, sector_t sector, } EXPORT_SYMBOL(blkdev_issue_discard); +static inline ssize_t blkdev_copy_sanity_check(struct block_device *bdev_in, + loff_t pos_in, + struct block_device *bdev_out, + loff_t pos_out, size_t len) +{ + unsigned int align = max(bdev_logical_block_size(bdev_out), + bdev_logical_block_size(bdev_in)) - 1; + + if ((pos_in & align) || (pos_out & align) || (len & align) || !len || + len >= BLK_COPY_MAX_BYTES) + return -EINVAL; + + return 0; +} + +static inline void blkdev_copy_endio(struct blkdev_copy_io *cio) +{ + if (cio->endio) { + cio->endio(cio->private, cio->status, cio->copied); + kfree(cio); + } else { + struct task_struct *waiter = cio->waiter; + + WRITE_ONCE(cio->waiter, NULL); + blk_wake_io_task(waiter); + } +} + +/* + * This must only be called once all bios have been issued so that the refcount + * can only decrease. This just waits for all bios to complete. + * Returns the length of bytes copied or error + */ +static ssize_t blkdev_copy_wait_for_completion_io(struct blkdev_copy_io *cio) +{ + ssize_t ret; + + for (;;) { + __set_current_state(TASK_UNINTERRUPTIBLE); + if (!READ_ONCE(cio->waiter)) + break; + blk_io_schedule(); + } + __set_current_state(TASK_RUNNING); + ret = cio->copied; + kfree(cio); + + return ret; +} + +static void blkdev_copy_offload_dst_endio(struct bio *bio) +{ + struct blkdev_copy_offload_io *offload_io = bio->bi_private; + struct blkdev_copy_io *cio = offload_io->cio; + + if (bio->bi_status) { + cio->copied = min_t(ssize_t, offload_io->offset, cio->copied); + if (!cio->status) + cio->status = blk_status_to_errno(bio->bi_status); + } + bio_put(bio); + kfree(offload_io); + + if (atomic_dec_and_test(&cio->refcount)) + blkdev_copy_endio(cio); +} + +/* + * @bdev: block device + * @pos_in: source offset + * @pos_out: destination offset + * @len: length in bytes to be copied + * @endio: endio function to be called on completion of copy operation, + * for synchronous operation this should be NULL + * @private: endio function will be called with this private data, + * for synchronous operation this should be NULL + * @gfp_mask: memory allocation flags (for bio_alloc) + * + * For synchronous operation returns the length of bytes copied or error + * For asynchronous operation returns -EIOCBQUEUED or error + * + * Description: + * Copy source offset to destination offset within block device, using + * device's native copy offload feature. + * We perform copy operation using 2 bio's. + * 1. We take a plug and send a REQ_OP_COPY_SRC bio along with source + * sector and length. Once this bio reaches request layer, we form a + * request and wait for dst bio to arrive. + * 2. We issue REQ_OP_COPY_DST bio along with destination sector, length. + * Once this bio reaches request layer and find a request with previously + * sent source info we merge the destination bio and return. + * 3. Release the plug and request is sent to driver + * This design works only for drivers with request queue. + */ +ssize_t blkdev_copy_offload(struct block_device *bdev, loff_t pos_in, + loff_t pos_out, size_t len, + void (*endio)(void *, int, ssize_t), + void *private, gfp_t gfp) +{ + struct blkdev_copy_io *cio; + struct blkdev_copy_offload_io *offload_io; + struct bio *src_bio, *dst_bio; + size_t rem, chunk; + size_t max_copy_bytes = bdev_max_copy_sectors(bdev) << SECTOR_SHIFT; + ssize_t ret; + struct blk_plug plug; + + if (!max_copy_bytes) + return -EOPNOTSUPP; + + ret = blkdev_copy_sanity_check(bdev, pos_in, bdev, pos_out, len); + if (ret) + return ret; + + cio = kzalloc(sizeof(*cio), gfp); + if (!cio) + return -ENOMEM; + atomic_set(&cio->refcount, 1); + cio->waiter = current; + cio->endio = endio; + cio->private = private; + + /* + * If there is a error, copied will be set to least successfully + * completed copied length + */ + cio->copied = len; + for (rem = len; rem > 0; rem -= chunk) { + chunk = min(rem, max_copy_bytes); + + offload_io = kzalloc(sizeof(*offload_io), gfp); + if (!offload_io) + goto err_free_cio; + offload_io->cio = cio; + /* + * For partial completion, we use offload_io->offset to truncate + * successful copy length + */ + offload_io->offset = len - rem; + + src_bio = bio_alloc(bdev, 0, REQ_OP_COPY_SRC, gfp); + if (!src_bio) + goto err_free_offload_io; + src_bio->bi_iter.bi_size = chunk; + src_bio->bi_iter.bi_sector = pos_in >> SECTOR_SHIFT; + + blk_start_plug(&plug); + dst_bio = blk_next_bio(src_bio, bdev, 0, REQ_OP_COPY_DST, gfp); + if (!dst_bio) + goto err_free_src_bio; + dst_bio->bi_iter.bi_size = chunk; + dst_bio->bi_iter.bi_sector = pos_out >> SECTOR_SHIFT; + dst_bio->bi_end_io = blkdev_copy_offload_dst_endio; + dst_bio->bi_private = offload_io; + + atomic_inc(&cio->refcount); + submit_bio(dst_bio); + blk_finish_plug(&plug); + pos_in += chunk; + pos_out += chunk; + } + + if (atomic_dec_and_test(&cio->refcount)) + blkdev_copy_endio(cio); + if (endio) + return -EIOCBQUEUED; + + return blkdev_copy_wait_for_completion_io(cio); + +err_free_src_bio: + bio_put(src_bio); +err_free_offload_io: + kfree(offload_io); +err_free_cio: + cio->copied = min_t(ssize_t, cio->copied, (len - rem)); + cio->status = -ENOMEM; + if (rem == len) { + ret = cio->status; + kfree(cio); + return ret; + } + if (cio->endio) + return cio->status; + + return blkdev_copy_wait_for_completion_io(cio); +} +EXPORT_SYMBOL_GPL(blkdev_copy_offload); + static int __blkdev_issue_write_zeroes(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop, unsigned flags) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 4e17945041c8..e8582a38adb7 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1042,6 +1042,10 @@ int __blkdev_issue_discard(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop); int blkdev_issue_secure_erase(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp); +ssize_t blkdev_copy_offload(struct block_device *bdev, loff_t pos_in, + loff_t pos_out, size_t len, + void (*endio)(void *, int, ssize_t), + void *private, gfp_t gfp_mask); #define BLKDEV_ZERO_NOUNMAP (1 << 0) /* do not free blocks */ #define BLKDEV_ZERO_NOFALLBACK (1 << 1) /* don't write explicit zeroes */ From patchwork Wed Dec 6 10:02:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 13481353 Received: from mailout3.samsung.com (mailout3.samsung.com [203.254.224.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0C86928E0E for ; Wed, 6 Dec 2023 10:11:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="Zvsgxf9B" Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout3.samsung.com (KnoxPortal) with ESMTP id 20231206101121epoutp03528fb8d1108854fd6b74a2d2b51ac547~eNly8FEVo2449424494epoutp03D for ; Wed, 6 Dec 2023 10:11:21 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout3.samsung.com 20231206101121epoutp03528fb8d1108854fd6b74a2d2b51ac547~eNly8FEVo2449424494epoutp03D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857481; bh=/6OhQ9ODX//umK/uU/u7BcJDEaHDVpwjz5uay8Kefew=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Zvsgxf9BFlB3by/cG0LjnxSw9PuIrmn+Ve6wLU+4nkYVKhCVyUYAV49SvJFxqXE2a FhUTk0u0yqd621sjpzzUXVqKKMTaaU+632OV7znew28DJLjcrqtZoIXt5ltX0G9DcT 1kamBpJgHgrhwn7TugpqnIZ17VLgwnjtYLYWMStw= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20231206101120epcas5p2f9ffcb1fb083bd23d3b4fb98147d54d9~eNlyWpE5K2940529405epcas5p27; Wed, 6 Dec 2023 10:11:20 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.174]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4SlY8Q5kFBz4x9Pv; Wed, 6 Dec 2023 10:11:18 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id F5.C6.10009.6C840756; Wed, 6 Dec 2023 19:11:18 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20231206101118epcas5p1cc77b49dbd8bc1601423d02527b03122~eNlwWIZO30449604496epcas5p1v; Wed, 6 Dec 2023 10:11:18 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20231206101118epsmtrp2882cd222b4c8fcca5aaafc6113b65840~eNlwVBeI-0924709247epsmtrp2d; Wed, 6 Dec 2023 10:11:18 +0000 (GMT) X-AuditID: b6c32a4a-261fd70000002719-2d-657048c6a55e Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id B1.52.08817.6C840756; Wed, 6 Dec 2023 19:11:18 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101113epsmtip278f834a99ee8a24a8447ab449801a2ff~eNlrq_aCz3173831738epsmtip2Z; Wed, 6 Dec 2023 10:11:13 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Vincent Fu , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 04/12] block: add emulation for copy Date: Wed, 6 Dec 2023 15:32:36 +0530 Message-Id: <20231206100253.13100-5-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01TbUxTVxjeufdyWyB1bdVxVhOBOtwE+ShruwMDNGLcjWMZC/90k93QO0BK 2/UDYdkyKoJYUPmQAVU+hM4PMDAKQ0C6ACqgC2AkdYMENqSdczAU2TBaYGtp3fz3vM/7PHnf 5z05bJxfzBKw0xVaRq2g5ULSh+i8vuOt0EFKxUQsVryJWm8P4mjxbweBjpas4qh56jSJ5q4/ AcjWdxyg+oYaAk30dWPocvNNDJUN3APIbjViyDIZgs4XmAjUa7lFoPGecySqu2BnoaKfukh0 cWgNQz+X2AEqLbRiqMumB6jTUYejlrlHBBqe3IJmiwoBGlsd8totoLqNUyxqbLqNoMZHdJS5 6QRJtZu+pn5vrwbUtYlckmo8Ve5FncxbIKlF+yRBPfrBSlKnOpoAtWTeSpltf2KJGw5kxKQx tIxRBzCKFKUsXZEaK3w/KTk+WSKNEIWKotA7wgAFncnECvcmJIbuS5c7zyEMyKLlOieVSGs0 wvC4GLVSp2UC0pQabayQUcnkKrEqTENnanSK1DAFo40WRURESpzCTzPS6hqrMdXanmxL/n1W LjBKDcCbDbliOL/8i5cB+LD53GsALhvPeIonABZYbbi7WAYw76aeeGEpMOtJd8MCYGmlwWNZ ArCrfcRpYbNJ7g54p1znMmziXsFhd5vIpcG5jTi0TNeRrsZGLoJPHzSsY4IbBEcfm4ELc5z8 1IkVzD3NH1bffcpyYW9uFCx/0IC7NTx4q9q2vhHu1OR9f3Z9Vcht9oa1fVbcbd4Ln/e3km68 Ef4x1MFyYwFcWrB4+BR4t3rUM0wLZ3v7PXgXzL99ej0M7gzT2hPunrUBnnTYMBcNuRxYWMB3 qwPhdJndy4394EyVyYMpWLFQ6bliMYDHzzjIEuBvfCmC8aUIxv+n1QO8CbzOqDSZqYxGoopU MEf+e9kUZaYZrH+L4P1dYObXx2EDAGODAQDZuHATRz6mZPgcGZ3zBaNWJqt1ckYzACTOG5fi gs0pSue/UmiTReKoCLFUKhVHvS0VCf04c/k1Mj43ldYyGQyjYtQvfBjbW5CLHY7P9x+TJSx8 80zTtnKBlEhHR2iJrmN5WF+/1h7MYz7x0dtO768N510y7Ml6pb9eEDqX/bmpNyZ2vr32K8eq 32vfPjRE51Z999lC/BX7fBnZ+WGsMr21SPXxoUFfwrRdygvPqpq+kR1/x5EjzNiWdMN3URx9 eXem0bYlsOX+q/T0wbOVB3lf6g/HV/GPgn1bI7OuXqo4BoNUVQdsjp6HV01HmH92jdeY69Y4 YY0zm5fj5tFHgiC6z3+4x94zqh191i+dNIRd3B4Xfqhv24xS917WwM4JVsK5D3LLBn8MTmxr M4WcX13xKy66R/NUwbOBLbbfQnz7n79h/QvtfDfmWFKOkNCk0aJgXK2h/wVQ6RsynwQAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA02Ra0hTYRjHe885nh1H2mmWvnnJGhSpNZtEvF3JCDsFUlFGFJRLD2ptc+xo N4000VLzHtROhdes1tVlMq/pLM2oVt4qaRW5sUwst252meUcQd9+PL//8/w/PBQuMhK+VIIy iVUrZXIxKSTq2sWBizoYFbvYZpmKbj7swJHt6y8CHS904OiqqYBEw+12gMytJwAqq7hAoJet 9Ri6cvU+hooN/QBZ+ngMNQ+EoPKsKgI1NXcRqKfhPIlKqy0ClPtcT6JLneMYelFoAajoZB+G 9OZ0gOp+leLoxvAnAj0Y8EODuScBMjo63db4MvW8ScAYX9cQTM/jZEanzSaZ21XHmPe3NYBp fJlGMpX5JW5MXsZHkrFZBgjmU0sfyeTXagHzWTeb0ZlHsM2eO4UrY1l5wgFWHbo6WhhfWqnB VONrDzVnvhOkAX5pDnCnIL0EZunSyRwgpER0I4D8rRuES/jAjP4xgYu94JVxq8AVsgE4ducD ngMoiqSD4NOSZOd8Bq3H4Y/rGZhzAadrcNjdMdngRSP43VpBOpmg58EnozrgZI+JuSn7N+Yq CISa7u+TZe70MlhircCdLJrItKSfIVz56bBLYyZc9wNhxp1zeCGg+f8U/58qA5gWzGJVnCJO wUlVYUr2oISTKbhkZZwkJlGhA5M/Dw7WgybtqMQAMAoYAKRw8QwPuTGRFXnEyg4fYdWJe9TJ cpYzAD+KEPt4fBvOixXRcbIkdj/Lqlj1P4tR7r5pmK+3/4lDl8FyxR/vftHCqI2nPmyTvA2r Dh9ZtY4vmGKd/7AoZWSBnbPSx66fXv5z9OC13AsJbdPW1wfE7mjcG3W47Uv7G+ub2tQAPmlm zM8QB2OXrM8acjQmxUVuN68TzpVL+reeDY2I8equHHmWPX50S4GxIf5Mb2ZDud27q/yden6R dMXrojFoEM/pexXR+nhtVdfgzUf+X03f2jR8AdguLUv1CQ5x643Y5Zmiq0HS81HVOdy2WcYN vSWiVLtJHxSWN9QTXdEb6B8Ejea9LcW7Pod3rrzorcxuytwXWuu+ZIi+Fxm+W5tG2Ui30Rr4 xdOUUnx36uLaOsfg4KbIu9FigouXSYNxNSf7C2PAyuliAwAA X-CMS-MailID: 20231206101118epcas5p1cc77b49dbd8bc1601423d02527b03122 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101118epcas5p1cc77b49dbd8bc1601423d02527b03122 References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty For the devices which does not support copy, copy emulation is added. It is required for in-kernel users like fabrics, where file descriptor is not available and hence they can't use copy_file_range. Copy-emulation is implemented by reading from source into memory and writing to the corresponding destination. At present in kernel user of emulation is fabrics. Signed-off-by: Nitesh Shetty Signed-off-by: Vincent Fu Signed-off-by: Anuj Gupta --- block/blk-lib.c | 223 +++++++++++++++++++++++++++++++++++++++++ include/linux/blkdev.h | 4 + 2 files changed, 227 insertions(+) diff --git a/block/blk-lib.c b/block/blk-lib.c index 05dbe7fa5354..e32e00e4a1eb 100644 --- a/block/blk-lib.c +++ b/block/blk-lib.c @@ -26,6 +26,20 @@ struct blkdev_copy_offload_io { loff_t offset; }; +/* Keeps track of single outstanding copy emulation IO */ +struct blkdev_copy_emulation_io { + struct blkdev_copy_io *cio; + struct work_struct emulation_work; + void *buf; + ssize_t buf_len; + loff_t pos_in; + loff_t pos_out; + ssize_t len; + struct block_device *bdev_in; + struct block_device *bdev_out; + gfp_t gfp; +}; + static sector_t bio_discard_limit(struct block_device *bdev, sector_t sector) { unsigned int discard_granularity = bdev_discard_granularity(bdev); @@ -319,6 +333,215 @@ ssize_t blkdev_copy_offload(struct block_device *bdev, loff_t pos_in, } EXPORT_SYMBOL_GPL(blkdev_copy_offload); +static void *blkdev_copy_alloc_buf(ssize_t req_size, ssize_t *alloc_size, + gfp_t gfp) +{ + int min_size = PAGE_SIZE; + char *buf; + + while (req_size >= min_size) { + buf = kvmalloc(req_size, gfp); + if (buf) { + *alloc_size = req_size; + return buf; + } + req_size >>= 1; + } + + return NULL; +} + +static struct bio *bio_map_buf(void *data, unsigned int len, gfp_t gfp) +{ + unsigned long kaddr = (unsigned long)data; + unsigned long end = (kaddr + len + PAGE_SIZE - 1) >> PAGE_SHIFT; + unsigned long start = kaddr >> PAGE_SHIFT; + const int nr_pages = end - start; + bool is_vmalloc = is_vmalloc_addr(data); + struct page *page; + int offset, i; + struct bio *bio; + + bio = bio_kmalloc(nr_pages, gfp); + if (!bio) + return ERR_PTR(-ENOMEM); + bio_init(bio, NULL, bio->bi_inline_vecs, nr_pages, 0); + + if (is_vmalloc) { + flush_kernel_vmap_range(data, len); + bio->bi_private = data; + } + + offset = offset_in_page(kaddr); + for (i = 0; i < nr_pages; i++) { + unsigned int bytes = PAGE_SIZE - offset; + + if (len <= 0) + break; + + if (bytes > len) + bytes = len; + + if (!is_vmalloc) + page = virt_to_page(data); + else + page = vmalloc_to_page(data); + if (bio_add_page(bio, page, bytes, offset) < bytes) { + /* we don't support partial mappings */ + bio_uninit(bio); + kfree(bio); + return ERR_PTR(-EINVAL); + } + + data += bytes; + len -= bytes; + offset = 0; + } + + return bio; +} + +static void blkdev_copy_emulation_work(struct work_struct *work) +{ + struct blkdev_copy_emulation_io *emulation_io = container_of(work, + struct blkdev_copy_emulation_io, emulation_work); + struct blkdev_copy_io *cio = emulation_io->cio; + struct bio *read_bio, *write_bio; + loff_t pos_in = emulation_io->pos_in, pos_out = emulation_io->pos_out; + ssize_t rem, chunk; + int ret = 0; + + for (rem = emulation_io->len; rem > 0; rem -= chunk) { + chunk = min_t(int, emulation_io->buf_len, rem); + + read_bio = bio_map_buf(emulation_io->buf, + emulation_io->buf_len, + emulation_io->gfp); + if (IS_ERR(read_bio)) { + ret = PTR_ERR(read_bio); + break; + } + read_bio->bi_opf = REQ_OP_READ | REQ_SYNC; + bio_set_dev(read_bio, emulation_io->bdev_in); + read_bio->bi_iter.bi_sector = pos_in >> SECTOR_SHIFT; + read_bio->bi_iter.bi_size = chunk; + ret = submit_bio_wait(read_bio); + kfree(read_bio); + if (ret) + break; + + write_bio = bio_map_buf(emulation_io->buf, + emulation_io->buf_len, + emulation_io->gfp); + if (IS_ERR(write_bio)) { + ret = PTR_ERR(write_bio); + break; + } + write_bio->bi_opf = REQ_OP_WRITE | REQ_SYNC; + bio_set_dev(write_bio, emulation_io->bdev_out); + write_bio->bi_iter.bi_sector = pos_out >> SECTOR_SHIFT; + write_bio->bi_iter.bi_size = chunk; + ret = submit_bio_wait(write_bio); + kfree(write_bio); + if (ret) + break; + + pos_in += chunk; + pos_out += chunk; + } + cio->status = ret; + kvfree(emulation_io->buf); + kfree(emulation_io); + blkdev_copy_endio(cio); +} + +static inline ssize_t queue_max_hw_bytes(struct request_queue *q) +{ + return min_t(ssize_t, queue_max_hw_sectors(q) << SECTOR_SHIFT, + queue_max_segments(q) << PAGE_SHIFT); +} +/* + * @bdev_in: source block device + * @pos_in: source offset + * @bdev_out: destination block device + * @pos_out: destination offset + * @len: length in bytes to be copied + * @endio: endio function to be called on completion of copy operation, + * for synchronous operation this should be NULL + * @private: endio function will be called with this private data, + * for synchronous operation this should be NULL + * @gfp_mask: memory allocation flags (for bio_alloc) + * + * For synchronous operation returns the length of bytes copied or error + * For asynchronous operation returns -EIOCBQUEUED or error + * + * Description: + * If native copy offload feature is absent, caller can use this function + * to perform copy. + * We store information required to perform the copy along with temporary + * buffer allocation. We async punt copy emulation to a worker. And worker + * performs copy in 2 steps. + * 1. Read data from source to temporary buffer + * 2. Write data to destination from temporary buffer + */ +ssize_t blkdev_copy_emulation(struct block_device *bdev_in, loff_t pos_in, + struct block_device *bdev_out, loff_t pos_out, + size_t len, void (*endio)(void *, int, ssize_t), + void *private, gfp_t gfp) +{ + struct request_queue *in = bdev_get_queue(bdev_in); + struct request_queue *out = bdev_get_queue(bdev_out); + struct blkdev_copy_emulation_io *emulation_io; + struct blkdev_copy_io *cio; + ssize_t ret; + size_t max_hw_bytes = min(queue_max_hw_bytes(in), + queue_max_hw_bytes(out)); + + ret = blkdev_copy_sanity_check(bdev_in, pos_in, bdev_out, pos_out, len); + if (ret) + return ret; + + cio = kzalloc(sizeof(*cio), gfp); + if (!cio) + return -ENOMEM; + + cio->waiter = current; + cio->copied = len; + cio->endio = endio; + cio->private = private; + + emulation_io = kzalloc(sizeof(*emulation_io), gfp); + if (!emulation_io) + goto err_free_cio; + emulation_io->cio = cio; + INIT_WORK(&emulation_io->emulation_work, blkdev_copy_emulation_work); + emulation_io->pos_in = pos_in; + emulation_io->pos_out = pos_out; + emulation_io->len = len; + emulation_io->bdev_in = bdev_in; + emulation_io->bdev_out = bdev_out; + emulation_io->gfp = gfp; + + emulation_io->buf = blkdev_copy_alloc_buf(min(max_hw_bytes, len), + &emulation_io->buf_len, gfp); + if (!emulation_io->buf) + goto err_free_emulation_io; + + schedule_work(&emulation_io->emulation_work); + + if (cio->endio) + return -EIOCBQUEUED; + + return blkdev_copy_wait_for_completion_io(cio); + +err_free_emulation_io: + kfree(emulation_io); +err_free_cio: + kfree(cio); + return -ENOMEM; +} +EXPORT_SYMBOL_GPL(blkdev_copy_emulation); + static int __blkdev_issue_write_zeroes(struct block_device *bdev, sector_t sector, sector_t nr_sects, gfp_t gfp_mask, struct bio **biop, unsigned flags) diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index e8582a38adb7..9fa1ad68beb5 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1046,6 +1046,10 @@ ssize_t blkdev_copy_offload(struct block_device *bdev, loff_t pos_in, loff_t pos_out, size_t len, void (*endio)(void *, int, ssize_t), void *private, gfp_t gfp_mask); +ssize_t blkdev_copy_emulation(struct block_device *bdev_in, loff_t pos_in, + struct block_device *bdev_out, loff_t pos_out, + size_t len, void (*endio)(void *, int, ssize_t), + void *private, gfp_t gfp); #define BLKDEV_ZERO_NOUNMAP (1 << 0) /* do not free blocks */ #define BLKDEV_ZERO_NOFALLBACK (1 << 1) /* don't write explicit zeroes */ From patchwork Wed Dec 6 10:02:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 13481354 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3A552D059 for ; Wed, 6 Dec 2023 10:11:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="DphrSG+X" Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20231206101135epoutp04aea7c5c13299a001e30239103b060961~eNmArbfv_1401914019epoutp04M for ; Wed, 6 Dec 2023 10:11:35 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20231206101135epoutp04aea7c5c13299a001e30239103b060961~eNmArbfv_1401914019epoutp04M DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857495; bh=3rmMeBvqvZnLG710cTQe8+pDj6/LHJ1Es1eTLbldN3I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DphrSG+Xwn2/GmRM0Y5eZ21WK+jKGIYamGjzIFChOCw5HfsG25bvH5ubi7CdzGiGk pGMYchCk2MEIHJanydNnElpttb/sY+ji+46RVummntAYDa3CM3Ek1Fcnag3rbgM8Z9 WRJRe0WNPzLJb4+1EQ3stmDnLnuSn3sek9qy/9u8= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20231206101135epcas5p366d5b258f1c0a8617a18ef7ac61e9e00~eNmAQkMlO0036000360epcas5p36; Wed, 6 Dec 2023 10:11:35 +0000 (GMT) Received: from epsmgec5p1new.samsung.com (unknown [182.195.38.177]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4SlY8j3VPtz4x9Ps; Wed, 6 Dec 2023 10:11:33 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmgec5p1new.samsung.com (Symantec Messaging Gateway) with SMTP id 12.52.08567.5D840756; Wed, 6 Dec 2023 19:11:33 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20231206101133epcas5p28efae88fbc435e0a42bc89c94eec3ac2~eNl_BD5Zu3161231612epcas5p27; Wed, 6 Dec 2023 10:11:33 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20231206101132epsmtrp2312dbf0d8724cfaaf3b06742b41fe8ee~eNl9--tcq0990909909epsmtrp2J; Wed, 6 Dec 2023 10:11:32 +0000 (GMT) X-AuditID: b6c32a44-3abff70000002177-42-657048d5891e Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 24.52.08817.4D840756; Wed, 6 Dec 2023 19:11:32 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101126epsmtip2a3dcf704089d260dd01017179cde49db~eNl4WElIo1181511815epsmtip2G; Wed, 6 Dec 2023 10:11:26 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Anuj Gupta , Hannes Reinecke , Nitesh Shetty , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 05/12] fs/read_write: Enable copy_file_range for block device. Date: Wed, 6 Dec 2023 15:32:37 +0530 Message-Id: <20231206100253.13100-6-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02TaVBTVxTHe98LL4ESfY10uJCKmBk7QgWSst1YoLYGeQydlpn2g9h2Qgae gUlI0ixuoGItIJsg2FYiS1iqI4IIqOzWwlRERKiUilhSCgl1DEtl0U4z0iYEW7/9zrn//5zl zmHhnAKmJytZoaXVComcR7gwrvX4+PiNUCqaf6EcoobbN3H0ZNnKQF8WPsfRxfECAll6FgAy 3cgCyFBVxkAPbrRhqLOqCEMXLv6IoaLuXwAyj+gx1DX2FqrMrGGgzq4+BhpuLyVQxTkzE+Xe byXQ+d4VDI0WmgE6dWIEQ62mYwBds1bg6JJlnoFujXHR4PNepx2eVJt+nEkNGhsZ1PCAjmqq zSao5pqj1KPmEkB1PEgnqOqTxU5U/vE5gnpiHmNQ89dHCOrklVpANfenUotNXlSTaRaLXb9H FpZESxJptTetSFAmJiuk4byYj8U7xcEhfIGfQIhCed4KSQodzhN9EOu3K1luWwnPe59ErrOl YiUaDS8gIkyt1Glp7ySlRhvOo1WJclWQyl8jSdHoFFJ/Ba3dLuDz3w62CeNlSUZDNqGaYx+o t46CdDD0ag5wZkEyCOYuXydygAuLQ3YAeL44H3MECwDetdYBR/AUwO9KpvAXlobSvDVLF4Dj My24I1gEsLc/i5kDWCyC9IFDxTq7wY2sw2Fbo8CuwclyHE7/YwD2hw3kJ3D2VjnTzgxyC2zU D68ym0RwObMfOKptgiX3nq3mnUkhLP6jCndoXoN9JSaGnXGb5vjVs6tNQLLDGS6cu8p0mEVw 8r6R4eAN8HHvlbW8J1yc6yIcnADvldzFHKyFU50/rPG7MON2AW4fBrcN09Ae4Ki1DuZbTZg9 DUk2PJHJcag3Q2OR2cnB7vD3MzVrTMGM+YHVDjhkHoDfWKMLwSb9SxPoX5pA/38xA8BrgQet 0qRI6YRglUBB7//vYxOUKU1g9TJ8Ra1gtGLFvxtgLNANIAvnubHlg0qaw06UHDxEq5VitU5O a7pBsG3Fp3DP1xOUttNSaMWCICE/KCQkJEgYGCLgubMtGWWJHFIq0dIymlbR6hc+jOXsmY5N 9aXyZ4yz4evzOqeR61Tk0tkj3A73HS20zjIBpJ2lFbKDae4ajzFRfVjZLlZNjPSNrIdRT8Wi qIKcw68w+3bXB8ZEdIRPurumPbxc2R+vdMld56uT4eOijXmP7hi27I97v+Vw5RfcuMCbez/C k2Dq5cilZzHlUP8r+1iyxWAa96mO8esZ2irfLEv+6U2vo2nE5JAm5fPs1Ljdbp8NHAnt+Xlj e0a18Wth5tifUStC8d5pwVe80IZtjRNbF8v8DnmM8v+KiN9u4Xd9us/1RJrPbzMTBdS2rOgP v+U+Lu7Pjax854D36TNxhqXuS2an6L/r9hTUeH2/UzD/3nDAAHfB4w7F5TE0SRKBL67WSP4F G3zg0qIEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrMIsWRmVeSWpSXmKPExsWy7bCSvO4Vj4JUg+0nTC3WnzrGbPHx628W i6YJf5ktVt/tZ7N4ffgTo8WTA+2MFgsWzWWxuHlgJ5PFnkWTmCxWrj7KZDHp0DVGi6dXZzFZ 7L2lbbGwbQmLxZ69J1ksLu+aw2Yxf9lTdovu6zvYLJYf/8dkcWPCU0aLiR1XmSx2PGlktNj2 ez6zxbrX71ksTtyStjj/9zirg5THzll32T3O39vI4nH5bKnHplWdbB6bl9R7vNg8k9Fj980G No/FfZNZPXqb37F5fHx6i8Xj/b6rbB59W1Yxemw+Xe3xeZOcx6Ynb5kC+KO4bFJSczLLUov0 7RK4Mu4t6GQreMdbsfb3DcYGxgvcXYycHBICJhLr5/SwdTFycQgJ7GaU2Lv1NBtEQlyi+doP dghbWGLlv+dgtpDAR0aJUwukuxg5ONgENCUuTC4F6RUR2MEs8XNtMxOIwyywhlniytWZYIOE BYIk/h4+xwhiswioSmycdRlsEK+AhcTXttOMEAvkJWZe+g4W5xSwlJj8fBEzxDILiX2N01kg 6gUlTs58AmYzA9U3b53NPIFRYBaS1CwkqQWMTKsYJVMLinPTc4sNC4zyUsv1ihNzi0vz0vWS 83M3MYLjXUtrB+OeVR/0DjEycTAeYpTgYFYS4c05n58qxJuSWFmVWpQfX1Sak1p8iFGag0VJ nPfb694UIYH0xJLU7NTUgtQimCwTB6dUA5PUHM5wjdroWP0Gv1PnEqVn1Hp0BlZP1Tix3n5Z 8owX9tuc+Badc1snvnR79272JVZN4gVVt+Nt25pXpDpzJ2QFf1uWxtPkfJ/n6+si2ejuCXEX 98ovjvc7sGC/gR//l4XV069c+L171Tepqha5+l/fNx0LYPorL+Buf+p/2raHW6/IGLrknl5T 0l4nGLnnx8La9f8LZwlfnptSJPjuXavq7Kdv3+5cyt94uG7dQTGbkLaCpxf2yjEx3ApyCF13 eZt5j83SazKizuuO3+N45vX34DWhP3qlk7jffKpYbaq39vCd2gbvvT/uaCvGZBzN5urp9WJn X71uyzP+7OoVpfu3c5nZHblefOGWsKFb/2klluKMREMt5qLiRABeV9QYZgMAAA== X-CMS-MailID: 20231206101133epcas5p28efae88fbc435e0a42bc89c94eec3ac2 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101133epcas5p28efae88fbc435e0a42bc89c94eec3ac2 References: <20231206100253.13100-1-joshi.k@samsung.com> From: Anuj Gupta This is a prep patch. Allow copy_file_range to work for block devices. Relaxing generic_copy_file_checks allows us to reuse the existing infra, instead of adding a new user interface for block copy offload. Change generic_copy_file_checks to use ->f_mapping->host for both inode_in and inode_out. Allow block device in generic_file_rw_checks. Reviewed-by: Hannes Reinecke Signed-off-by: Anuj Gupta Signed-off-by: Nitesh Shetty --- fs/read_write.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/fs/read_write.c b/fs/read_write.c index e0c2c1b5962b..92729c7547d3 100644 --- a/fs/read_write.c +++ b/fs/read_write.c @@ -1441,8 +1441,8 @@ static int generic_copy_file_checks(struct file *file_in, loff_t pos_in, struct file *file_out, loff_t pos_out, size_t *req_count, unsigned int flags) { - struct inode *inode_in = file_inode(file_in); - struct inode *inode_out = file_inode(file_out); + struct inode *inode_in = file_in->f_mapping->host; + struct inode *inode_out = file_out->f_mapping->host; uint64_t count = *req_count; loff_t size_in; int ret; @@ -1750,7 +1750,9 @@ int generic_file_rw_checks(struct file *file_in, struct file *file_out) /* Don't copy dirs, pipes, sockets... */ if (S_ISDIR(inode_in->i_mode) || S_ISDIR(inode_out->i_mode)) return -EISDIR; - if (!S_ISREG(inode_in->i_mode) || !S_ISREG(inode_out->i_mode)) + if (!S_ISREG(inode_in->i_mode) && !S_ISBLK(inode_in->i_mode)) + return -EINVAL; + if ((inode_in->i_mode & S_IFMT) != (inode_out->i_mode & S_IFMT)) return -EINVAL; if (!(file_in->f_mode & FMODE_READ) || From patchwork Wed Dec 6 10:02:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 13481382 Received: from mailout1.samsung.com (mailout1.samsung.com [203.254.224.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 21C2F2D61B for ; Wed, 6 Dec 2023 10:11:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="cd1wParl" Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20231206101152epoutp011e31c7e13ff460b92873c7e6429acb3e~eNmP4Oyf40455004550epoutp01G for ; Wed, 6 Dec 2023 10:11:52 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20231206101152epoutp011e31c7e13ff460b92873c7e6429acb3e~eNmP4Oyf40455004550epoutp01G DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857512; bh=AOfFIjYI2wLqNYIL53RS+cLCGUBaMkDlBy8uIvV9EEU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cd1wParlIHUxsUbfkdHucKN5S4B108SkOdn2UE2QFoT5CT9a09J5xJIjTgbgMYLNr GB9uAp0SLhK6JPqS7QZJLg/pb3xUUY2nBJoUOm/6KxXmJT5Sa3lfEHj4R+WXWGWJoI 0X04DP+lwYEmhwwR1JG2g98PfxOo6mJC0L7NNfjE= Received: from epsnrtp1.localdomain (unknown [182.195.42.162]) by epcas5p1.samsung.com (KnoxPortal) with ESMTP id 20231206101151epcas5p13faeb3c54d5e6e95672f50b1041fd65d~eNmPQdroJ1468514685epcas5p1B; Wed, 6 Dec 2023 10:11:51 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.176]) by epsnrtp1.localdomain (Postfix) with ESMTP id 4SlY915zhlz4x9Pv; Wed, 6 Dec 2023 10:11:49 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 80.D6.10009.5E840756; Wed, 6 Dec 2023 19:11:49 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20231206101149epcas5p429e2a328af02819c71fa48ad01308a89~eNmNLRqTM2528425284epcas5p40; Wed, 6 Dec 2023 10:11:49 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20231206101149epsmtrp14047bbe569c539379849c452b0a67208~eNmNKMzPF1112611126epsmtrp1H; Wed, 6 Dec 2023 10:11:49 +0000 (GMT) X-AuditID: b6c32a4a-261fd70000002719-67-657048e52afe Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 89.52.08817.5E840756; Wed, 6 Dec 2023 19:11:49 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101141epsmtip2611ca7ae3531b6917df9b8dbc3213075~eNmF_pEh-1181011810epsmtip2G; Wed, 6 Dec 2023 10:11:41 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 06/12] fs, block: copy_file_range for def_blk_ops for direct block device Date: Wed, 6 Dec 2023 15:32:38 +0530 Message-Id: <20231206100253.13100-7-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Tf1CTdRzu+76v2zCWbwOPr+sHMEODDtgWG18UqDu13iuvSKs7oZrvsbdB G9vaD81+EKQCWvw+L5jyI0AtJAgGArIJQoBQAsGhAuEBskAoSIS4JKWNzfK/5/P9PM899zzf +3BwXhabz4lXGxidmlYJWOuJ823+WwNtlJYR/n3GH1V1d+Do9tIKgb7Iuoejc6OZLDTbtgDQ ZEsqQMUlBQQaamnEkKUkB0PfnWvHUE7rVYBsgyYMWYefQ9+klBHIYu0i0MCFUyxUdMbGRl9e a2Chs533MXQ9ywZQdtoghhomkwE6v1KEo8rZeQJdHn4C9d7rXPcin2o0jbKp3hvVBDVwxUjV lB9jUeayz6lpcz6gmoaSWFRpRu46Kv3wHIu6bRsmqPmLgywqo7YcUOafPqHu1DxN1Uz+gUVt iFaGxzG0nNH5MOpYjTxerYgQvLpXtkMmkQpFgaIwFCrwUdMJTIRg5+6owJfiVfZKBD4HaJXR /hRF6/WC4MhwncZoYHziNHpDhIDRylXaEG2Qnk7QG9WKIDVj2CYSCsUSO3G/Mu5E9i+YdmbD RxOtA3gSmHc/Dtw4kAyBtSbbuuNgPYdHNgGY8lsT4RwWAFy4aHFt/gKw8u4S+4GkLs/Kdi6s ACb3VGDO4Q6A1Uea7QOHwyL9YV+u0SHwJCtw2FgtcnBwshCH5olOzLHwIGVwdXUCd2CC9IO1 q8uEA3NJBK/MNRNON2+Y37+85uxGhsHcqRLcyXkcduVPrnFwO+dw3UncYQDJejdY3lXlEu+E fc0dmBN7wJnOWlcEPryVmeLCsbA/v8fFMcCblksu/AI82p2JO8Lg9jBVF4KdXo/B9JXJtYyQ 5MK0FJ6T7Qtv5Dh6dGAvOJ5X5sIUnM8vcPXzFYDT31eys4C36aEIpocimP53KwZ4OdjEaPUJ CkYv0YrVzMH/fjZWk1AD1k4j4JUGMD72Z1ArwDigFUAOLvDkqno1DI8rpw99zOg0Mp1Rxehb gcTecTbO3xirsd+W2iAThYQJQ6RSaUjY81KRwIs7e7RAziMVtIFRMoyW0T3QYRw3fhJWtN1/ KS3GN6eTM0ZbEy1vmJ6t2pLD8p3K2CTcU6kkxJHbgtzfrS+b4DZ5FtIJKlm0R30/VTJWGtb2 T9ojQk/3LYtVZtmJ7mVr3Y91qQt9978NZe6+6f7UgbmDqUXk67vxr6//sPlUu7/8mmKEV2Ns U1amh/56cl7djgqpIaVkxA/xn9nlfnof8U6A+GVpxYpO3Le93k/U5XVV5F382mdwbLjIMrK8 lLeYnOitvTSbXNo9xZh10++VLz5ZHzmzeqwjr+X9/s3my6P7RwvfCh435CXSn34IDv188+zv og/woF1bib2SWzGn3x40Rdn24JkNw6mPKsKj96lKeqQbxdNHdsQICH0cLQrAdXr6X6DehwWj BAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrMIsWRmVeSWpSXmKPExsWy7bCSvO5Tj4JUg92XuC3WnzrGbPHx628W i6YJf5ktVt/tZ7N4ffgTo8WTA+2MFgsWzWWxuHlgJ5PFnkWTmCxWrj7KZDHp0DVGi6dXZzFZ 7L2lbbGwbQmLxZ69J1ksLu+aw2Yxf9lTdovu6zvYLJYf/8dkcWPCU0aLiR1XmSx2PGlktNj2 ez6zxbrX71ksTtyStjj/9zirg5THzll32T3O39vI4nH5bKnHplWdbB6bl9R7vNg8k9Fj980G No/FfZNZPXqb37F5fHx6i8Xj/b6rbB59W1Yxemw+Xe3xeZOcx6Ynb5kC+KO4bFJSczLLUov0 7RK4MqZOvMhU8Iq/4tGhy8wNjO95uhg5OSQETCS2ztjL3sXIxSEksJtRon3CIjaIhLhE87Uf 7BC2sMTKf8+hij4ySvzc2cHYxcjBwSagKXFhcilIXERgB7PEz7XNTCANzAJrmCW2zOUFsYUF YiVO9r1mBbFZBFQltvz/zgJi8wpYSJx9t58FYoG8xMxL38GWcQpYSkx+vogZxBYCqtnXOB2q XlDi5MwnLBDz5SWat85mnsAoMAtJahaS1AJGplWMkqkFxbnpucWGBUZ5qeV6xYm5xaV56XrJ +bmbGMHxrqW1g3HPqg96hxiZOBgPMUpwMCuJ8Oacz08V4k1JrKxKLcqPLyrNSS0+xCjNwaIk zvvtdW+KkEB6YklqdmpqQWoRTJaJg1OqgclTvuZhtd+36MK4Yx08wR//CWft7HOoLxMyuXZ0 n63VJG2hzbLm6hPiVIsSKmx/cM3omPOFiSE29HqO3/KIecJskSu7X1jyCTQ65vT1nXhxeUZZ 6OaDO9TcYlvyy5Ztaz6YtvtbkMuq/T3Mtt9e2DfYLtaUNFu6RXHP7oDUORscJ8xev4XLOfzI Kcdw97z4hrxvD7O3CIXMr+lM9Hfclaj05I5Qy7YPAkV2v1YyLn17JVo6tiLndspbtitdjg8e 9LzTsy26GbN3q9Klme+t405e9VrpNut4FlOE9iuxmWJpjVsZXoacWKEifLu83WPR72TL0PB5 y5PzJ53837N63b/Fb660NpuxzDp8tc9cV4mlOCPRUIu5qDgRAAqih8RmAwAA X-CMS-MailID: 20231206101149epcas5p429e2a328af02819c71fa48ad01308a89 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101149epcas5p429e2a328af02819c71fa48ad01308a89 References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty For direct block device opened with O_DIRECT, use copy_file_range to issue device copy offload, or use generic_copy_file_range in case device copy offload capability is absent or the device files are not open with O_DIRECT. Reviewed-by: Hannes Reinecke Signed-off-by: Anuj Gupta Signed-off-by: Nitesh Shetty --- block/fops.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/block/fops.c b/block/fops.c index 0bdad1e8d514..08b652788e55 100644 --- a/block/fops.c +++ b/block/fops.c @@ -747,6 +747,30 @@ static ssize_t blkdev_read_iter(struct kiocb *iocb, struct iov_iter *to) return ret; } +static ssize_t blkdev_copy_file_range(struct file *file_in, loff_t pos_in, + struct file *file_out, loff_t pos_out, + size_t len, unsigned int flags) +{ + struct block_device *in_bdev = I_BDEV(bdev_file_inode(file_in)); + struct block_device *out_bdev = I_BDEV(bdev_file_inode(file_out)); + ssize_t copied = 0; + + if ((in_bdev == out_bdev) && bdev_max_copy_sectors(in_bdev) && + (file_in->f_iocb_flags & IOCB_DIRECT) && + (file_out->f_iocb_flags & IOCB_DIRECT)) { + copied = blkdev_copy_offload(in_bdev, pos_in, pos_out, len, + NULL, NULL, GFP_KERNEL); + if (copied < 0) + copied = 0; + } else { + copied = generic_copy_file_range(file_in, pos_in + copied, + file_out, pos_out + copied, + len - copied, flags); + } + + return copied; +} + #define BLKDEV_FALLOC_FL_SUPPORTED \ (FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE | \ FALLOC_FL_ZERO_RANGE | FALLOC_FL_NO_HIDE_STALE) @@ -851,6 +875,7 @@ const struct file_operations def_blk_fops = { .splice_read = filemap_splice_read, .splice_write = iter_file_splice_write, .fallocate = blkdev_fallocate, + .copy_file_range = blkdev_copy_file_range, }; static __init int blkdev_init(void) From patchwork Wed Dec 6 10:02:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 13481383 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 48DA72D78A for ; Wed, 6 Dec 2023 10:12:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="b1RAZUC8" Received: from epcas5p1.samsung.com (unknown [182.195.41.39]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20231206101210epoutp02cf78f28446a97d9e3a0131273b6e7c41~eNmhAy1Ct2240722407epoutp02s for ; Wed, 6 Dec 2023 10:12:10 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20231206101210epoutp02cf78f28446a97d9e3a0131273b6e7c41~eNmhAy1Ct2240722407epoutp02s DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857530; bh=fYNdJcNvOYM0NikPBiyxDw367vizzK5VGaV1CjKZzsI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=b1RAZUC8uO+8AzUpXOoPd6YsuspSU01uOumAgQ0kn+cHuuOcnOh+0osIYCLCGBAyP CPzQPUJjVVzYbTLI+b3H+LifExzahj74iTM8BeUOHarqmY/i1tBnpcjyWYxFAA24bX 4NyTrvg9VLU+aIXhe+1jV6F6syG1s1VcxGqQxeMM= Received: from epsnrtp3.localdomain (unknown [182.195.42.164]) by epcas5p4.samsung.com (KnoxPortal) with ESMTP id 20231206101209epcas5p4ba8098a79c211d36797ac54c3af58367~eNmga0QOd2550525505epcas5p4b; Wed, 6 Dec 2023 10:12:09 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.183]) by epsnrtp3.localdomain (Postfix) with ESMTP id 4SlY9L0zWRz4x9Pw; Wed, 6 Dec 2023 10:12:06 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 22.12.09672.5F840756; Wed, 6 Dec 2023 19:12:05 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20231206101205epcas5p2f156e36e10cb0a75d1752e5390c27021~eNmcYTQdD2757427574epcas5p2j; Wed, 6 Dec 2023 10:12:05 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20231206101205epsmtrp1abfb1a8c2c8cb5f73f8c5ab5ed8eed3a~eNmcWzAB31127111271epsmtrp1M; Wed, 6 Dec 2023 10:12:05 +0000 (GMT) X-AuditID: b6c32a4b-39fff700000025c8-3b-657048f5260d Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 2D.52.08817.5F840756; Wed, 6 Dec 2023 19:12:05 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101158epsmtip2827fe373811b8fab3d4dd523fa802b94~eNmVS4cJa1205412054epsmtip2K; Wed, 6 Dec 2023 10:11:57 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Kanchan Joshi , =?utf-8?q?Javier_Gonz=C3=A1lez?= , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 07/12] nvme: add copy offload support Date: Wed, 6 Dec 2023 15:32:39 +0530 Message-Id: <20231206100253.13100-8-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02Te1BUZRjG5zvn7NldZpZWoOGTHNlWs5S4rCz4IaAWDJ1Gm6isiElhgzNA sBf3oiVUFAELyMVlYtrlGqLIJRgu2iJgXCWgolghYZAEFi8QEBBBMmC77Fr+93vfed7vmef9 5mXhdlqmEytGoqTlElEcn7Qhrnbufd51hZLRHsVNvqi27waOFlfWCfRFzgaOqm5nk2i2cwkg Y1sqQCWlhQQaaWvCUEupBkMVVd0YmhpbZqLuR3Mk0nQMAzQ9pMNQ66gL+ialjEAtrb0EMlwr IFHxpWkmyvhNT6Lynk0M3cqZBui8eghDeuPnAF1dL8ZRzewCgX4YfQYNbPQwjuygmnS3mdTA eB1BGX5SUfWVaSTVUPYZdb9BC6jmkUSSupCVy6Ayk+ZJanF6lKAWrg+RVFZjJaAa+uOp5fqd VL1xDgt+KjTWL5oWRdJyHi2JkEbGSKL8+UffCgsI8/L2ELgKfNABPk8iEtP+/MBjwa5BMXGm 5fB5p0VxKlMrWKRQ8N0P+cmlKiXNi5YqlP58WhYZJxPK3BQisUIliXKT0MqDAg+P/V4mYXhs tLavipQZjn3Urz+HJwLDoXTAZkGuEKb/WsJIBzYsO24zgJtX1khLsQRgWVcDYSn+BrBQrSMf j7RvjFtHWgFMmjer2KZiGcDk1jPpgMUiuXvhL7kqc9uBW43DpjqBWY9zkwg4XJ219ZA91weW zdwFZia4z8G6la9wM3O4CM72P2BYzJyhdnCVaWa2SZ97r9Sq2QZ7tcYtX9ykSbqSj5sNIHec DQ3GNWAZDoQFminrQ/ZwpqeRaWEn+CA7xcoRcFD7M2ZhJZxqabfyYZjcl42bw+CmMLXX3C1e tjBz3YiZ25DLgeoUO4v6WTiumbY6OcKJr8usTMHfM82u5l2dA7C9O5WRA5x1T0TQPRFB979b CcArwXZaphBH0QovmaeEPvPfx0ZIxfVg60b2HdWDyTt/unUAjAU6AGThfAdO3ICUtuNEij4+ S8ulYXJVHK3oAF6mHZ/HnZ6OkJqOTKIMEwh9PITe3t5CH09vAd+RM5tcGGnHjRIp6VialtHy x3MYi+2UiBn9C34sp7xOqP65IPMZOmU79+47b55qq3V2Y7ZuL7qY212hj6nS57vrJl9TJ5xs PBz03VxD14eX99w8GZ3lcOPmvXhne5nab/hRYedx44n9b/9BGKI++KTlIp3kWcnIuGwDqgMC 2Lfii3if5l8XaXpeN347ET6ywLtftKe3Bfi+sm2XY+NqCCd8h1h4NvFIUGAIc/zlnvHTad2Y wwst32e8upnmq44QgDGNk0bNrEjd2Zw+svTliwe8EuYS7uaOBi7O6O7kvMfME+92GVsLq1qt 9rOtB6XzeV0uNcl98e+XlocSwS8V7rKdXLk0SB9nToQErryROhoi/iv0YZ4k82DbwxohyScU 0SLBPlyuEP0LZ37jYKwEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrCIsWRmVeSWpSXmKPExsWy7bCSvO5Xj4JUgx//JC3WnzrGbPHx628W i6YJf5ktVt/tZ7N4ffgTo8WTA+2MFgsWzWWxuHlgJ5PFnkWTmCxWrj7KZPH4zmd2i6P/37JZ TDp0jdHi6dVZTBZ7b2lbLGxbwmKxZ+9JFovLu+awWcxf9pTdovv6DjaL5cf/MVncmPCU0WJi x1Umix1PGhkttv2ez2yx7vV7FosTt6Qtzv89zuog47Fz1l12j/P3NrJ4XD5b6rFpVSebx+Yl 9R4vNs9k9Nh9s4HNY3HfZFaP3uZ3bB4fn95i8Xi/7yqbR9+WVYwem09Xe3zeJOex6clbpgD+ KC6blNSczLLUIn27BK6MmadWsxVc9qk4vaOHuYHxsl0XIyeHhICJxMG/91i7GLk4hAR2M0pc mH+HCSIhLtF87Qc7hC0ssfLfc3aIoo+MEt//tLF1MXJwsAloSlyYXAoSFxHYwSzxc20zE4jD LDCZRWLjw3OMIN3CApYSS149A7NZBFQlNn6dygxi8wpYSLw+/ZIVYoO8xMxL38G2cQLVT36+ CKxGCKhmX+N0Foh6QYmTM5+wgCxmFlCXWD9PCCTMDNTavHU28wRGwVlIqmYhVM1CUrWAkXkV o2RqQXFuem6xYYFRXmq5XnFibnFpXrpecn7uJkZwwtDS2sG4Z9UHvUOMTByMhxglOJiVRHhz zuenCvGmJFZWpRblxxeV5qQWH2KU5mBREuf99ro3RUggPbEkNTs1tSC1CCbLxMEp1cAk8aPA /NGmVqnL+r35TdLJzfVcH6MTOE9P+MTdZ7r2+9K27AWCOxhszBXnd5Van+ZXe3Klve5NwSVn ucdP4swzEmTeJees7XZtuPRctV89QevKLtnH/9+tVV6wOM86yrdy16uXRsIpivt0I2I02Mu+ WBn7fvBcJXWkc/XUsAlHp9Q9LNLMZuGaLyEdOiWvYHMhy9/7HsJqC7TS4rhCzT+9W51x/7GJ 9KaF+t9bT6VO3bPgX6/9Bw7l+bM9DjtfX79wHv+6aVzaYRP231j3u6bcwJmnak1x4umZC17r PG59rHugKV3HJG6qRpmeHoNGC+8ilZ9673xOh+9apmAw21fn8duIlvabr6zKW6Ym/1diKc5I NNRiLipOBADj7z7UhwMAAA== X-CMS-MailID: 20231206101205epcas5p2f156e36e10cb0a75d1752e5390c27021 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101205epcas5p2f156e36e10cb0a75d1752e5390c27021 References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty Current design only supports single source range. We receive a request with REQ_OP_COPY_SRC. Parse this request which consists of src(1st) and dst(2nd) bios. Form a copy command (TP 4065) trace event support for nvme_copy_cmd. Set the device copy limits to queue limits. Reviewed-by: Hannes Reinecke Signed-off-by: Kanchan Joshi Signed-off-by: Nitesh Shetty Signed-off-by: Javier González Signed-off-by: Anuj Gupta --- drivers/nvme/host/constants.c | 1 + drivers/nvme/host/core.c | 79 +++++++++++++++++++++++++++++++++++ drivers/nvme/host/trace.c | 19 +++++++++ include/linux/blkdev.h | 1 + include/linux/nvme.h | 43 +++++++++++++++++-- 5 files changed, 140 insertions(+), 3 deletions(-) diff --git a/drivers/nvme/host/constants.c b/drivers/nvme/host/constants.c index 20f46c230885..2f504a2b1fe8 100644 --- a/drivers/nvme/host/constants.c +++ b/drivers/nvme/host/constants.c @@ -19,6 +19,7 @@ static const char * const nvme_ops[] = { [nvme_cmd_resv_report] = "Reservation Report", [nvme_cmd_resv_acquire] = "Reservation Acquire", [nvme_cmd_resv_release] = "Reservation Release", + [nvme_cmd_copy] = "Copy Offload", [nvme_cmd_zone_mgmt_send] = "Zone Management Send", [nvme_cmd_zone_mgmt_recv] = "Zone Management Receive", [nvme_cmd_zone_append] = "Zone Append", diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 1be1ce522896..0ef218d5c2f5 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -763,6 +763,63 @@ static inline void nvme_setup_flush(struct nvme_ns *ns, cmnd->common.nsid = cpu_to_le32(ns->head->ns_id); } +static inline blk_status_t nvme_setup_copy_offload(struct nvme_ns *ns, + struct request *req, + struct nvme_command *cmnd) +{ + struct nvme_copy_range *range = NULL; + struct bio *bio; + u64 dst_lba = 0, src_lba = 0, n_lba = 0; + u16 nr_range = 1, control = 0, seg = 1; + + if (blk_rq_nr_phys_segments(req) != BLK_COPY_MAX_SEGMENTS) + return BLK_STS_IOERR; + + /* + * First bio contains information about source and last bio contains + * information about destination. + */ + __rq_for_each_bio(bio, req) { + if (seg == blk_rq_nr_phys_segments(req)) { + dst_lba = nvme_sect_to_lba(ns, bio->bi_iter.bi_sector); + if (n_lba != bio->bi_iter.bi_size >> ns->lba_shift) + return BLK_STS_IOERR; + } else { + src_lba = nvme_sect_to_lba(ns, bio->bi_iter.bi_sector); + n_lba = bio->bi_iter.bi_size >> ns->lba_shift; + } + seg++; + } + + if (req->cmd_flags & REQ_FUA) + control |= NVME_RW_FUA; + + if (req->cmd_flags & REQ_FAILFAST_DEV) + control |= NVME_RW_LR; + + memset(cmnd, 0, sizeof(*cmnd)); + cmnd->copy.opcode = nvme_cmd_copy; + cmnd->copy.nsid = cpu_to_le32(ns->head->ns_id); + cmnd->copy.control = cpu_to_le16(control); + cmnd->copy.sdlba = cpu_to_le64(dst_lba); + cmnd->copy.nr_range = 0; + + range = kmalloc_array(nr_range, sizeof(*range), + GFP_ATOMIC | __GFP_NOWARN); + if (!range) + return BLK_STS_RESOURCE; + + range[0].slba = cpu_to_le64(src_lba); + range[0].nlb = cpu_to_le16(n_lba - 1); + + req->special_vec.bv_page = virt_to_page(range); + req->special_vec.bv_offset = offset_in_page(range); + req->special_vec.bv_len = sizeof(*range) * nr_range; + req->rq_flags |= RQF_SPECIAL_PAYLOAD; + + return BLK_STS_OK; +} + static blk_status_t nvme_setup_discard(struct nvme_ns *ns, struct request *req, struct nvme_command *cmnd) { @@ -1005,6 +1062,11 @@ blk_status_t nvme_setup_cmd(struct nvme_ns *ns, struct request *req) case REQ_OP_ZONE_APPEND: ret = nvme_setup_rw(ns, req, cmd, nvme_cmd_zone_append); break; + case REQ_OP_COPY_SRC: + ret = nvme_setup_copy_offload(ns, req, cmd); + break; + case REQ_OP_COPY_DST: + return BLK_STS_IOERR; default: WARN_ON_ONCE(1); return BLK_STS_IOERR; @@ -1756,6 +1818,21 @@ static void nvme_config_discard(struct gendisk *disk, struct nvme_ns *ns) blk_queue_max_write_zeroes_sectors(queue, UINT_MAX); } +static void nvme_config_copy(struct gendisk *disk, struct nvme_ns *ns, + struct nvme_id_ns *id) +{ + struct nvme_ctrl *ctrl = ns->ctrl; + struct request_queue *q = disk->queue; + + if (!(ctrl->oncs & NVME_CTRL_ONCS_COPY)) { + blk_queue_max_copy_hw_sectors(q, 0); + return; + } + + blk_queue_max_copy_hw_sectors(q, nvme_lba_to_sect(ns, + le16_to_cpu(id->mssrl))); +} + static bool nvme_ns_ids_equal(struct nvme_ns_ids *a, struct nvme_ns_ids *b) { return uuid_equal(&a->uuid, &b->uuid) && @@ -1959,6 +2036,7 @@ static void nvme_update_disk_info(struct gendisk *disk, set_capacity_and_notify(disk, capacity); nvme_config_discard(disk, ns); + nvme_config_copy(disk, ns, id); blk_queue_max_write_zeroes_sectors(disk->queue, ns->ctrl->max_zeroes_sectors); } @@ -4696,6 +4774,7 @@ static inline void _nvme_check_size(void) BUILD_BUG_ON(sizeof(struct nvme_download_firmware) != 64); BUILD_BUG_ON(sizeof(struct nvme_format_cmd) != 64); BUILD_BUG_ON(sizeof(struct nvme_dsm_cmd) != 64); + BUILD_BUG_ON(sizeof(struct nvme_copy_command) != 64); BUILD_BUG_ON(sizeof(struct nvme_write_zeroes_cmd) != 64); BUILD_BUG_ON(sizeof(struct nvme_abort_cmd) != 64); BUILD_BUG_ON(sizeof(struct nvme_get_log_page_command) != 64); diff --git a/drivers/nvme/host/trace.c b/drivers/nvme/host/trace.c index 1c36fcedea20..82c6aef77c31 100644 --- a/drivers/nvme/host/trace.c +++ b/drivers/nvme/host/trace.c @@ -150,6 +150,23 @@ static const char *nvme_trace_read_write(struct trace_seq *p, u8 *cdw10) return ret; } +static const char *nvme_trace_copy(struct trace_seq *p, u8 *cdw10) +{ + const char *ret = trace_seq_buffer_ptr(p); + u64 sdlba = get_unaligned_le64(cdw10); + u8 nr_range = get_unaligned_le16(cdw10 + 8); + u16 control = get_unaligned_le16(cdw10 + 10); + u32 dsmgmt = get_unaligned_le32(cdw10 + 12); + u32 reftag = get_unaligned_le32(cdw10 + 16); + + trace_seq_printf(p, + "sdlba=%llu, nr_range=%u, ctrl=0x%x, dsmgmt=%u, reftag=%u", + sdlba, nr_range, control, dsmgmt, reftag); + trace_seq_putc(p, 0); + + return ret; +} + static const char *nvme_trace_dsm(struct trace_seq *p, u8 *cdw10) { const char *ret = trace_seq_buffer_ptr(p); @@ -243,6 +260,8 @@ const char *nvme_trace_parse_nvm_cmd(struct trace_seq *p, return nvme_trace_zone_mgmt_send(p, cdw10); case nvme_cmd_zone_mgmt_recv: return nvme_trace_zone_mgmt_recv(p, cdw10); + case nvme_cmd_copy: + return nvme_trace_copy(p, cdw10); default: return nvme_trace_common(p, cdw10); } diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index 9fa1ad68beb5..25a71944c5cb 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -1226,6 +1226,7 @@ static inline unsigned int bdev_discard_granularity(struct block_device *bdev) /* maximum copy offload length, this is set to 128MB based on current testing */ #define BLK_COPY_MAX_BYTES (1 << 27) +#define BLK_COPY_MAX_SEGMENTS 2 static inline unsigned int bdev_max_copy_sectors(struct block_device *bdev) { diff --git a/include/linux/nvme.h b/include/linux/nvme.h index 44325c068b6a..4afd2cd1a629 100644 --- a/include/linux/nvme.h +++ b/include/linux/nvme.h @@ -344,7 +344,7 @@ struct nvme_id_ctrl { __u8 nvscc; __u8 nwpc; __le16 acwu; - __u8 rsvd534[2]; + __le16 ocfs; __le32 sgls; __le32 mnan; __u8 rsvd544[224]; @@ -372,6 +372,7 @@ enum { NVME_CTRL_ONCS_WRITE_ZEROES = 1 << 3, NVME_CTRL_ONCS_RESERVATIONS = 1 << 5, NVME_CTRL_ONCS_TIMESTAMP = 1 << 6, + NVME_CTRL_ONCS_COPY = 1 << 8, NVME_CTRL_VWC_PRESENT = 1 << 0, NVME_CTRL_OACS_SEC_SUPP = 1 << 0, NVME_CTRL_OACS_NS_MNGT_SUPP = 1 << 3, @@ -421,7 +422,10 @@ struct nvme_id_ns { __le16 npdg; __le16 npda; __le16 nows; - __u8 rsvd74[18]; + __le16 mssrl; + __le32 mcl; + __u8 msrc; + __u8 rsvd91[11]; __le32 anagrpid; __u8 rsvd96[3]; __u8 nsattr; @@ -838,6 +842,7 @@ enum nvme_opcode { nvme_cmd_resv_report = 0x0e, nvme_cmd_resv_acquire = 0x11, nvme_cmd_resv_release = 0x15, + nvme_cmd_copy = 0x19, nvme_cmd_zone_mgmt_send = 0x79, nvme_cmd_zone_mgmt_recv = 0x7a, nvme_cmd_zone_append = 0x7d, @@ -861,7 +866,8 @@ enum nvme_opcode { nvme_opcode_name(nvme_cmd_resv_release), \ nvme_opcode_name(nvme_cmd_zone_mgmt_send), \ nvme_opcode_name(nvme_cmd_zone_mgmt_recv), \ - nvme_opcode_name(nvme_cmd_zone_append)) + nvme_opcode_name(nvme_cmd_zone_append), \ + nvme_opcode_name(nvme_cmd_copy)) @@ -1038,6 +1044,36 @@ struct nvme_dsm_range { __le64 slba; }; +struct nvme_copy_command { + __u8 opcode; + __u8 flags; + __u16 command_id; + __le32 nsid; + __u64 rsvd2; + __le64 metadata; + union nvme_data_ptr dptr; + __le64 sdlba; + __u8 nr_range; + __u8 rsvd12; + __le16 control; + __le16 rsvd13; + __le16 dspec; + __le32 ilbrt; + __le16 lbat; + __le16 lbatm; +}; + +struct nvme_copy_range { + __le64 rsvd0; + __le64 slba; + __le16 nlb; + __le16 rsvd18; + __le32 rsvd20; + __le32 eilbrt; + __le16 elbat; + __le16 elbatm; +}; + struct nvme_write_zeroes_cmd { __u8 opcode; __u8 flags; @@ -1802,6 +1838,7 @@ struct nvme_command { struct nvme_download_firmware dlfw; struct nvme_format_cmd format; struct nvme_dsm_cmd dsm; + struct nvme_copy_command copy; struct nvme_write_zeroes_cmd write_zeroes; struct nvme_zone_mgmt_send_cmd zms; struct nvme_zone_mgmt_recv_cmd zmr; From patchwork Wed Dec 6 10:02:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 13481384 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 400EB2D02B for ; Wed, 6 Dec 2023 10:12:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="L6bKngid" Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20231206101221epoutp02a1b2532f80e0128647dd21760a745750~eNmrIbO2n2304023040epoutp02g for ; Wed, 6 Dec 2023 10:12:21 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20231206101221epoutp02a1b2532f80e0128647dd21760a745750~eNmrIbO2n2304023040epoutp02g DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857541; bh=uOAKePf/EC3LoZnFEX09yOYQgAaLwiBeLuJB0GWSSoA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=L6bKngidO1Tuve/sM0yGzW4aHSvJ4V8zR65igPFyX04th1v/DVpinxheb80zFGx1q +LLk5iRa+j2rigQFDStB1DggeOGZAxb6LtK64w2XM+zcsHq2GIc3lCmb2B4lyuTDnI pfm9Y0ynPIF1LRf/a3jXIHYchL9PLVnfzb0l5iwc= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20231206101220epcas5p35bbaf46727278f8d48a5fe4d6be0f5c4~eNmqZcRLT0789207892epcas5p3j; Wed, 6 Dec 2023 10:12:20 +0000 (GMT) Received: from epsmges5p2new.samsung.com (unknown [182.195.38.177]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4SlY9b22qSz4x9Pw; Wed, 6 Dec 2023 10:12:19 +0000 (GMT) Received: from epcas5p1.samsung.com ( [182.195.41.39]) by epsmges5p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 7E.D6.10009.30940756; Wed, 6 Dec 2023 19:12:19 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20231206101218epcas5p3bd5005a84adf67a80d394c3e05796bd7~eNmomUm130791407914epcas5p3g; Wed, 6 Dec 2023 10:12:18 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20231206101218epsmtrp217a2e5b0a9cb015fb653b55a7c6ddaed~eNmolF_vN1007210072epsmtrp2O; Wed, 6 Dec 2023 10:12:18 +0000 (GMT) X-AuditID: b6c32a4a-ff1ff70000002719-aa-6570490359be Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 80.62.08817.20940756; Wed, 6 Dec 2023 19:12:18 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101214epsmtip248d44b17f67df20f2c2e9a1abb6e15b2~eNmkNOL7S1181011810epsmtip2L; Wed, 6 Dec 2023 10:12:13 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 08/12] nvmet: add copy command support for bdev and file ns Date: Wed, 6 Dec 2023 15:32:40 +0530 Message-Id: <20231206100253.13100-9-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02TeVBTVxTG57738pLQiU2DHW/jTAmvwzBgWWIhvSh0UcY+u4HTzbbTgUje AJKtWYq2MqKMIBjZUh1Jla1oKahUAkyQTaHsBSyILC1M0aRQEAFRawWkCYmt//3OPd83Z75z 53BwQRZbyIlX6hiNUiqnSDeipsXH2w/fqWYCr8+xUEVXG44W7i8R6HD2Co7Kx7JINNNyFyDr lTSACovPEGjkSi2G6otzMfRjeSuGcptvAGQbNGGoYXQTKkotIVB9QyeBBi6fJlHBORsbHRuy kOiH9scYGs62AZRzdBBDFushgGqWCnB0cWaOQB2jG1HfSjvrDSFdaxpj033jlwh6oEdPV5al k7S55CA9Zc4DdN1IMkl/n2lk0cdT7pD0gm2UoOcaB0k6s6oM0Obub+jFyhfpSussFvnsZwmh cYxUxmhEjDJGJYtXxoZR73wQtT0qWBIo9hOHoFcpkVKqYMKo8Hcj/XbEy+0roURfSeV6+1Ok VKulAl4L1aj0OkYUp9LqwihGLZOrg9T+WqlCq1fG+isZ3RZxYODmYLswOiFu6oSJpV55a1/3 xWZWMhjZmgG4HMgPgv+cbmNnADeOgF8H4NDtacJZ3AXw0MlO0lk8APDX9A78iWUy/RzubDQA +KjjKHAWiwC2ZUzaLRwOyfeB14x6h2E9/zwOay+JHRqcn49D8812zNFw5++C5ZnGNSb4XnBo Nm1tAo+P4NipXtc0D5jX/zfbwVx+CDROFrs0z8HOPCvhYNyuSan+zqU3c6HFFu3kcNhbXc92 sjucbq9ysRAu3mkgnRwD+/N6MSfr4K36qy5+HR7pysIdWXB7lorLAc5R6+DxJSvmeIZ8Hjya KnCqPeF4ro3l5A1w4lSJi2m4kFvi2pUBwIdNtXg28DA9lcD0VALT/9MKAV4GXmDUWkUsow1W b1Yyif99bIxKUQnWLsP3bQuY+GPevxlgHNAMIAen1vPkfSpGwJNJ93/NaFRRGr2c0TaDYPuK c3Dh8zEq+2kpdVHioJDAIIlEEhTyikRMbeDNHDkjE/BjpTomgWHUjOaJD+NwhckY+3NbOHe5 2hIqyxhpOtByU8UpldvCuCM+6WRFkpsgvWhf4liKovUjr9/H0wL0ewr0XfNJ7z3w33r4ZGPh 2Y2d+lRvQ+Evnn37/bv2nk8ZBDI3fdJP5ZI9J7wOrOK2VTKfKBpdKfRsXfx5Sw2W3ZdIf/Ib DB/YPvzho2DR3r/4UztFEfcUuzzyL4BYWjz+6b0bMyaN+ury/Lbll67tPthIVmWLjvlGZO0W eVPRPfc1mjr3WWvPt13W5QvPrGuiAtyn/+xcNrxf2i8c8lr98vrZiNtk/sOZ9oQJcwFza/7l KKOl0WZo25HzhXEuQrXNQL8ZU5on6daG+RQYHidu+piXOEwR2jip2BfXaKX/AvwJ/J6iBAAA X-Brightmail-Tracker: H4sIAAAAAAAAA02Ra0hTYRjHe895d3ZmnTytsjdXSetCTfJG0RtdIcMTREqRRAS58jStTdeW VlZkjm4zTVtRrtRaZWZWOC9Z04jZbZqaiZepC5ItV2Jq0Q2dlY6gbz/+v//zPB8emhQ3Q386 PmE/r0mQK6WUD6yokQYsItar+ZBqx2z8oPYFiQe/DUGcluUh8V3HOQr31nwB2Pn0FMDXTLkQ 258+InCV6TyB79x9TuDz1laAXS1GAld3BOLrJ29CXFVtg7j58VUK5xe4hDi9rZLCt1+OELg9 ywVw9ukWAlc6jwNcMZRP4vu9/RC/6pDgRs9LwRp/7pHRIeQa35VArrk+iTMXnaG40pvHOHdp DuAs9lSKu5FpEHAZus8UN+jqgFz/kxaKyywrAlxp3WHuq3kWZ3b2EVG+23xWxPLK+GReE7wq xifOfdEoUHsiDtbdtwpSgX25HohoxC5GPWcKSD3wocWsBaDu4WHgFdOQrvWn0MuT0Z2RHqG3 NAiQ3lgL9YCmKXYhemNIGs2nsJUk+nVPR4wOkGwxicpymVGezEaivlsl5ChDdh5q6zs1xgyL keNyA+k9EIBy3v4YOyZilyFDj2ksF//tPDl+CXr7k5Atxwm9+wOQrvwKmQVY43/K+J+6Bogi MJ1Xa1UKlTZUHZbAHwjSylXapARF0K5ElRmM/V0mqwRVRQNBVkDQwAoQTUqnMMrGRF7MxMoP pfCaxB2aJCWvtQIJDaXTmO+9GbFiViHfz+/leTWv+WcJWuSfSqgF4rpXaPxKOniiIvWL343n TaLVSt2m4hhT4+uS3p65Z9MeZttTVq3rCmeEfsPPGGk9nufosxwoZiw1uzdKLLnEjKjuH7kb DB/7yzPTrOajwU6bUJU3VDgQUZZyjlo5J1LmiW3VJrc3bU0u6LKta+iw7YjrHNpOLI9ksvvj 00LCfX3XaNySPV0lnR8UBk+0M15XvoCE/nlnjcMZM/XQLXEfUiqiVm/eNS4mLzoqzDVVtv3W 6++7j+wrLgs/8T5wrshlbvpUG1gxNfDDQL3kmH2prLMzfUEYSeJvDTFBS9q5nW9Me+enTJ/Q nY7rA7pCfx9Zq3OtHWm7EPH49pZCKdTGyUNlpEYr/wMbdITmZgMAAA== X-CMS-MailID: 20231206101218epcas5p3bd5005a84adf67a80d394c3e05796bd7 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101218epcas5p3bd5005a84adf67a80d394c3e05796bd7 References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty Add support for handling nvme_cmd_copy command on target. For bdev-ns if backing device supports copy offload we call device copy offload (blkdev_copy_offload). In case of absence of device copy offload capability, we use copy emulation (blkdev_copy_emulation) For file-ns we call vfs_copy_file_range to service our request. Currently target always shows copy capability by setting NVME_CTRL_ONCS_COPY in controller ONCS. loop target has copy support, which can be used to test copy offload. trace event support for nvme_cmd_copy. Reviewed-by: Hannes Reinecke Signed-off-by: Nitesh Shetty Signed-off-by: Anuj Gupta --- drivers/nvme/target/admin-cmd.c | 9 +++- drivers/nvme/target/io-cmd-bdev.c | 71 +++++++++++++++++++++++++++++++ drivers/nvme/target/io-cmd-file.c | 50 ++++++++++++++++++++++ drivers/nvme/target/nvmet.h | 1 + drivers/nvme/target/trace.c | 19 +++++++++ 5 files changed, 148 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c index 39cb570f833d..4e1a6ca09937 100644 --- a/drivers/nvme/target/admin-cmd.c +++ b/drivers/nvme/target/admin-cmd.c @@ -433,8 +433,7 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req) id->nn = cpu_to_le32(NVMET_MAX_NAMESPACES); id->mnan = cpu_to_le32(NVMET_MAX_NAMESPACES); id->oncs = cpu_to_le16(NVME_CTRL_ONCS_DSM | - NVME_CTRL_ONCS_WRITE_ZEROES); - + NVME_CTRL_ONCS_WRITE_ZEROES | NVME_CTRL_ONCS_COPY); /* XXX: don't report vwc if the underlying device is write through */ id->vwc = NVME_CTRL_VWC_PRESENT; @@ -536,6 +535,12 @@ static void nvmet_execute_identify_ns(struct nvmet_req *req) if (req->ns->bdev) nvmet_bdev_set_limits(req->ns->bdev, id); + else { + id->msrc = (__force u8)to0based(BIO_MAX_VECS - 1); + id->mssrl = cpu_to_le16(BIO_MAX_VECS << + (PAGE_SHIFT - SECTOR_SHIFT)); + id->mcl = cpu_to_le32(le16_to_cpu(id->mssrl)); + } /* * We just provide a single LBA format that matches what the diff --git a/drivers/nvme/target/io-cmd-bdev.c b/drivers/nvme/target/io-cmd-bdev.c index f11400a908f2..f974858ae5a0 100644 --- a/drivers/nvme/target/io-cmd-bdev.c +++ b/drivers/nvme/target/io-cmd-bdev.c @@ -46,6 +46,18 @@ void nvmet_bdev_set_limits(struct block_device *bdev, struct nvme_id_ns *id) id->npda = id->npdg; /* NOWS = Namespace Optimal Write Size */ id->nows = to0based(bdev_io_opt(bdev) / bdev_logical_block_size(bdev)); + + if (bdev_max_copy_sectors(bdev)) { + id->msrc = id->msrc; + id->mssrl = cpu_to_le16((bdev_max_copy_sectors(bdev) << + SECTOR_SHIFT) / bdev_logical_block_size(bdev)); + id->mcl = cpu_to_le32((__force u32)id->mssrl); + } else { + id->msrc = (__force u8)to0based(BIO_MAX_VECS - 1); + id->mssrl = cpu_to_le16((BIO_MAX_VECS << PAGE_SHIFT) / + bdev_logical_block_size(bdev)); + id->mcl = cpu_to_le32((__force u32)id->mssrl); + } } void nvmet_bdev_ns_disable(struct nvmet_ns *ns) @@ -451,6 +463,61 @@ static void nvmet_bdev_execute_write_zeroes(struct nvmet_req *req) } } +static void nvmet_bdev_copy_endio(void *private, int status, + ssize_t copied) +{ + struct nvmet_req *rq = (struct nvmet_req *)private; + u16 nvme_status; + + if (copied == rq->copy_len) + rq->cqe->result.u32 = cpu_to_le32(1); + else + rq->cqe->result.u32 = cpu_to_le32(0); + + nvme_status = errno_to_nvme_status(rq, status); + nvmet_req_complete(rq, nvme_status); +} + +/* + * At present we handle only one range entry, since copy offload is aligned with + * copy_file_range, only one entry is passed from block layer. + */ +static void nvmet_bdev_execute_copy(struct nvmet_req *rq) +{ + struct nvme_copy_range range; + struct nvme_command *cmd = rq->cmd; + ssize_t ret; + off_t dst, src; + + u16 status; + + status = nvmet_copy_from_sgl(rq, 0, &range, sizeof(range)); + if (status) + goto err_rq_complete; + + dst = le64_to_cpu(cmd->copy.sdlba) << rq->ns->blksize_shift; + src = le64_to_cpu(range.slba) << rq->ns->blksize_shift; + rq->copy_len = ((__force size_t)range.nlb + 1) << rq->ns->blksize_shift; + + if (bdev_max_copy_sectors(rq->ns->bdev)) { + ret = blkdev_copy_offload(rq->ns->bdev, dst, src, rq->copy_len, + nvmet_bdev_copy_endio, + (void *)rq, GFP_KERNEL); + } else { + ret = blkdev_copy_emulation(rq->ns->bdev, dst, + rq->ns->bdev, src, rq->copy_len, + nvmet_bdev_copy_endio, + (void *)rq, GFP_KERNEL); + } + if (ret == -EIOCBQUEUED) + return; + + rq->cqe->result.u32 = cpu_to_le32(0); + status = errno_to_nvme_status(rq, ret); +err_rq_complete: + nvmet_req_complete(rq, status); +} + u16 nvmet_bdev_parse_io_cmd(struct nvmet_req *req) { switch (req->cmd->common.opcode) { @@ -469,6 +536,10 @@ u16 nvmet_bdev_parse_io_cmd(struct nvmet_req *req) case nvme_cmd_write_zeroes: req->execute = nvmet_bdev_execute_write_zeroes; return 0; + case nvme_cmd_copy: + req->execute = nvmet_bdev_execute_copy; + return 0; + default: return nvmet_report_invalid_opcode(req); } diff --git a/drivers/nvme/target/io-cmd-file.c b/drivers/nvme/target/io-cmd-file.c index 2d068439b129..0a8337596f0c 100644 --- a/drivers/nvme/target/io-cmd-file.c +++ b/drivers/nvme/target/io-cmd-file.c @@ -322,6 +322,47 @@ static void nvmet_file_dsm_work(struct work_struct *w) } } +static void nvmet_file_copy_work(struct work_struct *w) +{ + struct nvmet_req *req = container_of(w, struct nvmet_req, f.work); + int nr_range = req->cmd->copy.nr_range + 1; + u16 status = 0; + int src, id; + ssize_t len, ret; + loff_t pos; + + pos = le64_to_cpu(req->cmd->copy.sdlba) << req->ns->blksize_shift; + if (unlikely(pos + req->transfer_len > req->ns->size)) { + nvmet_req_complete(req, errno_to_nvme_status(req, -ENOSPC)); + return; + } + + for (id = 0 ; id < nr_range; id++) { + struct nvme_copy_range range; + + status = nvmet_copy_from_sgl(req, id * sizeof(range), &range, + sizeof(range)); + if (status) + break; + + src = (le64_to_cpu(range.slba) << (req->ns->blksize_shift)); + len = (le16_to_cpu(range.nlb) + 1) << (req->ns->blksize_shift); + ret = vfs_copy_file_range(req->ns->file, src, req->ns->file, + pos, len, COPY_FILE_SPLICE); + pos += ret; + if (ret != len) { + req->cqe->result.u32 = cpu_to_le32(id); + if (ret < 0) + status = errno_to_nvme_status(req, ret); + else + status = errno_to_nvme_status(req, -EIO); + break; + } + } + + nvmet_req_complete(req, status); +} + static void nvmet_file_execute_dsm(struct nvmet_req *req) { if (!nvmet_check_data_len_lte(req, nvmet_dsm_len(req))) @@ -330,6 +371,12 @@ static void nvmet_file_execute_dsm(struct nvmet_req *req) queue_work(nvmet_wq, &req->f.work); } +static void nvmet_file_execute_copy(struct nvmet_req *req) +{ + INIT_WORK(&req->f.work, nvmet_file_copy_work); + queue_work(nvmet_wq, &req->f.work); +} + static void nvmet_file_write_zeroes_work(struct work_struct *w) { struct nvmet_req *req = container_of(w, struct nvmet_req, f.work); @@ -376,6 +423,9 @@ u16 nvmet_file_parse_io_cmd(struct nvmet_req *req) case nvme_cmd_write_zeroes: req->execute = nvmet_file_execute_write_zeroes; return 0; + case nvme_cmd_copy: + req->execute = nvmet_file_execute_copy; + return 0; default: return nvmet_report_invalid_opcode(req); } diff --git a/drivers/nvme/target/nvmet.h b/drivers/nvme/target/nvmet.h index 6c8acebe1a1a..b648baeb52cf 100644 --- a/drivers/nvme/target/nvmet.h +++ b/drivers/nvme/target/nvmet.h @@ -405,6 +405,7 @@ struct nvmet_req { struct device *p2p_client; u16 error_loc; u64 error_slba; + size_t copy_len; }; #define NVMET_MAX_MPOOL_BVEC 16 diff --git a/drivers/nvme/target/trace.c b/drivers/nvme/target/trace.c index bff454d46255..551fdf029381 100644 --- a/drivers/nvme/target/trace.c +++ b/drivers/nvme/target/trace.c @@ -92,6 +92,23 @@ static const char *nvmet_trace_dsm(struct trace_seq *p, u8 *cdw10) return ret; } +static const char *nvmet_trace_copy(struct trace_seq *p, u8 *cdw10) +{ + const char *ret = trace_seq_buffer_ptr(p); + u64 sdlba = get_unaligned_le64(cdw10); + u8 nr_range = get_unaligned_le16(cdw10 + 8); + u16 control = get_unaligned_le16(cdw10 + 10); + u32 dsmgmt = get_unaligned_le32(cdw10 + 12); + u32 reftag = get_unaligned_le32(cdw10 + 16); + + trace_seq_printf(p, + "sdlba=%llu, nr_range=%u, ctrl=1x%x, dsmgmt=%u, reftag=%u", + sdlba, nr_range, control, dsmgmt, reftag); + trace_seq_putc(p, 0); + + return ret; +} + static const char *nvmet_trace_common(struct trace_seq *p, u8 *cdw10) { const char *ret = trace_seq_buffer_ptr(p); @@ -129,6 +146,8 @@ const char *nvmet_trace_parse_nvm_cmd(struct trace_seq *p, return nvmet_trace_read_write(p, cdw10); case nvme_cmd_dsm: return nvmet_trace_dsm(p, cdw10); + case nvme_cmd_copy: + return nvmet_trace_copy(p, cdw10); default: return nvmet_trace_common(p, cdw10); } From patchwork Wed Dec 6 10:02:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 13481385 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4FF502D7A9 for ; Wed, 6 Dec 2023 10:12:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="KqByg77n" Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20231206101234epoutp029bf29e7d8c8832806d201ff710e0e3eb~eNm3YiWIU2240722407epoutp025 for ; Wed, 6 Dec 2023 10:12:34 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20231206101234epoutp029bf29e7d8c8832806d201ff710e0e3eb~eNm3YiWIU2240722407epoutp025 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857554; bh=vUmnCkDPVKAomoDbL/LzqNJSOMEo+hX7Rt95xB1wr2E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KqByg77ns7Ftfh/iSyw1FE0X9KoX53pCfRlXGZizpC8WuakukSG3cCRs71EhTTcv4 SWM5XNQf+5hnr/Lf412nWG1WPFyqucYycg4t2oJARPTql2f8oS1CxsiIBnwC4SMK46 pvmHRQgZkxpurBGtzmew/ThtntxrYhtLy85homjU= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20231206101234epcas5p3d2021e6e34c58b154e7090c86f997ba8~eNm20kUqr0791407914epcas5p37; Wed, 6 Dec 2023 10:12:34 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.177]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4SlY9r0DV3z4x9Pw; Wed, 6 Dec 2023 10:12:32 +0000 (GMT) Received: from epcas5p2.samsung.com ( [182.195.41.40]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 64.22.09672.F0940756; Wed, 6 Dec 2023 19:12:31 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p1.samsung.com (KnoxPortal) with ESMTPA id 20231206101231epcas5p1756d9f527df621f3a0eebb37ca257f06~eNm0Wee4t1084010840epcas5p1B; Wed, 6 Dec 2023 10:12:31 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20231206101231epsmtrp241901a3d3967ff948874ef6197994654~eNm0UoYOX1007210072epsmtrp2W; Wed, 6 Dec 2023 10:12:31 +0000 (GMT) X-AuditID: b6c32a4b-60bfd700000025c8-7c-6570490f4a3d Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id E2.62.08817.F0940756; Wed, 6 Dec 2023 19:12:31 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101227epsmtip2deeb866484ba7d4dc0142cdd65304b11~eNmwjZbBJ0221302213epsmtip2H; Wed, 6 Dec 2023 10:12:27 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 09/12] dm: Add support for copy offload Date: Wed, 6 Dec 2023 15:32:41 +0530 Message-Id: <20231206100253.13100-10-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01TbVBUVRju3Hv37i4TuC0yHMmU7tSoyNfGQmdBigrzpuUw9Kf8sy67dxaG ZXfdj0SdpkXiQxCWDzU+Q4ii2IwPNwKElKUgmACV4dNkCHcHFWQRrFDEaZeL5b/nec/znud5 3zOHhwufkH68JLWB0allKor0IFq6d+0M2vSelgm9VhWOGvp7cHT/r1UCWW6aSTTXvQSQ/UoW QOdrKgk0caUNQ99ZfsVQkW0UIMdIGYY6J3ej6sxaAnV09hFouL2CRFXfOLgod6yVRHW9TzA0 XuAAqDB7BEOt9jSAWlarcPTDnJNAv02+iIbWejkxkG4ru8mlh6aaCHp4wEg3158i6Yu1n9G3 L5YC+tKEiaS/yi/m0HnpCyR93zFJ0M6fR0g631oP6OXmbXSz/R4W53UoeU8iI1MwOn9GLdco ktTKaOrAh9J3pOERoaIgkQS9TvmrZSlMNBX7flzQu0kq1xIo/09kKqOrFCfT66mQN/boNEYD 45+o0RuiKUarUGnF2mC9LEVvVCuD1YwhUhQa+lq4S3g4OdE5/xDTLm9JnR+14iaQ75MD+Dwo EMPefjsnB3jwhIJLAHZkP9ggSwAuZjwlfwNosn6N5QDeestcD4+tdwJ4tbqRZMkygNduD+Nu ESnYBa8WG90WmwXf47CtSeTW4IIJDA5cbsHcB96CKNiyWslxY0LwKlw41wTc2FMggYXtWRw2 33ZYev0frhvzXfXi2Rqc1bwA+0rthBvjLk36j+W42wAKqvhwotvkSsR1kVi4upW9xhve7bVy WewHlxc6SRbL4fXSQYzFBniro2sDvwkz+s3ro+CuURraQ1gnL5i3at9YgyfMzhSy6pfhVJFj I7Av/LOklsNKaLjwmGCXcxrAP9JzOQVge9kz+cueyV/2v9l5gNeDLYxWn6Jk9OHaMDVz9L9X lWtSmsH6Rwg40ApmpheDbQDjARuAPJza7Kka0jBCT4Xs2HFGp5HqjCpGbwPhrgUX4n4+co3r J6kNUpFYEiqOiIgQS8IiRJSv51xGpUIoUMoMTDLDaBnd0z6Mx/czYXv5SueoPKZqd8XknG8x CDz4S+anrV++1FC3zZFw79uwG6VYtU/0prQjR3Pp46rCBMlBXJNv3hnIv2U9C617VwTc+Le5 UadOWsxa7zOKux/sON0DlF5eX4yM5ZdXZK08H9l0OXLpcOOgcX487Mi5qLaf7iTPlsys9c9X WgYyD0nX1FOD6TMJZo059ff2xzMnHkxbStaQx2id7Tme30mhKquwe1/MfOxbxwJvcMWSHWOz H0u7Mpu7puXxn1vM07opm3O8vMB3mVpcyl15tL/kbElevamve98Fvrboo6hiWrz/kVoZ0hgQ 5Qx7mHIirYYima1IbrlQYdaeyb3zSkNqPKIIfaJMFIDr9LJ/AeggkU+RBAAA X-Brightmail-Tracker: H4sIAAAAAAAAA02Ra0xSYRjHe885HA8QdcSar7Wy2LrpsKvtray1anX6UGvVp2pLyjNjAdJB LOuDt2VmQyW2mnZBzSxx1SR1qIGKF7IySlKTZbdJuFGQVpZZluja+vZ7/pc9z/ZQuLiVmEPJ Vcksp5IpJKSAqG2RREpn7lSzKxr1QnTvUTuOhr6NEaiyP59EvpZhgAaazgFUXHqNQH1NdRiq qGzD0EV7D0Ce7iIMWd3RqCS7jEAPrB0EctVfJZGx3BOCLvRaSHTLMY6hlwUegPQ53RiyDGQA VDtmxNFdX4BAD91zkfO3g7cZMnVF/SGM83UVwbg6tYzZdJ5k7pelMYP3CwHT0JdOMjfyDDxG l+UnmSGPm2ACtm6Syas2AeaLeT5jHviE7ZlxQBCXwCrkKSy3fFO84Fjg4yim/hJx6mNPNZ4O 8mbnAoqC9Broa6dyAZ8S0w0AZlq3BxnS4TCr50fIFIfBinHvBAsmMkMA9tozecEuSS+Dzwza oD6LtuBw9E4WFhxw2oPBodEREGyH0Rtg7dg1XpAJehH0X6qa1EX0OqivP8eb2hAJC7u+T27j T+gGbyk+dRGCtozLxFQ+FHYUDkwyPpHPqrmCFwC66D+r6D+rGGAmEMGqNcpEpWalepWKPRmj kSk1WlVizNEkpRlMPjoqygIemD7H2AFGATuAFC6ZJVI4k1ixKEGWeprlkg5zWgWrsYO5FCEJ F434dAliOlGWzB5nWTXL/XMxij8nHSOvFh+o0R0nzygGvV2jPrnladPhtCWvUt0bv15ujSMN T1vbe2Olzx5n+xr59q35Dv88x9LmbTs74/OF5TXijH1SscfY6Ne3ddpyjQv7xh/CkmS2NbfZ 0SUQvvDvlun5wrvPG+Qny8Qtu+pzAiLZB2bHiiOu6Q3DF35JU9tt7vC1qbgrvsD3HvxIifbe iAzdaDx7ernuyfWfnW+qB4XP0/xf6+sMr0YWXNztPGSOi91L7DsSWP0ue8HtKuuJg6aIxV4n V+Dd8vImXVbxpNcWKpjmkGe4Dv18uyaOU81e/65Ul6OVWlP2f1BZTf2/hzPFeV2xO9TN46uV 3M3Emj8jEkJzTLYyCuc0sr8K69AIVwMAAA== X-CMS-MailID: 20231206101231epcas5p1756d9f527df621f3a0eebb37ca257f06 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101231epcas5p1756d9f527df621f3a0eebb37ca257f06 References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty Before enabling copy for dm target, check if underlying devices and dm target support copy. Avoid split happening inside dm target. Fail early if the request needs split, currently splitting copy request is not supported. Signed-off-by: Nitesh Shetty --- drivers/md/dm-table.c | 37 +++++++++++++++++++++++++++++++++++ drivers/md/dm.c | 7 +++++++ include/linux/device-mapper.h | 3 +++ 3 files changed, 47 insertions(+) diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c index 198d38b53322..a3e1695d111f 100644 --- a/drivers/md/dm-table.c +++ b/drivers/md/dm-table.c @@ -1899,6 +1899,38 @@ static bool dm_table_supports_nowait(struct dm_table *t) return true; } +static int device_not_copy_capable(struct dm_target *ti, struct dm_dev *dev, + sector_t start, sector_t len, void *data) +{ + struct request_queue *q = bdev_get_queue(dev->bdev); + + return !q->limits.max_copy_sectors; +} + +static bool dm_table_supports_copy(struct dm_table *t) +{ + struct dm_target *ti; + unsigned int i; + + for (i = 0; i < t->num_targets; i++) { + ti = dm_table_get_target(t, i); + + if (!ti->copy_offload_supported) + return false; + + /* + * target provides copy support (as implied by setting + * 'copy_offload_supported') + * and it relies on _all_ data devices having copy support. + */ + if (!ti->type->iterate_devices || + ti->type->iterate_devices(ti, device_not_copy_capable, NULL)) + return false; + } + + return true; +} + static int device_not_discard_capable(struct dm_target *ti, struct dm_dev *dev, sector_t start, sector_t len, void *data) { @@ -1981,6 +2013,11 @@ int dm_table_set_restrictions(struct dm_table *t, struct request_queue *q, q->limits.discard_misaligned = 0; } + if (!dm_table_supports_copy(t)) { + q->limits.max_copy_sectors = 0; + q->limits.max_copy_hw_sectors = 0; + } + if (!dm_table_supports_secure_erase(t)) q->limits.max_secure_erase_sectors = 0; diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 8dcabf84d866..c209348a0064 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -1721,6 +1721,13 @@ static blk_status_t __split_and_process_bio(struct clone_info *ci) if (unlikely(ci->is_abnormal_io)) return __process_abnormal_io(ci, ti); + if ((unlikely(op_is_copy(ci->bio->bi_opf)) && + max_io_len(ti, ci->sector) < ci->sector_count)) { + DMERR("Error, IO size(%u) > max target size(%llu)\n", + ci->sector_count, max_io_len(ti, ci->sector)); + return BLK_STS_IOERR; + } + /* * Only support bio polling for normal IO, and the target io is * exactly inside the dm_io instance (verified in dm_poll_dm_io) diff --git a/include/linux/device-mapper.h b/include/linux/device-mapper.h index 772ab4d74d94..d840ed38516c 100644 --- a/include/linux/device-mapper.h +++ b/include/linux/device-mapper.h @@ -397,6 +397,9 @@ struct dm_target { * bio_set_dev(). NOTE: ideally a target should _not_ need this. */ bool needs_bio_set_dev:1; + + /* copy offload is supported */ + bool copy_offload_supported:1; }; void *dm_per_bio_data(struct bio *bio, size_t data_size); From patchwork Wed Dec 6 10:02:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 13481386 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A169E2DF72 for ; Wed, 6 Dec 2023 10:12:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="F90pFI3S" Received: from epcas5p4.samsung.com (unknown [182.195.41.42]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20231206101247epoutp0252c496475119a6a440d9478496b28ad1~eNnDsxK7j2310423104epoutp02j for ; Wed, 6 Dec 2023 10:12:47 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20231206101247epoutp0252c496475119a6a440d9478496b28ad1~eNnDsxK7j2310423104epoutp02j DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857567; bh=til+KvRoG+1Iq7dmMib83hgZfmKc66+HX7+qqGC9x84=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F90pFI3S3E8AN0HKk4FzmhMPO2pnz6KMJAfes8vFUt5UUpjnvfEw8vqGJ0OjyfVGj Z6dTzZJ3Uijz3LJIkpG94H+vlLwojwa+mqCPclRMn+p+ETok1c7si8akUnYiqV47Ve wnX/jVWQdaZmSz5EWcxv2KJjTL1qodnf3y5Yu308= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p3.samsung.com (KnoxPortal) with ESMTP id 20231206101246epcas5p34ee90169766a41f9474ed71bea400f35~eNnC6IJGp0789207892epcas5p3S; Wed, 6 Dec 2023 10:12:46 +0000 (GMT) Received: from epsmges5p3new.samsung.com (unknown [182.195.38.178]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4SlYB50Nk1z4x9Q5; Wed, 6 Dec 2023 10:12:45 +0000 (GMT) Received: from epcas5p4.samsung.com ( [182.195.41.42]) by epsmges5p3new.samsung.com (Symantec Messaging Gateway) with SMTP id 9A.22.09672.C1940756; Wed, 6 Dec 2023 19:12:44 +0900 (KST) Received: from epsmtrp1.samsung.com (unknown [182.195.40.13]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20231206101244epcas5p38c9d5f1cb01158321f59dbfb4f957470~eNnAdnHs90698906989epcas5p3Z; Wed, 6 Dec 2023 10:12:44 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp1.samsung.com (KnoxPortal) with ESMTP id 20231206101244epsmtrp1c90fe9cdb177c4af9e602b20f796b0ae~eNnAcgHiz1112611126epsmtrp1x; Wed, 6 Dec 2023 10:12:44 +0000 (GMT) X-AuditID: b6c32a4b-60bfd700000025c8-98-6570491c2584 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 46.62.08817.C1940756; Wed, 6 Dec 2023 19:12:44 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101240epsmtip26a0b39642efe1b16d765e62dd823455a~eNm8nTf371205412054epsmtip2Q; Wed, 6 Dec 2023 10:12:39 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 10/12] dm: Enable copy offload for dm-linear target Date: Wed, 6 Dec 2023 15:32:42 +0530 Message-Id: <20231206100253.13100-11-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA01Ta0xTZxjOd87pBRb0rDL5wAWwcUuEgC203YcBNzc3jmOLZGKWjGzQweHa m70okywyuQyZgFDRgRKkK3KRyCiIlNugXrgsCuGi0gWmox1lDBDUqWGOtbRz/nve932ePN/z fnnZOKeA5cNOlalppUws4TLdibar2wOCXt+roHlzReGoaegGjpYfrxLo4lQxE81fXQHI0vst QOd1lQSa7DViqEtXiqH6i9cxVGq6DZB1ogJD3eZAVJ2nJ1BX9yCBxjrOMVHVBSsLfXennYlq +//B0N2TVoBK8icw1G75BqC21SocXZpfItCAeQsaft7PeMebMlZMsajh6WaCGrupoQwNx5lU i/4oZWspB1TnZBaT+qFIy6AKsxeZ1LLVTFBLPRNMqqi1AVAtP2dSDw2+lMGygEVv/Cw9PIUW J9JKf1qWIE9MlSVHcKP2x70XJxTx+EH8MPQW118mltIR3D0fRQd9kCqxb4Prf0gs0dhb0WKV irtjV7hSrlHT/ilylTqCSysSJQqBIlgllqo0suRgGa3eyefxQoR2Ynx6yuT0FFBMMTJqTvVg WcBGFAA3NiQF0DxyASsA7mwO2Qlg56M1wlmsAFhoaXYVfwE4O5b3QlJ/pRw4B90A1twyulgP ARybsbEKAJvNJLfDEa3GIfAkG3FobOY7ODj5DIPZdQPAMdhERsKnl/9mOTBBvgHrsrIwB/Yg w2DfUKvLzQ+Wjz5Z57jZ+9pZHe7kvAoHyy3rHNzOyb58FncYQLLJDV6qa2U4xXugrbWD5cSb 4B/9rS7sA+eK81w4AY6W38KcWA1nuvpc+G2YO1SMO8Lg9jBNHTucXhtg4aoFc7Qh6QHz8zhO 9lY4XWp1uXrB+9/rXZiCLabf15/JIU8A2Fuz/yTwq3gpQcVLCSr+NzsP8AbgTStU0mRaJVSE yujDLz42QS41gPWjCIhqB7/dexBsAhgbmABk41xPD8mwnOZ4JIq/OkIr5XFKjYRWmYDQvuIS 3Oe1BLn9qmTqOL4gjCcQiUSCsFARn+vlMZ9bmcghk8VqOp2mFbTyPx3GdvPJwnJqZYvWB59X nQ1IKwVx0qVTbtWfMFK1aeEZezMXr23OHh0v2z0TklFyQphkno4/x4/6YtIQ+UTgk13tdyy+ egP7vlZz+GnswrOZKZk0Q3DTa3jtXuOHvxDalbaD0bFDQv+dP9nWcg6afA9tfk5ONA0Mvs8f PiYJ7Pi0+871kvF9oddsoq7mNK+g3AM13TfMSSMHdO+Oz3Se1tO/9jze90i98eP5K8d1vrpd R3lpXzOq9YE9yXM12iGTNDKpvzL/rmCLIfzNnIzMM2XbPLsKte7TsVtnf+xUe0+E/Nm3wHpl MabodmZj/XIMmK/TrxnntKZtxiO5Ft4ZU8puXlnMeO2XpCeXUKWI+QG4UiX+F+4VIRudBAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrPIsWRmVeSWpSXmKPExsWy7bCSvK6MZ0Gqwbu/fBbrTx1jtvj49TeL xeq7/WwWrw9/YrR4cqCd0WLBorksFjcP7GSy2LNoEpPFytVHmSwmHbrGaPH06iwmi723tC0W ti1hsdiz9ySLxeVdc9gs5i97ym7RfX0Hm8Xy4/+YLG5MeMpoMbHjKpPFjieNjBbbfs9ntlj3 +j2LxYlb0hbn/x5ndZD02DnrLrvH+XsbWTwuny312LSqk81j85J6jxebZzJ67L7ZwOaxuG8y q0dv8zs2j49Pb7F4vN93lc2jb8sqRo/Np6s9Pm+S89j05C1TAH8Ul01Kak5mWWqRvl0CV8bN e3cZC+6yViydso+pgfEFSxcjJ4eEgInEyu0zGbsYuTiEBHYzSpy9fZwJIiEu0XztBzuELSyx 8t9zdoiij4wS0/uuAxVxcLAJaEpcmFwKEhcR2MEs8XNtMxOIwyzQzixx+tVrsEnCAu4SP7b+ AZvEIqAqsaKhASzOK2ApcfDUFqgz5CVmXvoOVsMJFJ/8fBEziC0kYCGxr3E6C0S9oMTJmU/A bGag+uats5knMArMQpKahSS1gJFpFaNkakFxbnpusWGBUV5quV5xYm5xaV66XnJ+7iZGcJxr ae1g3LPqg94hRiYOxkOMEhzMSiK8OefzU4V4UxIrq1KL8uOLSnNSiw8xSnOwKInzfnvdmyIk kJ5YkpqdmlqQWgSTZeLglGpgkrzp4N/ssnTl1ZksvfcP5OkeODfPYqvm6ux7sYvzlkw4Lba6 vMXc85oy7/PCiBNvl1wtdjy2/MsvTg0Rqw8XyjL8dsquUtZVWqSUPfu72l6uNfFVed/vG896 veWbWEjCIqXebWkt9gEmyy+fndMh6n3nwYn96lWdqse0vFmLJ6pscZ7U9CSLiW2ujonw7mkH pudufF63+2XbVLeyUP+PM8Lfp24Tc9y5kHl9nL8R29wQ/WmXLq+T7v4vwzY7PO1Qd+Gt3tWr lDbYBz3iWnkkfXLty0OCNdxzpr0sySqUn881/eDPj4UJ5Ur69r7XdZVYozM+/Kq3UvugobRs /fJf+eENNh8XfI3RimXh61ozRYmlOCPRUIu5qDgRABxSzHRiAwAA X-CMS-MailID: 20231206101244epcas5p38c9d5f1cb01158321f59dbfb4f957470 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101244epcas5p38c9d5f1cb01158321f59dbfb4f957470 References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty Setting copy_offload_supported flag to enable offload. Reviewed-by: Hannes Reinecke Signed-off-by: Nitesh Shetty --- drivers/md/dm-linear.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/md/dm-linear.c b/drivers/md/dm-linear.c index 2d3e186ca87e..cfec2fac28e1 100644 --- a/drivers/md/dm-linear.c +++ b/drivers/md/dm-linear.c @@ -62,6 +62,7 @@ static int linear_ctr(struct dm_target *ti, unsigned int argc, char **argv) ti->num_discard_bios = 1; ti->num_secure_erase_bios = 1; ti->num_write_zeroes_bios = 1; + ti->copy_offload_supported = 1; ti->private = lc; return 0; From patchwork Wed Dec 6 10:02:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 13481387 Received: from mailout2.samsung.com (mailout2.samsung.com [203.254.224.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B76482D04C for ; Wed, 6 Dec 2023 10:13:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="M0GB+QCV" Received: from epcas5p3.samsung.com (unknown [182.195.41.41]) by mailout2.samsung.com (KnoxPortal) with ESMTP id 20231206101300epoutp027a3ba8ed408abb65c0ebc9c2e22c3b0b~eNnPyfYQX2310423104epoutp02r for ; Wed, 6 Dec 2023 10:13:00 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.samsung.com 20231206101300epoutp027a3ba8ed408abb65c0ebc9c2e22c3b0b~eNnPyfYQX2310423104epoutp02r DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857580; bh=1x20GT9zf7S1tCU9RCvaOg6FImzWr8BSd/HxmmWgI7M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=M0GB+QCVe6oVxdni4uuAcVQ8SWK0IHezmwmBx3mZd5wskujheiE/EcHHqoGbfHSmT zBvniwmg9rf0s5Z7D4h3/hzJPAbwvDydR3ePyNdatyszdacjkYdpuRqImOboOHkH6m pzOzldeDPcGmAUJOl1gZSkeo6GMn+6cwAfiyrTM8= Received: from epsnrtp4.localdomain (unknown [182.195.42.165]) by epcas5p4.samsung.com (KnoxPortal) with ESMTP id 20231206101300epcas5p4e0ba5cbc5ff683cdd722f4d403b71780~eNnPIVgZC2294422944epcas5p4s; Wed, 6 Dec 2023 10:13:00 +0000 (GMT) Received: from epsmgec5p1-new.samsung.com (unknown [182.195.38.177]) by epsnrtp4.localdomain (Postfix) with ESMTP id 4SlYBK6PQbz4x9Q2; Wed, 6 Dec 2023 10:12:57 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmgec5p1-new.samsung.com (Symantec Messaging Gateway) with SMTP id 9A.98.19369.92940756; Wed, 6 Dec 2023 19:12:57 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p2.samsung.com (KnoxPortal) with ESMTPA id 20231206101257epcas5p253880d2f8f6318483cd2361b109350d3~eNnMhaHzO2308423084epcas5p24; Wed, 6 Dec 2023 10:12:57 +0000 (GMT) Received: from epsmgms1p2new.samsung.com (unknown [182.195.42.42]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20231206101257epsmtrp2611297c07d68894fd770de346b022efd~eNnMfOfmP0990909909epsmtrp2V; Wed, 6 Dec 2023 10:12:57 +0000 (GMT) X-AuditID: b6c32a50-c99ff70000004ba9-28-657049290644 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgms1p2new.samsung.com (Symantec Messaging Gateway) with SMTP id 09.62.08817.92940756; Wed, 6 Dec 2023 19:12:57 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101252epsmtip2f2b615fa7c066041d23ec7348dd3a4e9~eNnIUmOIz0221302213epsmtip2K; Wed, 6 Dec 2023 10:12:52 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Anuj Gupta , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 11/12] null: Enable trace capability for null block Date: Wed, 6 Dec 2023 15:32:43 +0530 Message-Id: <20231206100253.13100-12-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02Te0ybVRjGc76vfG2HxQrTHaoO+BKj4MpaB+xUgZFs04+gGQnRuP1hbehH YZS29uKYRGUwGJcxCngr47IVcHLZgMK4rThgKrSo5Va56NgGRVSyEWC4ZIxpSzvdf7/3vM9z 3jzvyWHhviVMHitFoaXVComcJLYxOq4FB/ODY1W0YCmPj5qtP+BoZX2DgbL0mzhqvF5MoKVr qwA5+k4BdM5YyUDTfd0YMhtLMVTf+D2GSgd+AWjBXo6h3pmX0fncWgYy91oYaLyngkDVXy8w UeFkF4EuDD7E0JR+AaCSPDuGuhwnAOrYqMbRpaVlBhqaeRbZNge9YnhUd/l1JmWbbWVQ4z/p KFNDPkG11X5K/dFmANSV6UyCqjlT5kUVZd8hqJWFGQa1/K2doM60NwCqbTiDWjPtpEyO21j8 k0dSI5NpiZRWB9KKRKU0RSGLIuMSxPvF4RECIV8oQnvJQIUkjY4iD7wZz389Re5cCRn4oUSu cx7FSzQacnd0pFqp09KByUqNNoqkVVK5KkwVqpGkaXQKWaiC1r4qFAheCXcK309Nbh2oJFQd fuk3aidBJjBzCwCbBblhcMI6wSwA21i+XDOAn1+wEO5iFcB1kx64i78BfJCr93pkyc/LwdyN XgDzWxo9/jUAe25mOjssFsENhiNlOpdhO7cJh92tQpcG51bhsG1uEHM1/LhvwFN1w1vM4L4A h/T2LeZwRfDqjzame1oANIzd22K287xs0Yi7NU9Bi8HBcDHu1GRfPou7BkBuJxsWnV7E3eYD 8ObFKQ/7wb8G2z2X8uDanV7CzYlwzPAz5mYtnDf3e3gfzLEW464wuDNMc89u9ywfWLTh2MoI uRyYl+vrVgfB2dIFz4J2wFtf1XqYgqN1uZ5lnQZw9Mqilx4ElD8WofyxCOX/TzsH8AbAo1Wa NBmdGK4S8hX0sf+eNlGZZgJbfyMkvgs0tmyGDgCMBQYAZOHkdo7cpqR9OVLJ8Y9otVKs1slp zQAIdy65BOc9nah0fi6FViwMEwnCIiIiwkR7IoTkDs5STqXUlyuTaOlUmlbR6kc+jMXmZWJF 05lz+xwpCTard4I3LKpp6SDbMl7qye6/0S5LKlzs19my1NaV6PwhosKn1F8hPzEfc9l21PiW kRG03/Euudw4Ibr0Dy+m7zXzeyO/P98wInn7YIXs45q9Sz0b/rO/FhiCpF1+POX4B3aW2T4W TbONm5oj3yWMJsVOxRYealr0r7KG7An54p2sANnYff39+rvVlZ2H/MbZppLO1fS4P+9eLLb8 dtui/+aJGGaTZVkkXJ9n7/K5l+Egjqcf3SWWDX1ysvA5rZfms2oD9eVhy/kSo/fkTv4xTvPh 8LoZyzMxwy82yPOI6aSqs7dOjgri5rIfVK+qrkZGxzWL64NVyQ7hQ5KhSZYIQ3C1RvIvdqji VqQEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrMIsWRmVeSWpSXmKPExsWy7bCSvK6mZ0GqwawjXBbrTx1jtvj49TeL RdOEv8wWq+/2s1m8PvyJ0eLJgXZGiwWL5rJY3Dywk8liz6JJTBYrVx9lsph06BqjxdOrs5gs 9t7StljYtoTFYs/ekywWl3fNYbOYv+wpu0X39R1sFsuP/2OyuDHhKaPFxI6rTBY7njQyWmz7 PZ/ZYt3r9ywWJ25JW5z/e5zVQcpj56y77B7n721k8bh8ttRj06pONo/NS+o9Xmyeyeix+2YD m8fivsmsHr3N79g8Pj69xeLxft9VNo++LasYPTafrvb4vEnOY9OTt0wB/FFcNimpOZllqUX6 dglcGRsPzWUr2CZccX/JdcYGxj0CXYycHBICJhKdHa1MXYxcHEICuxkluh6cZYdIiEs0X/sB ZQtLrPz3nB2i6COjxLIZfSxdjBwcbAKaEhcml4LERQR2MEv8XNvMBNLALLCGWWLLXF4QW1jA XaJ96WmwOIuAqsSJCVfBbF4BS4n9Z85DLZCXmHnpO5jNCRSf/HwRM4gtJGAhsa9xOgtEvaDE yZlPWCDmy0s0b53NPIFRYBaS1CwkqQWMTKsYJVMLinPTc4sNC4zyUsv1ihNzi0vz0vWS83M3 MYLjXUtrB+OeVR/0DjEycTAeYpTgYFYS4c05n58qxJuSWFmVWpQfX1Sak1p8iFGag0VJnPfb 694UIYH0xJLU7NTUgtQimCwTB6dUA9MGnfk2ka/sCgwSNnidOucimHrtZZXsTef1ktVO94zW Gu6Kr3CUDn1f3bFhU7/C0a/VnAHyvU0nnU5NFl70df7xyJvxJ7yzrwQ8MfyzQmODiFFD8Y+2 9z5JaX1V/g8rYjmNxQOe78r0457ZppbXv5DlZ1QP72F52ZUmU+w/1u9+u8n5isvesgkZvh+v PGC4c/s4f5igitqLGvuwbMeg3vQOC957W+XFWyYl3/q4flnS9OzbXXvFFtq5laSk5xps/aed cr3gzSvWCdUbTr7ueD9FOkJa7dvr/deZit2TzlraPTv6/KfSA02untW3un8unli9rnPH1dQl KgyL/M9eeGycXtZr2L56kcuBm08TZyuxFGckGmoxFxUnAgAxHznxZgMAAA== X-CMS-MailID: 20231206101257epcas5p253880d2f8f6318483cd2361b109350d3 X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101257epcas5p253880d2f8f6318483cd2361b109350d3 References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty This is a prep patch to enable copy trace capability. At present only zoned null_block is using trace, so we decoupled trace and zoned dependency to make it usable in null_blk driver also. Reviewed-by: Hannes Reinecke Signed-off-by: Nitesh Shetty Signed-off-by: Anuj Gupta --- drivers/block/null_blk/Makefile | 2 -- drivers/block/null_blk/main.c | 3 +++ drivers/block/null_blk/trace.h | 2 ++ drivers/block/null_blk/zoned.c | 1 - 4 files changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/block/null_blk/Makefile b/drivers/block/null_blk/Makefile index 84c36e512ab8..672adcf0ad24 100644 --- a/drivers/block/null_blk/Makefile +++ b/drivers/block/null_blk/Makefile @@ -5,7 +5,5 @@ ccflags-y += -I$(src) obj-$(CONFIG_BLK_DEV_NULL_BLK) += null_blk.o null_blk-objs := main.o -ifeq ($(CONFIG_BLK_DEV_ZONED), y) null_blk-$(CONFIG_TRACING) += trace.o -endif null_blk-$(CONFIG_BLK_DEV_ZONED) += zoned.o diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c index 3021d58ca51c..1b40c674f62b 100644 --- a/drivers/block/null_blk/main.c +++ b/drivers/block/null_blk/main.c @@ -11,6 +11,9 @@ #include #include "null_blk.h" +#define CREATE_TRACE_POINTS +#include "trace.h" + #undef pr_fmt #define pr_fmt(fmt) "null_blk: " fmt diff --git a/drivers/block/null_blk/trace.h b/drivers/block/null_blk/trace.h index 6b2b370e786f..91446c34eac2 100644 --- a/drivers/block/null_blk/trace.h +++ b/drivers/block/null_blk/trace.h @@ -30,6 +30,7 @@ static inline void __assign_disk_name(char *name, struct gendisk *disk) } #endif +#ifdef CONFIG_BLK_DEV_ZONED TRACE_EVENT(nullb_zone_op, TP_PROTO(struct nullb_cmd *cmd, unsigned int zone_no, unsigned int zone_cond), @@ -67,6 +68,7 @@ TRACE_EVENT(nullb_report_zones, TP_printk("%s nr_zones=%u", __print_disk_name(__entry->disk), __entry->nr_zones) ); +#endif /* CONFIG_BLK_DEV_ZONED */ #endif /* _TRACE_NULLB_H */ diff --git a/drivers/block/null_blk/zoned.c b/drivers/block/null_blk/zoned.c index 55c5b48bc276..9694461a31a4 100644 --- a/drivers/block/null_blk/zoned.c +++ b/drivers/block/null_blk/zoned.c @@ -3,7 +3,6 @@ #include #include "null_blk.h" -#define CREATE_TRACE_POINTS #include "trace.h" #undef pr_fmt From patchwork Wed Dec 6 10:02:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kanchan Joshi X-Patchwork-Id: 13481388 Received: from mailout1.samsung.com (mailout1.samsung.com [203.254.224.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4FB572D047 for ; Wed, 6 Dec 2023 10:13:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=samsung.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=samsung.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=samsung.com header.i=@samsung.com header.b="RONvNQbf" Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout1.samsung.com (KnoxPortal) with ESMTP id 20231206101313epoutp018531eae8c7bbcb0567513833c9366536~eNnbksvDH0454704547epoutp01z for ; Wed, 6 Dec 2023 10:13:13 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.samsung.com 20231206101313epoutp018531eae8c7bbcb0567513833c9366536~eNnbksvDH0454704547epoutp01z DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1701857593; bh=xTrfyi/yJidoNrl3TPzzopI+GnPn+ZUhLrlBrAJpZmk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RONvNQbfKPP7JVvjs7/Cza28A52wzlIx90cwYe3YvAdVCJEITxSXUxRQRkVuU4Hrs EXjltQKrrtIHLDJNzBZc0Si7sXIvm09t8hUWceIR4z3qQVGCJjLsRAkPxiyMnS4Ifn sKZNZuXoukFqHOHKNCzHaWtMMOTo3WvokxoY+HMs= Received: from epsnrtp2.localdomain (unknown [182.195.42.163]) by epcas5p2.samsung.com (KnoxPortal) with ESMTP id 20231206101312epcas5p27ba89066024e85afe3dcbfa68f2ad196~eNna9UUf43251632516epcas5p20; Wed, 6 Dec 2023 10:13:12 +0000 (GMT) Received: from epsmgec5p1-new.samsung.com (unknown [182.195.38.175]) by epsnrtp2.localdomain (Postfix) with ESMTP id 4SlYBb3qKRz4x9Pw; Wed, 6 Dec 2023 10:13:11 +0000 (GMT) Received: from epcas5p3.samsung.com ( [182.195.41.41]) by epsmgec5p1-new.samsung.com (Symantec Messaging Gateway) with SMTP id 74.A8.19369.73940756; Wed, 6 Dec 2023 19:13:11 +0900 (KST) Received: from epsmtrp2.samsung.com (unknown [182.195.40.14]) by epcas5p4.samsung.com (KnoxPortal) with ESMTPA id 20231206101310epcas5p43f06695868958526fed828762e728e7b~eNnZC_V9U2295022950epcas5p46; Wed, 6 Dec 2023 10:13:10 +0000 (GMT) Received: from epsmgmcp1.samsung.com (unknown [182.195.42.82]) by epsmtrp2.samsung.com (KnoxPortal) with ESMTP id 20231206101310epsmtrp24b03b0bf079829d4f931fce53db76085~eNnZBqF9B1007210072epsmtrp21; Wed, 6 Dec 2023 10:13:10 +0000 (GMT) X-AuditID: b6c32a50-c99ff70000004ba9-4a-657049372387 Received: from epsmtip2.samsung.com ( [182.195.34.31]) by epsmgmcp1.samsung.com (Symantec Messaging Gateway) with SMTP id 6A.85.18939.63940756; Wed, 6 Dec 2023 19:13:10 +0900 (KST) Received: from localhost.localdomain (unknown [107.99.41.245]) by epsmtip2.samsung.com (KnoxPortal) with ESMTPA id 20231206101305epsmtip2045c6dd8db41916fbde2ae92d7cd27d5~eNnUSGSf50087600876epsmtip2e; Wed, 6 Dec 2023 10:13:05 +0000 (GMT) From: Kanchan Joshi To: Jens Axboe , Jonathan Corbet , Alasdair Kergon , Mike Snitzer , Mikulas Patocka , dm-devel@lists.linux.dev, Keith Busch , Christoph Hellwig , Sagi Grimberg , Chaitanya Kulkarni , Alexander Viro , Christian Brauner Cc: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, nitheshshetty@gmail.com, anuj1072538@gmail.com, gost.dev@samsung.com, mcgrof@kernel.org, Nitesh Shetty , Hannes Reinecke , Damien Le Moal , Anuj Gupta , Vincent Fu , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-nvme@lists.infradead.org, linux-fsdevel@vger.kernel.org Subject: [PATCH v18 12/12] null_blk: add support for copy offload Date: Wed, 6 Dec 2023 15:32:44 +0530 Message-Id: <20231206100253.13100-13-joshi.k@samsung.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231206100253.13100-1-joshi.k@samsung.com> Precedence: bulk X-Mailing-List: dm-devel@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA02Te0xbdRTH97v3trTL2K5lZL+wZbBGEoEUWuXxAwq6bBlXRqSJURPRQC03 hVDa2seGmmUdWB7teIxOM2CjvNwDEOQpRVgmxCE4Vjbs1pKAOlo0MkFgAycPbS2b++9zTr4n 33zPyWHhnFqfAFa2XEOr5GIZl7mT6B0OCeHFvK6k+Vd1ELWP3cTR0uN1AuVXbOKoZbqcieaH lwFy3igCaH3ciqO6hksEctywYGigoRJD11q+w1Dl0D2AXLZqDA1OhaH6wiYCDQyOEmiy/yIT mS+7fJDxfh8TXRnZwpC9wgXQuWIbhvqcZwDqXTfjqG1+kUDfT+1Hs8ZigKybI4zXDlCW6mkf yjrTQVCT41qqs7mESXU1naZ+66oC1DcOHZNqLDMxqNKCBSZl0f/EoJZcUwS1eN3GpMq6mwHV 9cMn1ErnQarT+QcmIt/NEWbR4kxaFUTLJYrMbLk0gXv8zfQj6VHRfAFPEItiuEFycS6dwD2a IuIdy5a5N8QNOiGWad0tkVit5kYkClUKrYYOylKoNQlcWpkpU0Yqw9XiXLVWLg2X05o4AZ// cpRbmJGTVblhYCh1SXl9owoduBdnAGwWJCPhF2sjTAPYyeKQAwAWuK5j3mIZQN2VVeaz4tP2 HuzpyNn8y7iHOaQFwGIz6RWtAPijfY5hACwWkwyBEyatR7OXbMWhpUPg0eDkXzhsHPH4sVl+ 5GHomGoCHibIYPjAXsrwsC8ZCwcetTK9ZoGw6u6aj4fZ7r7p1wbcq3kBjlY5CQ/jbk1BTw3u MYDkMBtOzzkY3uGjcO7rEuBlP/j7SLePlwPgysLgtoEE3q26vZ1MA2cHvt3mV6F+rBz3hMHd Ydr7I7xeu2HpuhPztCHpC4sLOV71IThT6dp23Qd/udC0zRRsKRwnvPs5C+DMmJlRAQKrn4tQ /VyE6v/d6gDeDAJopTpXSkuilAKenD757LASRW4n+O9RQkV9oOWrzfAhgLHAEIAsnLvXV2ZV 0BzfTPFHH9MqRbpKK6PVQyDKveRzeIC/ROH+NLkmXRAZy4+Mjo6OjH0lWsDd5zuvv5TJIaVi DZ1D00pa9XQOY7EDdJhRa5d9UGtvTbJFSZMNh4vUX9ruhE4K+/qPv3jnVINT/49vPB4u+RCr HytvPBXSSpaJ8Pbyh0OzQRt80UL8g5vx72vyKvP0Vtd+aczJ3vsm09buNHNYkZ82P0N45G+j xS+wUGgvnre9ETbNS1UR4wfNu2rXUopqatK3DMNntOLzvHrhbWE9uWcpyXWiZtV84MKxxccR s8Fs04Sxvz0Ye+JI1qMWxi3jjl1pC4aS+rbEsoDOlw619cSTDVlPEvf8PGidTDE4Coyp7yRn 2FZXrxUkmicu/il9+FZ4U+6y/60i7XshaY/eFkn9ZdTnNhiXlLrx2TRMqxtpO301t7tjB5dQ Z4kFobhKLf4Xr0p7nLEEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrLIsWRmVeSWpSXmKPExsWy7bCSvK6ZZ0Gqwbb1KhbrTx1jtvj49TeL RdOEv8wWq+/2s1m8PvyJ0eLJgXZGi99nzzNbLFg0l8Xi5oGdTBZ7Fk1isli5+iiTxaRD1xgt nl6dxWSx95a2xcK2JSwWe/aeZLG4vGsOm8X8ZU/ZLbqv72CzWH78H5PFjQlPGS0mdlxlstjx pJHRYtvv+cwW616/Z7E4cUva4nF3B6PF+b/HWR1kPHbOusvucf7eRhaPy2dLPTat6mTz2Lyk 3uPF5pmMHrtvNrB5LO6bzOrR2/yOzWNn631Wj49Pb7F4vN93lc2jb8sqRo/Np6s9Pm+S89j0 5C1TgEAUl01Kak5mWWqRvl0CV8akP12sBQ3uFTtO5jcwXrPqYuTkkBAwkehpWsYMYgsJbGeU mPFXHiIuLtF87Qc7hC0ssfLfcyCbC6jmI6PE3487WLoYOTjYBDQlLkwuBYmLCOxglvi5tpkJ xGEW6GCRuLBtL1i3sICjxM1bSxhBbBYBVYlHN3pZQWxeAUuJPV/WsEFskJeYeek7WD0nUHzy 80VQF1lI7GuczgJRLyhxcuYTMJsZqL5562zmCYwCs5CkZiFJLWBkWsUomlpQnJuem1xgqFec mFtcmpeul5yfu4kRnAK0gnYwLlv/V+8QIxMH4yFGCQ5mJRHenPP5qUK8KYmVValF+fFFpTmp xYcYpTlYlMR5lXM6U4QE0hNLUrNTUwtSi2CyTBycUg1M3iXFdZdzDiSpbVI1MvnQbW25/k4/ k5I99w2unWuuPjvNkWF2933znmezbBM69h9tXLix7gDH+W9zZeJMdp17fbdOdFZL3pUPBv9C XknlvPgaIzN9DZvj1YS3y5J9cqI3PlthfKTY7+PRGWuvBrzJOtQQaH9cmkdYZH7+Jpddk9rN 5jwTfuloJR8cwB9zacMayQ1ClqXhP0xu8Cp7LZh3syr5cdPm3BdG7XN+pb+z38nDcJfhu1CE Y/TboJut/gvylXSWi89Me8KzvuZsSsjKKaG3O1r6FMTzZ6YVb6sJXfiryfviGfOzLqqPMz59 Mp1829o4OOP08y61fvYjoT+3PVjQeC6Z8ROXrPPUOdd3KrEUZyQaajEXFScCAPx2ZblwAwAA X-CMS-MailID: 20231206101310epcas5p43f06695868958526fed828762e728e7b X-Msg-Generator: CA CMS-TYPE: 105P DLP-Filter: Pass X-CFilter-Loop: Reflected X-CMS-RootMailID: 20231206101310epcas5p43f06695868958526fed828762e728e7b References: <20231206100253.13100-1-joshi.k@samsung.com> From: Nitesh Shetty Implementation is based on existing read and write infrastructure. copy_max_bytes: A new configfs and module parameter is introduced, which can be used to set hardware/driver supported maximum copy limit. Only request based queue mode will support for copy offload. Added tracefs support to copy IO tracing. Reviewed-by: Hannes Reinecke Suggested-by: Damien Le Moal Signed-off-by: Anuj Gupta Signed-off-by: Nitesh Shetty Signed-off-by: Vincent Fu --- Documentation/block/null_blk.rst | 5 ++ drivers/block/null_blk/main.c | 97 ++++++++++++++++++++++++++++++- drivers/block/null_blk/null_blk.h | 1 + drivers/block/null_blk/trace.h | 23 ++++++++ 4 files changed, 123 insertions(+), 3 deletions(-) diff --git a/Documentation/block/null_blk.rst b/Documentation/block/null_blk.rst index 4dd78f24d10a..6153e02fcf13 100644 --- a/Documentation/block/null_blk.rst +++ b/Documentation/block/null_blk.rst @@ -149,3 +149,8 @@ zone_size=[MB]: Default: 256 zone_nr_conv=[nr_conv]: Default: 0 The number of conventional zones to create when block device is zoned. If zone_nr_conv >= nr_zones, it will be reduced to nr_zones - 1. + +copy_max_bytes=[size in bytes]: Default: COPY_MAX_BYTES + A module and configfs parameter which can be used to set hardware/driver + supported maximum copy offload limit. + COPY_MAX_BYTES(=128MB at present) is defined in fs.h diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c index 1b40c674f62b..5b1b401a68fb 100644 --- a/drivers/block/null_blk/main.c +++ b/drivers/block/null_blk/main.c @@ -160,6 +160,10 @@ static int g_max_sectors; module_param_named(max_sectors, g_max_sectors, int, 0444); MODULE_PARM_DESC(max_sectors, "Maximum size of a command (in 512B sectors)"); +static unsigned long g_copy_max_bytes = BLK_COPY_MAX_BYTES; +module_param_named(copy_max_bytes, g_copy_max_bytes, ulong, 0444); +MODULE_PARM_DESC(copy_max_bytes, "Maximum size of a copy command (in bytes)"); + static unsigned int nr_devices = 1; module_param(nr_devices, uint, 0444); MODULE_PARM_DESC(nr_devices, "Number of devices to register"); @@ -412,6 +416,7 @@ NULLB_DEVICE_ATTR(home_node, uint, NULL); NULLB_DEVICE_ATTR(queue_mode, uint, NULL); NULLB_DEVICE_ATTR(blocksize, uint, NULL); NULLB_DEVICE_ATTR(max_sectors, uint, NULL); +NULLB_DEVICE_ATTR(copy_max_bytes, uint, NULL); NULLB_DEVICE_ATTR(irqmode, uint, NULL); NULLB_DEVICE_ATTR(hw_queue_depth, uint, NULL); NULLB_DEVICE_ATTR(index, uint, NULL); @@ -553,6 +558,7 @@ static struct configfs_attribute *nullb_device_attrs[] = { &nullb_device_attr_queue_mode, &nullb_device_attr_blocksize, &nullb_device_attr_max_sectors, + &nullb_device_attr_copy_max_bytes, &nullb_device_attr_irqmode, &nullb_device_attr_hw_queue_depth, &nullb_device_attr_index, @@ -659,7 +665,8 @@ static ssize_t memb_group_features_show(struct config_item *item, char *page) "poll_queues,power,queue_mode,shared_tag_bitmap,size," "submit_queues,use_per_node_hctx,virt_boundary,zoned," "zone_capacity,zone_max_active,zone_max_open," - "zone_nr_conv,zone_offline,zone_readonly,zone_size\n"); + "zone_nr_conv,zone_offline,zone_readonly,zone_size," + "copy_max_bytes\n"); } CONFIGFS_ATTR_RO(memb_group_, features); @@ -725,6 +732,7 @@ static struct nullb_device *null_alloc_dev(void) dev->queue_mode = g_queue_mode; dev->blocksize = g_bs; dev->max_sectors = g_max_sectors; + dev->copy_max_bytes = g_copy_max_bytes; dev->irqmode = g_irqmode; dev->hw_queue_depth = g_hw_queue_depth; dev->blocking = g_blocking; @@ -1274,6 +1282,81 @@ static int null_transfer(struct nullb *nullb, struct page *page, return err; } +static inline int nullb_setup_copy(struct nullb *nullb, struct request *req, + bool is_fua) +{ + sector_t sector_in = 0, sector_out = 0; + loff_t offset_in, offset_out; + void *in, *out; + ssize_t chunk, rem = 0; + struct bio *bio; + struct nullb_page *t_page_in, *t_page_out; + u16 seg = 1; + int status = -EIO; + + if (blk_rq_nr_phys_segments(req) != BLK_COPY_MAX_SEGMENTS) + return status; + + /* + * First bio contains information about source and last bio contains + * information about destination. + */ + __rq_for_each_bio(bio, req) { + if (seg == blk_rq_nr_phys_segments(req)) { + sector_out = bio->bi_iter.bi_sector; + if (rem != bio->bi_iter.bi_size) + return status; + } else { + sector_in = bio->bi_iter.bi_sector; + rem = bio->bi_iter.bi_size; + } + seg++; + } + + trace_nullb_copy_op(req, sector_out << SECTOR_SHIFT, + sector_in << SECTOR_SHIFT, rem); + + spin_lock_irq(&nullb->lock); + while (rem > 0) { + chunk = min_t(size_t, nullb->dev->blocksize, rem); + offset_in = (sector_in & SECTOR_MASK) << SECTOR_SHIFT; + offset_out = (sector_out & SECTOR_MASK) << SECTOR_SHIFT; + + if (null_cache_active(nullb) && !is_fua) + null_make_cache_space(nullb, PAGE_SIZE); + + t_page_in = null_lookup_page(nullb, sector_in, false, + !null_cache_active(nullb)); + if (!t_page_in) + goto err; + t_page_out = null_insert_page(nullb, sector_out, + !null_cache_active(nullb) || + is_fua); + if (!t_page_out) + goto err; + + in = kmap_local_page(t_page_in->page); + out = kmap_local_page(t_page_out->page); + + memcpy(out + offset_out, in + offset_in, chunk); + kunmap_local(out); + kunmap_local(in); + __set_bit(sector_out & SECTOR_MASK, t_page_out->bitmap); + + if (is_fua) + null_free_sector(nullb, sector_out, true); + + rem -= chunk; + sector_in += chunk >> SECTOR_SHIFT; + sector_out += chunk >> SECTOR_SHIFT; + } + + status = 0; +err: + spin_unlock_irq(&nullb->lock); + return status; +} + static int null_handle_rq(struct nullb_cmd *cmd) { struct request *rq = cmd->rq; @@ -1283,13 +1366,16 @@ static int null_handle_rq(struct nullb_cmd *cmd) sector_t sector = blk_rq_pos(rq); struct req_iterator iter; struct bio_vec bvec; + bool fua = rq->cmd_flags & REQ_FUA; + + if (op_is_copy(req_op(rq))) + return nullb_setup_copy(nullb, rq, fua); spin_lock_irq(&nullb->lock); rq_for_each_segment(bvec, rq, iter) { len = bvec.bv_len; err = null_transfer(nullb, bvec.bv_page, len, bvec.bv_offset, - op_is_write(req_op(rq)), sector, - rq->cmd_flags & REQ_FUA); + op_is_write(req_op(rq)), sector, fua); if (err) { spin_unlock_irq(&nullb->lock); return err; @@ -2074,6 +2160,9 @@ static int null_validate_conf(struct nullb_device *dev) return -EINVAL; } + if (dev->queue_mode == NULL_Q_BIO) + dev->copy_max_bytes = 0; + return 0; } @@ -2193,6 +2282,8 @@ static int null_add_dev(struct nullb_device *dev) dev->max_sectors = queue_max_hw_sectors(nullb->q); dev->max_sectors = min(dev->max_sectors, BLK_DEF_MAX_SECTORS); blk_queue_max_hw_sectors(nullb->q, dev->max_sectors); + blk_queue_max_copy_hw_sectors(nullb->q, + dev->copy_max_bytes >> SECTOR_SHIFT); if (dev->virt_boundary) blk_queue_virt_boundary(nullb->q, PAGE_SIZE - 1); diff --git a/drivers/block/null_blk/null_blk.h b/drivers/block/null_blk/null_blk.h index 929f659dd255..e82e53a2e2df 100644 --- a/drivers/block/null_blk/null_blk.h +++ b/drivers/block/null_blk/null_blk.h @@ -107,6 +107,7 @@ struct nullb_device { unsigned int queue_mode; /* block interface */ unsigned int blocksize; /* block size */ unsigned int max_sectors; /* Max sectors per command */ + unsigned long copy_max_bytes; /* Max copy offload length in bytes */ unsigned int irqmode; /* IRQ completion handler */ unsigned int hw_queue_depth; /* queue depth */ unsigned int index; /* index of the disk, only valid with a disk */ diff --git a/drivers/block/null_blk/trace.h b/drivers/block/null_blk/trace.h index 91446c34eac2..2f2c1d1c2b48 100644 --- a/drivers/block/null_blk/trace.h +++ b/drivers/block/null_blk/trace.h @@ -70,6 +70,29 @@ TRACE_EVENT(nullb_report_zones, ); #endif /* CONFIG_BLK_DEV_ZONED */ +TRACE_EVENT(nullb_copy_op, + TP_PROTO(struct request *req, + sector_t dst, sector_t src, size_t len), + TP_ARGS(req, dst, src, len), + TP_STRUCT__entry( + __array(char, disk, DISK_NAME_LEN) + __field(enum req_op, op) + __field(sector_t, dst) + __field(sector_t, src) + __field(size_t, len) + ), + TP_fast_assign( + __entry->op = req_op(req); + __assign_disk_name(__entry->disk, req->q->disk); + __entry->dst = dst; + __entry->src = src; + __entry->len = len; + ), + TP_printk("%s req=%-15s: dst=%llu, src=%llu, len=%lu", + __print_disk_name(__entry->disk), + blk_op_str(__entry->op), + __entry->dst, __entry->src, __entry->len) +); #endif /* _TRACE_NULLB_H */ #undef TRACE_INCLUDE_PATH