From patchwork Tue Dec 10 19:47:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 13901998 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1429226181 for ; Tue, 10 Dec 2024 19:58:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.145.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733860724; cv=none; b=B30uEekTKOLUIHj+3w33HdFMMjxxDW+n7Zuu2TjMqxqKjHDXv8D3GP9ypT8w5ZngfxxwxO5Vc9QMWAIM8bMbv203bV7BxqZ0zZiDqKd/F9kq7EwhvGeo3c+b/hZAz9154tWe7HZWSZmjYEyT0IeJVcZgezJw8mJtRwRRjCE2UzM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733860724; c=relaxed/simple; bh=9bdeO/gQYhJwlrC6m59FA2mdSVREETfOSiU3VZ800AU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=rBqDcfF2tFAnK8xZSWetgFsDhDgoZXPKrby5WBqZhwyLjTeHPIogWKzv5QJwIuy/5K212D/89DidKh61wfK8qPR0XgBRYg2RxkgkVQpolM0fTlVLlFvJMq5PG+4MOUiC93n3Lwn2YP/uRH7mIcoPADmz5UC/JazpbrWU0wnftkw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=D0xnDSIi; arc=none smtp.client-ip=67.231.145.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="D0xnDSIi" Received: from pps.filterd (m0109333.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4BAEatlV003672 for ; Tue, 10 Dec 2024 11:58:42 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2021-q4; bh=mgFHOgHaE4jw+dGMPOn2vA2NFSP+ToU+o7GNI61mlAU=; b=D0xnDSIiPRiW j93myHXtOI/RtGihPRVrqLwI8ggzoBlhuvUgZLXh0ypnc28LCZGwNAQ96xtYWUQ3 cX2Wxozs4noVcHXr5amhDe4YraaTjUBKosroiirT7sg6A+Ke41DigUl6+svn+dG5 EcESsvySIdKHxPQL4l6WkwIbjBcVZnlUJTx01zgVh/lgHr8WtyqvquSrH7nOJ766 Yw6DzZnzW4rTDh30o83IilcHzi+4rQxwKTyMux7DWgXlWnBVSp/AY7hkziO17MMa 9JKPCvC0vm2sgEQNBsG2uMeIJw+dWpdDPs6B62TWWIbZyQs206v+L4Bo9YXOh8Oe lI7nygUdEw== Received: from mail.thefacebook.com ([163.114.134.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 43eqja2nhv-11 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Tue, 10 Dec 2024 11:58:41 -0800 (PST) Received: from twshared11082.06.ash8.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c08b:78::2ac9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.11; Tue, 10 Dec 2024 19:58:39 +0000 Received: by devbig638.nha1.facebook.com (Postfix, from userid 544533) id 7D8EC15D5C7C7; Tue, 10 Dec 2024 11:48:08 -0800 (PST) From: Keith Busch To: , , , , , CC: , , , , Hannes Reinecke , Keith Busch Subject: [PATCHv13 05/11] block: expose write streams for block device nodes Date: Tue, 10 Dec 2024 11:47:16 -0800 Message-ID: <20241210194722.1905732-6-kbusch@meta.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241210194722.1905732-1-kbusch@meta.com> References: <20241210194722.1905732-1-kbusch@meta.com> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: gZTBX_DlX6pjl3OKlrEVFYdxUweyPg2n X-Proofpoint-GUID: gZTBX_DlX6pjl3OKlrEVFYdxUweyPg2n X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_03,2024-10-04_01,2024-09-30_01 From: Christoph Hellwig Use the per-kiocb write stream if provided, or map temperature hints to write streams (which is a bit questionable, but this shows how it is done). Reviewed-by: Hannes Reinecke Signed-off-by: Christoph Hellwig [kbusch: removed statx reporting] Signed-off-by: Keith Busch Reviewed-by: Nitesh Shetty --- block/fops.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/block/fops.c b/block/fops.c index 6d5c4fc5a2168..f16aa39bf5bad 100644 --- a/block/fops.c +++ b/block/fops.c @@ -73,6 +73,7 @@ static ssize_t __blkdev_direct_IO_simple(struct kiocb *iocb, } bio.bi_iter.bi_sector = pos >> SECTOR_SHIFT; bio.bi_write_hint = file_inode(iocb->ki_filp)->i_write_hint; + bio.bi_write_stream = iocb->ki_write_stream; bio.bi_ioprio = iocb->ki_ioprio; if (iocb->ki_flags & IOCB_ATOMIC) bio.bi_opf |= REQ_ATOMIC; @@ -206,6 +207,7 @@ static ssize_t __blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter, for (;;) { bio->bi_iter.bi_sector = pos >> SECTOR_SHIFT; bio->bi_write_hint = file_inode(iocb->ki_filp)->i_write_hint; + bio->bi_write_stream = iocb->ki_write_stream; bio->bi_private = dio; bio->bi_end_io = blkdev_bio_end_io; bio->bi_ioprio = iocb->ki_ioprio; @@ -333,6 +335,7 @@ static ssize_t __blkdev_direct_IO_async(struct kiocb *iocb, dio->iocb = iocb; bio->bi_iter.bi_sector = pos >> SECTOR_SHIFT; bio->bi_write_hint = file_inode(iocb->ki_filp)->i_write_hint; + bio->bi_write_stream = iocb->ki_write_stream; bio->bi_end_io = blkdev_bio_end_io_async; bio->bi_ioprio = iocb->ki_ioprio; @@ -398,6 +401,26 @@ static ssize_t blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter) if (blkdev_dio_invalid(bdev, iocb, iter)) return -EINVAL; + if (iov_iter_rw(iter) == WRITE) { + u16 max_write_streams = bdev_max_write_streams(bdev); + + if (iocb->ki_write_stream) { + if (iocb->ki_write_stream > max_write_streams) + return -EINVAL; + } else if (max_write_streams) { + enum rw_hint write_hint = + file_inode(iocb->ki_filp)->i_write_hint; + + /* + * Just use the write hint as write stream for block + * device writes. This assumes no file system is + * mounted that would use the streams differently. + */ + if (write_hint <= max_write_streams) + iocb->ki_write_stream = write_hint; + } + } + nr_pages = bio_iov_vecs_to_alloc(iter, BIO_MAX_VECS + 1); if (likely(nr_pages <= BIO_MAX_VECS)) { if (is_sync_kiocb(iocb)) From patchwork Tue Dec 10 19:47:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 13902003 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 700CC23ED44 for ; Tue, 10 Dec 2024 20:26:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.145.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733862363; cv=none; b=deEFRdLSvw6Sq0AMmynZ+npiWW9xLfHGzgRw5dqm5HQVt87i5j76f/flDyk/wVHJpTUy7mD0zMbxHDa1rwmzPCxh3YoHvn8DxWDXU5ytHOLY/gbwmihv1M0biB6hW0S69SpkRA8E3B9WU16OY88bpJwDLqkyUQiI1ZfxM5oENe8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733862363; c=relaxed/simple; bh=E9ZNDSxL0eVvcu/oD61YwID/6ZeGsMTPAFATVdwy/F4=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=u0WupMAjrkdTDRxTBzfEE9gNC0mkVx59SvdZmI0GOdSKHJHRVrFimy0/3BJe4GweXGvxjBSbSdxQYFgOqd6qGppV4+Xli+2gxIAcrl0arstktmihidd+JkmUR5wF/liapwvvlUsYrvpuPGmJPrGYrmLbTZUo0UAEYLkl8xrlyVo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=eWP60xGX; arc=none smtp.client-ip=67.231.145.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="eWP60xGX" Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4BAHvRp8030100 for ; Tue, 10 Dec 2024 12:26:01 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2021-q4; bh=924Fq1a7MqtuYjIm12r9i7+BviK2AxhmOrAIwHZ2dfo=; b=eWP60xGXAGym dxeGptt8caqa2+Ns4mCDH5t9b0UWGwXUeYZOPjoXvsISKt2yJRL10YPKaiyi33l4 X2MwC37yktGh/iGjqkHQ0Hu6f3xZPRY3bdmrN3/PT9/QLN/gQMdAbxT5zyDy2ytF YKlZW+E3UkNIRe4w13kTsGF788lzTtzw+NEoVxWVHceX1naVDm0OARGz7DZCgwMR ZR/6mpyA1eNv68gXf/FlkoQxJHjwaHOcpcTPVxMDsc3nc9QDpOq7rYCGy1SEXAyc zH8m7GMtQKBQ4oOExjoAZej+NT83xJo/xgjwmsBMe/ANTho9/RPtqA27GuEUndM0 nGTZZ8OSdQ== Received: from maileast.thefacebook.com ([163.114.135.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 43etg996xk-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Tue, 10 Dec 2024 12:26:01 -0800 (PST) Received: from twshared9216.15.frc2.facebook.com (2620:10d:c0a8:1b::30) by mail.thefacebook.com (2620:10d:c0a9:6f::237c) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.11; Tue, 10 Dec 2024 20:25:39 +0000 Received: by devbig638.nha1.facebook.com (Postfix, from userid 544533) id 8DB2115D5C7CA; Tue, 10 Dec 2024 11:48:08 -0800 (PST) From: Keith Busch To: , , , , , CC: , , , , Keith Busch , Hannes Reinecke Subject: [PATCHv13 06/11] io_uring: enable per-io write streams Date: Tue, 10 Dec 2024 11:47:17 -0800 Message-ID: <20241210194722.1905732-7-kbusch@meta.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241210194722.1905732-1-kbusch@meta.com> References: <20241210194722.1905732-1-kbusch@meta.com> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-ORIG-GUID: Z_cBBjgjMN-qTeMOCi8ruPb1tDfNiCgN X-Proofpoint-GUID: Z_cBBjgjMN-qTeMOCi8ruPb1tDfNiCgN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_03,2024-10-04_01,2024-09-30_01 From: Keith Busch Allow userspace to pass a per-I/O write stream in the SQE: __u8 write_stream; The __u8 type matches the size the filesystems and block layer support. Application can query the supported values from the statx max_write_streams field. Unsupported values are ignored by file operations that do not support write streams or rejected with an error by those that support them. Reviewed-by: Hannes Reinecke Signed-off-by: Keith Busch Reviewed-by: Nitesh Shetty --- include/uapi/linux/io_uring.h | 4 ++++ io_uring/io_uring.c | 2 ++ io_uring/rw.c | 1 + 3 files changed, 7 insertions(+) diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h index 38f0d6b10eaf7..986a480e3b9c2 100644 --- a/include/uapi/linux/io_uring.h +++ b/include/uapi/linux/io_uring.h @@ -92,6 +92,10 @@ struct io_uring_sqe { __u16 addr_len; __u16 __pad3[1]; }; + struct { + __u8 write_stream; + __u8 __pad4[3]; + }; }; union { struct { diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index ae36aa702f463..b561c5e8879ac 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -3869,6 +3869,8 @@ static int __init io_uring_init(void) BUILD_BUG_SQE_ELEM(44, __s32, splice_fd_in); BUILD_BUG_SQE_ELEM(44, __u32, file_index); BUILD_BUG_SQE_ELEM(44, __u16, addr_len); + BUILD_BUG_SQE_ELEM(44, __u8, write_stream); + BUILD_BUG_SQE_ELEM(45, __u8, __pad4[0]); BUILD_BUG_SQE_ELEM(46, __u16, __pad3[0]); BUILD_BUG_SQE_ELEM(48, __u64, addr3); BUILD_BUG_SQE_ELEM_SIZE(48, 0, cmd); diff --git a/io_uring/rw.c b/io_uring/rw.c index 5b24fd8b69f62..416ccd89a77ed 100644 --- a/io_uring/rw.c +++ b/io_uring/rw.c @@ -316,6 +316,7 @@ static int io_prep_rw(struct io_kiocb *req, const struct io_uring_sqe *sqe, } rw->kiocb.dio_complete = NULL; rw->kiocb.ki_flags = 0; + rw->kiocb.ki_write_stream = READ_ONCE(sqe->write_stream); rw->addr = READ_ONCE(sqe->addr); rw->len = READ_ONCE(sqe->len); From patchwork Tue Dec 10 19:47:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 13902001 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E06B921422E for ; Tue, 10 Dec 2024 20:20:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=67.231.145.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733862045; cv=none; b=NXg4Xx1jBztuaHH7/hGA0Rb2+6YhBtW2in6qfvq6KdyjBB1R6opZxfmCXzyz8rJRI+Yf2GR5j1l9QtroUKwcBslB1bwxom9Ltj0wahU7uE1awedI5P9keo37lItjguJ/81aKrXcyHaXFRgaZdVLjVawk2xsJ/kdy3pcnbBCBDDM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733862045; c=relaxed/simple; bh=XSYbfQssCKJmZyH36P/jxlzPZAATgx4U8L30BcYUU2k=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=eZko7GHL3ywTAKRX9EOhMWawFH3WyD54VeOT2oBwkqIijiy5Wzz1wfW35bRtICJIvU1ropDU+On7RjoaUGlpxfSFfui/Jvd6IV5QtSrzIx2VFJXVhKy0nYpS/jPaHuWMbb3G1kD/GeUSvYqVMHGosYAsNkp5Bap13w/AXiL5Xfo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com; spf=pass smtp.mailfrom=meta.com; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b=a9wj6GLM; arc=none smtp.client-ip=67.231.145.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=meta.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=meta.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=meta.com header.i=@meta.com header.b="a9wj6GLM" Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4BAIaN5F018802 for ; Tue, 10 Dec 2024 12:20:43 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=meta.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=s2048-2021-q4; bh=OeAFDFh9nDrzvjn6Ci3E1x+se/W+HHmUUWFaWlwZTnk=; b=a9wj6GLMA11F WEUqOQTu37AEgKkKLivQe+NgFDwUvbubrwBFIl+6dXGMVweiglEyUfqQkQ3yW+Nf mrcjbDvcFwKjBxjd1fL0hn3sGicoowq/3f+UAocB8dWBK+J3XTeDmRbKeSVeVsN5 DefdKEB0NZNDlyQ9DPjHPHTVWmBmSyHFRbG0HIsjFy6McwPqqdzhVHh+omJ1O3Ic EfEtPwRq+zYwdYCMUqoxE7qdWxoIH8YCNQDiwVmE+FzYIda8WqTXExtnMjms01E/ TABJev7V3INYhRrVjdh9l5idp6eJXiG8uw/0VgUTeKPN+p3WBosm7NMfI+mkhVXT EWxMMC9A5Q== Received: from maileast.thefacebook.com ([163.114.135.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 43eu2s8swg-5 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Tue, 10 Dec 2024 12:20:42 -0800 (PST) Received: from twshared40462.17.frc2.facebook.com (2620:10d:c0a8:1b::8e35) by mail.thefacebook.com (2620:10d:c0a9:6f::8fd4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.2.1544.11; Tue, 10 Dec 2024 20:20:37 +0000 Received: by devbig638.nha1.facebook.com (Postfix, from userid 544533) id D0DAD15D5C7D4; Tue, 10 Dec 2024 11:48:08 -0800 (PST) From: Keith Busch To: , , , , , CC: , , , , Keith Busch Subject: [PATCHv13 10/11] nvme: register fdp parameters with the block layer Date: Tue, 10 Dec 2024 11:47:21 -0800 Message-ID: <20241210194722.1905732-11-kbusch@meta.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241210194722.1905732-1-kbusch@meta.com> References: <20241210194722.1905732-1-kbusch@meta.com> Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-GUID: jLCfYIEGcwCUa-oyS2tIepM-flnYsHmJ X-Proofpoint-ORIG-GUID: jLCfYIEGcwCUa-oyS2tIepM-flnYsHmJ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-05_03,2024-10-04_01,2024-09-30_01 From: Keith Busch Register the device data placement limits if supported. This is just registering the limits with the block layer. Nothing beyond reporting these attributes is happening in this patch. Merges parts from a patch by Christoph Hellwig Link: https://lore.kernel.org/linux-nvme/20241119121632.1225556-15-hch@lst.de/ Signed-off-by: Keith Busch Reviewed-by: Nitesh Shetty --- drivers/nvme/host/core.c | 139 +++++++++++++++++++++++++++++++++++++++ drivers/nvme/host/nvme.h | 2 + 2 files changed, 141 insertions(+) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index c2a3585a3fa59..f7aeda601fcd6 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -38,6 +38,8 @@ struct nvme_ns_info { u32 nsid; __le32 anagrpid; u8 pi_offset; + u16 endgid; + u64 runs; bool is_shared; bool is_readonly; bool is_ready; @@ -1613,6 +1615,7 @@ static int nvme_ns_info_from_identify(struct nvme_ctrl *ctrl, info->is_shared = id->nmic & NVME_NS_NMIC_SHARED; info->is_readonly = id->nsattr & NVME_NS_ATTR_RO; info->is_ready = true; + info->endgid = le16_to_cpu(id->endgid); if (ctrl->quirks & NVME_QUIRK_BOGUS_NID) { dev_info(ctrl->device, "Ignoring bogus Namespace Identifiers\n"); @@ -1653,6 +1656,7 @@ static int nvme_ns_info_from_id_cs_indep(struct nvme_ctrl *ctrl, info->is_ready = id->nstat & NVME_NSTAT_NRDY; info->is_rotational = id->nsfeat & NVME_NS_ROTATIONAL; info->no_vwc = id->nsfeat & NVME_NS_VWC_NOT_PRESENT; + info->endgid = le16_to_cpu(id->endgid); } kfree(id); return ret; @@ -2147,6 +2151,127 @@ static int nvme_update_ns_info_generic(struct nvme_ns *ns, return ret; } +static int nvme_query_fdp_granularity(struct nvme_ctrl *ctrl, + struct nvme_ns_info *info, u8 fdp_idx) +{ + struct nvme_fdp_config_log hdr, *h; + struct nvme_fdp_config_desc *desc; + size_t size = sizeof(hdr); + int i, n, ret; + void *log; + + ret = nvme_get_log_lsi(ctrl, 0, NVME_LOG_FDP_CONFIGS, 0, + NVME_CSI_NVM, &hdr, size, 0, info->endgid); + if (ret) { + dev_warn(ctrl->device, + "FDP configs log header status:0x%x endgid:%x\n", ret, + info->endgid); + return ret; + } + + size = le32_to_cpu(hdr.sze); + h = kzalloc(size, GFP_KERNEL); + if (!h) { + dev_warn(ctrl->device, + "failed to allocate %lu bytes for FDP config log\n", + size); + return -ENOMEM; + } + + ret = nvme_get_log_lsi(ctrl, 0, NVME_LOG_FDP_CONFIGS, 0, + NVME_CSI_NVM, h, size, 0, info->endgid); + if (ret) { + dev_warn(ctrl->device, + "FDP configs log status:0x%x endgid:%x\n", ret, + info->endgid); + goto out; + } + + n = le16_to_cpu(h->numfdpc) + 1; + if (fdp_idx > n) { + dev_warn(ctrl->device, "FDP index:%d out of range:%d\n", + fdp_idx, n); + /* Proceed without registering FDP streams */ + ret = 0; + goto out; + } + + log = h + 1; + desc = log; + for (i = 0; i < fdp_idx; i++) { + log += le16_to_cpu(desc->dsze); + desc = log; + } + + if (le32_to_cpu(desc->nrg) > 1) { + dev_warn(ctrl->device, "FDP NRG > 1 not supported\n"); + ret = 0; + goto out; + } + + info->runs = le64_to_cpu(desc->runs); +out: + kfree(h); + return ret; +} + +static int nvme_query_fdp_info(struct nvme_ns *ns, struct nvme_ns_info *info) +{ + struct nvme_ns_head *head = ns->head; + struct nvme_ctrl *ctrl = ns->ctrl; + struct nvme_fdp_ruh_status *ruhs; + struct nvme_fdp_config fdp; + struct nvme_command c = {}; + size_t size; + int ret; + + /* + * The FDP configuration is static for the lifetime of the namespace, + * so return immediately if we've already registered this namespaces's + * streams. + */ + if (head->nr_plids) + return 0; + + ret = nvme_get_features(ctrl, NVME_FEAT_FDP, info->endgid, NULL, 0, + &fdp); + if (ret) { + dev_warn(ctrl->device, "FDP get feature status:0x%x\n", ret); + return ret; + } + + if (!(fdp.flags & FDPCFG_FDPE)) + return 0; + + ret = nvme_query_fdp_granularity(ctrl, info, fdp.fdpcidx); + if (!info->runs) + return ret; + + size = struct_size(ruhs, ruhsd, S8_MAX - 1); + ruhs = kzalloc(size, GFP_KERNEL); + if (!ruhs) { + dev_warn(ctrl->device, + "failed to allocate %lu bytes for FDP io-mgmt\n", + size); + return -ENOMEM; + } + + c.imr.opcode = nvme_cmd_io_mgmt_recv; + c.imr.nsid = cpu_to_le32(head->ns_id); + c.imr.mo = NVME_IO_MGMT_RECV_MO_RUHS; + c.imr.numd = cpu_to_le32(nvme_bytes_to_numd(size)); + ret = nvme_submit_sync_cmd(ns->queue, &c, ruhs, size); + if (ret) { + dev_warn(ctrl->device, "FDP io-mgmt status:0x%x\n", ret); + goto free; + } + + head->nr_plids = le16_to_cpu(ruhs->nruhsd); +free: + kfree(ruhs); + return ret; +} + static int nvme_update_ns_info_block(struct nvme_ns *ns, struct nvme_ns_info *info) { @@ -2183,6 +2308,12 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns, goto out; } + if (ns->ctrl->ctratt & NVME_CTRL_ATTR_FDPS) { + ret = nvme_query_fdp_info(ns, info); + if (ret < 0) + goto out; + } + blk_mq_freeze_queue(ns->disk->queue); ns->head->lba_shift = id->lbaf[lbaf].ds; ns->head->nuse = le64_to_cpu(id->nuse); @@ -2216,6 +2347,12 @@ static int nvme_update_ns_info_block(struct nvme_ns *ns, if (!nvme_init_integrity(ns->head, &lim, info)) capacity = 0; + lim.max_write_streams = ns->head->nr_plids; + if (lim.max_write_streams) + lim.write_stream_granularity = max(info->runs, U32_MAX); + else + lim.write_stream_granularity = 0; + ret = queue_limits_commit_update(ns->disk->queue, &lim); if (ret) { blk_mq_unfreeze_queue(ns->disk->queue); @@ -2318,6 +2455,8 @@ static int nvme_update_ns_info(struct nvme_ns *ns, struct nvme_ns_info *info) ns->head->disk->flags |= GENHD_FL_HIDDEN; else nvme_init_integrity(ns->head, &lim, info); + lim.max_write_streams = ns_lim->max_write_streams; + lim.write_stream_granularity = ns_lim->write_stream_granularity; ret = queue_limits_commit_update(ns->head->disk->queue, &lim); set_capacity_and_notify(ns->head->disk, get_capacity(ns->disk)); diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index c1995d89ffdb8..4b412cd8001f1 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -491,6 +491,8 @@ struct nvme_ns_head { struct device cdev_device; struct gendisk *disk; + + u16 nr_plids; #ifdef CONFIG_NVME_MULTIPATH struct bio_list requeue_list; spinlock_t requeue_lock;